AWSTemplateFormatVersion: '2010-09-09' Description: > MassEngine Vector Database — 1-Click AWS Marketplace Cluster Deployment. Supports both the 1-Core Free Tier (1 vCPU standalone node, up to 1M vectors) and Production Multi-Node HA Clusters (RF=2 NVMe-tiered) with automated AWS Marketplace per-vCPU hourly metering. Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: "1. License Tier & Cluster Topology" Parameters: - DeploymentTier - NodeCount - InstanceType - StorageMode - Label: default: "2. Networking & Security" Parameters: - VpcId - SubnetIds - AllowedClientCidr - Label: default: "3. Optional Offline / Custom License Key" Parameters: - LicenseKey - MarketplaceAmiId Parameters: DeploymentTier: Type: String Default: Production-HA-Cluster AllowedValues: - 1-Core-Free-Single-Node - Production-HA-Cluster Description: > Select '1-Core-Free-Single-Node' for $0/mo testing (1 vCPU, 1 node, up to 1M vectors), or 'Production-HA-Cluster' for multi-node HA metered hourly per vCPU via AWS Marketplace. NodeCount: Type: Number Default: 2 AllowedValues: [1, 2, 4, 8, 16, 32, 64] Description: > Number of MassEngine cluster nodes (must be 1 for 1-Core-Free-Single-Node; 2 or more recommended for RF=2 High Availability). InstanceType: Type: String Default: i4i.large AllowedValues: - t3.small - c7i.large - i4i.large - i4i.xlarge - i4i.2xlarge - i4i.4xlarge - i4i.8xlarge - i4i.16xlarge Description: > EC2 instance type. Each active vCPU includes 2 TB of NVMe storage allowance. Use t3.small (pinned to 1 vCPU) for Free Tier testing or i4i.* for local NVMe SSD production. StorageMode: Type: String Default: nvme-mode AllowedValues: - nvme-mode - in-memory Description: > nvme-mode stores compressed vectors on local NVMe SSD with hot index caching in RAM; in-memory keeps all vectors in RAM for ultra-low latency. VpcId: Type: AWS::EC2::VPC::Id Description: VPC where the MassEngine cluster will be deployed. SubnetIds: Type: List Description: Subnets for cluster nodes (select at least 2 Availability Zones for HA). AllowedClientCidr: Type: String Default: 10.0.0.0/8 Description: CIDR block allowed to access the MassEngine REST API on port 8080. LicenseKey: Type: String Default: "" NoEcho: true Description: > Optional signed per-core license token (meli_...). Leave blank when using 1-Core Free Tier or automatic AWS Marketplace Metering. MarketplaceAmiId: Type: AWS::SSM::Parameter::Value Default: /aws/service/canonical/ubuntu/server/24.04/stable/current/amd64/hvm/ebs-gp3/ami-id Description: AMI ID (defaults to Ubuntu 24.04 LTS HVM; replaced by Marketplace AMI ID in AWS Marketplace listing). Conditions: IsFreeTier: !Equals [!Ref DeploymentTier, "1-Core-Free-Single-Node"] HasSignedKey: !Not [!Equals [!Ref LicenseKey, ""]] Resources: ClusterAuthSecret: Type: AWS::SecretsManager::Secret Properties: Name: !Sub "${AWS::StackName}-massengine-cluster-secrets" Description: Auto-generated HMAC cluster secret and initial admin API token for MassEngine GenerateSecretString: SecretStringTemplate: '{"admin_role":"cluster-admin"}' GenerateStringKey: "secret_token" PasswordLength: 48 ExcludePunctuation: true ClusterSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: MassEngine Cluster Security Group (API 8080 + internal peer replication 9410) VpcId: !Ref VpcId SecurityGroupIngress: - Description: MassEngine HTTP/REST API IpProtocol: tcp FromPort: 8080 ToPort: 8080 CidrIp: !Ref AllowedClientCidr ClusterInternalPeerIngress: Type: AWS::EC2::SecurityGroupIngress Properties: GroupId: !Ref ClusterSecurityGroup IpProtocol: tcp FromPort: 9410 ToPort: 9410 SourceSecurityGroupId: !Ref ClusterSecurityGroup Description: Internal inter-node shard replication and gossip (strictly VPC-internal) MarketplaceMeteringRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: ec2.amazonaws.com Action: sts:AssumeRole Policies: - PolicyName: MassEngineMarketplacePerCoreMetering PolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Action: - aws-marketplace:MeterUsage - aws-marketplace:RegisterUsage - aws-marketplace:ResolveCustomer Resource: "*" - Effect: Allow Action: - secretsmanager:GetSecretValue Resource: !Ref ClusterAuthSecret - Effect: Allow Action: - ec2:DescribeInstances Resource: "*" MarketplaceInstanceProfile: Type: AWS::IAM::InstanceProfile Properties: Roles: - !Ref MarketplaceMeteringRole ClusterLaunchTemplate: Type: AWS::EC2::LaunchTemplate Properties: LaunchTemplateName: !Sub "${AWS::StackName}-lt" LaunchTemplateData: ImageId: !Ref MarketplaceAmiId InstanceType: !Ref InstanceType IamInstanceProfile: Arn: !GetAtt MarketplaceInstanceProfile.Arn MetadataOptions: HttpEndpoint: enabled HttpTokens: required HttpPutResponseHopLimit: 1 SecurityGroupIds: - !Ref ClusterSecurityGroup BlockDeviceMappings: - DeviceName: /dev/sda1 Ebs: VolumeSize: 64 VolumeType: gp3 Encrypted: true UserData: Fn::Base64: !Sub - | #!/usr/bin/env bash set -euo pipefail export MASSENGINE_LICENSE_MODE="${ResolvedLicenseMode}" export MASSENGINE_MAX_CORES="${ResolvedMaxCores}" export MASSENGINE_NODE_COUNT="${ResolvedNodeCount}" export MASSENGINE_STORAGE_MODE="${StorageMode}" export MASSENGINE_SECRET_ARN="${ClusterAuthSecret}" export MASSENGINE_LICENSE_KEY="${LicenseKey}" export AWS_REGION="${AWS::Region}" mkdir -p /var/lib/massengine /etc/massengine if [[ -b /dev/nvme1n1 ]]; then mkfs.ext4 -F /dev/nvme1n1 || true mount -o noatime /dev/nvme1n1 /var/lib/massengine || true fi if [[ -x /usr/local/bin/massengine-node ]]; then systemctl enable --now massengine-node fi - ResolvedLicenseMode: !If [IsFreeTier, "free", !If [HasSignedKey, "signed-key", "aws-marketplace"]] ResolvedMaxCores: !If [IsFreeTier, "1", "0"] ResolvedNodeCount: !If [IsFreeTier, "1", !Ref NodeCount] ClusterAutoScalingGroup: Type: AWS::AutoScaling::AutoScalingGroup Properties: VPCZoneIdentifier: !Ref SubnetIds LaunchTemplate: LaunchTemplateId: !Ref ClusterLaunchTemplate Version: !GetAtt ClusterLaunchTemplate.LatestVersionNumber MinSize: !If [IsFreeTier, "1", !Ref NodeCount] MaxSize: !If [IsFreeTier, "1", !Ref NodeCount] DesiredCapacity: !If [IsFreeTier, "1", !Ref NodeCount] Tags: - Key: Name Value: !Sub "${AWS::StackName}-massengine-node" PropagateAtLaunch: true - Key: MassEngineCluster Value: !Ref AWS::StackName PropagateAtLaunch: true - Key: MassEngineLicenseTier Value: !Ref DeploymentTier PropagateAtLaunch: true Outputs: DeploymentTierOutput: Description: Active MassEngine Per-Core License Mode Value: !Ref DeploymentTier AdminSecretArn: Description: AWS Secrets Manager ARN containing the cluster HMAC secret and initial admin API key Value: !Ref ClusterAuthSecret LicenseStatusCommand: Description: Command to verify per-core license & metering status on any node Value: "curl -s http://localhost:8080/v1/license | jq ."