Architecture
Note
There is no single “default” CCME architecture, only illustrative examples. The architecture actually deployed for a given platform is not driven by CCME alone: it is highly dependent on the customer’s project (existing network topology, security and compliance constraints, identity integration, storage and workload requirements, high availability needs…). The diagram and description below present one representative example, not a mandatory or unique topology.
Example architecture
The following diagram illustrates one example of a CCME-based HPC platform deployed on AWS, in the context of a customer project. It shows the customer AWS account boundary and target Region, the VPC and its Availability Zones, public and private subnets, the CCME components, the shared storage, the AWS managed services used, and the integration points with systems outside of the VPC (on-premises users and administrators, corporate identity, license servers, and DNS).
Network
A CCME platform is deployed in a single target AWS Region, in a single VPC.
Private “FrontEnd” subnets host the Application Load Balancer (multi-AZ) that publishes the NI-SP EF Portal and the login nodes to its users.
Private “BackEnd” subnets host the CCME Management Host (CMH), the cluster Head Node(s), the login nodes, the compute and visualization fleets, and the regional AWS service endpoints.
For performance reasons, the compute nodes of a given Slurm partition are placed in a single Availability Zone, inside a Cluster Placement Group (the target AZ may vary with the availability of the requested instance types). The front-end/load-balancing layer, and shared storage where applicable, can be deployed across multiple Availability Zones.
An AWS Transit Gateway attachment, and optionally AWS Direct Connect or an AWS Site-to-Site VPN, connect the VPC to the customer’s existing network and on-premises systems.
Management and cluster layers
The CCME Management Host (CMH) hosts the services and tools common to all clusters (license management, notifications, optional OKA add-on) and the DCV Proxy and login nodes used to access the clusters.
Each cluster is built around a Head Node (Slurm controller, SlurmDB accounting database, cluster-wide shared configuration), one or more Slurm partitions of compute nodes for HPC/AI workloads, and, optionally, dedicated visualization node partitions (Amazon DCV, with or without GPU).
Several independent clusters (for example a “production” and a “non-production” cluster, as illustrated above) can coexist in the same account and VPC, each with its own Head Node, Slurm partitions and storage mounts.
The CCME Ingress Controller (Amazon EventBridge and AWS Lambda) reacts to platform events (e.g., cluster and node lifecycle) to trigger automated actions.
AWS managed services
A CCME platform combines a broad set of AWS managed services, including (non-exhaustive, see Requirements for the authoritative list): Amazon VPC, Amazon EC2 (and EC2 Auto Scaling, EC2 Image Builder), Elastic Load Balancing, AWS Certificate Manager, AWS CloudFormation, AWS IAM, AWS KMS, AWS Secrets Manager, Amazon Route 53, Amazon S3, Amazon EFS, Amazon FSx, Amazon CloudWatch, Amazon EventBridge, Amazon SNS, Amazon DynamoDB, AWS STS, AWS Lambda, AWS Resource Groups Tagging, AWS Backup, AWS Budgets, AWS Cost Explorer, AWS Cost and Usage Report, and, optionally, AWS WAF & Shield.
All diagrams in this documentation use the official AWS Architecture Icons to identify AWS services.
Integration points
A CCME platform typically integrates with systems outside of the VPC:
Users and administrators connecting from on-premises, over SSH, HTTPS or AWS Systems Manager.
Corporate identity: a customer self-managed Active Directory, a CCME-managed Active Directory or OpenLDAP, and/or single sign-on (SSO) through an external OIDC identity provider. See Authentication.
DNS: a hosted zone under the customer’s own domain, pointing to the Application Load Balancer.
On-premises or hybrid license servers (e.g., FlexLM), integrated as Slurm remote licenses. See License Management.
Third-party software running inside the platform: NI-SP EF Portal, Amazon DCV, Slurm.
Key architectural building blocks
Independently of any specific example, every CCME-based platform is built from the same generic building blocks:
a network layer: one VPC, one or more Availability Zones, public and private subnets, and connectivity to the customer’s existing network;
a management layer: the CCME Management Host and the services shared by all clusters;
one or more cluster layers: a Head Node, Slurm partitions for compute and, optionally, for visualization, and cluster-wide shared configuration;
a shared storage layer: Amazon EFS, Amazon FSx for Lustre and/or Amazon FSx for NetApp ONTAP;
an identity and access layer: directory services, SSO, IAM roles;
an ingress and automation layer: load balancing, and event-driven automation through the CCME Ingress Controller;
an observability and cost layer: logging, monitoring, notifications, budgets and cost reporting;
a set of integration points with systems outside of the platform.
No default architecture
As stated above, CCME does not impose a single, default architecture. The combination, sizing and configuration of the building blocks listed above is adapted, project by project, to the customer’s own constraints: existing network and security posture, identity provider, compliance requirements, target workloads, storage performance needs, availability requirements, and budget. The example shown in this section is only one possible outcome of these choices, used here to illustrate the generic concepts.