Deployment

Deploying CCME involves two main steps:

  1. Deploying the Management Layer, including the CCME Management Host along with the services and tools common to all clusters

  2. Configuring and deploying clusters

Technically, as soon as the prerequisites are there (KMS keys, S3 buckets, VPC/subnets, Secrets Manager secrets, ACM certificate), the deployment of the “generic/standard” CCME components used to build up a customer’s HPC platform on AWS is a process that can be divided into the following 3 parts (the first two parts being included in the Management Layer step mentioned above):

  1. CCME Roles Stack (CRS) deployment, which takes approximately 5 minutes.

  2. CCME Management stack (including the ALB and the CCME Management Host, a.k.a. CMH) deployment, which takes approximately 15 minutes.

  3. First cluster creation (using the pcluster create-cluster command on the CMH), which takes approximately 30 to 45 minutes.

Note

The customer-specific configurations and components added to the HPC platform besides the CCME components can trigger some additional deployment time.

The following diagram details an example of a deployment sequence, corresponding to a simple generic customer environment.

        %%{init: {"theme": "base", "treeView": {"showIcons": true}, "themeVariables": {"treeView": {"lineColor": "#2576b7"}}}}%%
treeView-beta
    "1. Create (or check the existence of) the required resources in AWS/" :::highlight
        "1.1 AWS Account"
        "1.2 Network foundations (VPC, subnets)"
        "1.3 Additional network requirements: VPC Endpoints"
        "1.4 KMS keys"
        "1.5 Secrets"
        "1.6 S3 buckets"
        "1.7 SNS topic"
        "1.8 SSL Certificate"
    "2. Get CCME from UCit repository/" :::highlight
        "2.1 Download CCME artifacts (code + SHA256)"
        "2.2 Check SHA256 of CCME zip file"
    "3. Create initial setup environment/" :::highlight
        "3.1 Create an initial setup EC2 instance in the targeted VPC"
        "3.2 Assign a privileged IAM instance profile to the EC2 instance"
        "3.3 Configure the EC2 instance/"
            "3.3.1 Upload CCME in /tmp/ccme"
            "3.3.2 Edit a copy of roles.ccme.conf"
            "3.3.3 Deploy Roles Stack with deployCCME.sh/"
                "IAM Roles"
                "IAM Policies"
            "3.3.4 Build a copy of deployment.ccme.conf with CRS outputs/"
                "Run deployCCME.sh (-s + -cm)"
                "Edit and complete manually"
            "3.3.5 Deploy Management Stack with deployCCME.sh/"
                "Security Groups"
                "DCV Proxy ASG"
                "DCV Proxy Instance"
                "Lambda functions"
                "SNS topics"
                "CCME Management Host (CMH)"
        "3.4 Delete the initial setup EC2 instance"
    "4. Manage clusters lifecycle using the CCME Management Host/" :::highlight
        "4.1 Check CMH logs to ensure all configurations are applied correctly/"
            "4.1.1 In case of UPDATE, run manually the cmh-init.sh script"
        "4.2 Check clusters configuration file /home/ec2-user/.parallelcluster/pcluster.config"
        "4.3 Manage the clusters lifecycle using AWS ParallelCluster CLI/"
            "4.3.1 Cluster creation"
            "4.3.2 Cluster update"
            "4.3.3 Cluster deletion"
    "5. Apply additional configurations/" :::highlight
        "5.1 In EF Portal/"
            "5.1.1 Import users & groups definitions from previously exported file (if any)"
            "5.1.2 Publish existing batch EF Portal services to users using the predefined groups"
            "5.1.3 Publish existing VDI EF Portal services to users using the predefined groups"
        "5.2 Other configurations or custom playbooks?"
    "6. Run the tests & acceptance plan" :::highlight

Please refer to the subsections below describing the main steps and also the details of an update to the latest major version.