Management Host

Prerequisites

CCME deployment can be done with the basic CCME automation configuration or manually. Even with the automated deployment, you still have the possibility to manually modify the CCME and AWS ParallelCluster configuration of your clusters.

The CCME_x.y.z directory contains the following files and directories:

deployCCME.sh: the script to start the Management Host

roles.ccme.conf: the configuration file for the CCME Roles Stack (CRS)

deployment.ccme.conf: the configuration file for the CCME Management Host (CMH)

dependencies.yaml: the version of the dependencies used in CCME

CCME/: the scripts to configure a CCME cluster

management/: the scripts to deploy the Management Host and associated services

doc/: this documentation

You will first need to untar the CCME_x.y.z.tar.gz archive to get access to the code on your local machine (which needs to have access to AWS and have the AWS CLI installed and configured):

mkdir "CCME_x.y.z"
tar xvf "CCME_x.y.z.tar.gz" -C "CCME_x.y.z"
cd "CCME_x.y.z"

`deployCCME.sh`

This is the script that you need to use to deploy the initial stacks for the CCME roles, the Management Host and the associated services.

$ ../../deployCCME.sh --help
Usage: deployCCME.sh [-h|--help] [-r|--create-roles-stack] [-m|--create-management-stack] [-cr|--conf-roles <crs_conf_file>] [-cm|--conf-management <cmh_conf_file>] [-n|--name <stack_name>] [-u|--update] [-nr|--no-rollback] [-p|--profile <aws_profile>]

deployCCME can be used to update the CCME bucket set in <conf_file> and/or to deploy a CCME Management Host

Options :
  -r|--create-roles-stack                           Create CCME Roles Stack (CRS) with a name built with the prefix 'CRS-' and the selected -n|--name option
  -m|--create-management-host                       Create CCME Management Host (CMH) with a name built with the the prefix 'CMH-' and the -n|--name option
  -ur|--update-roles-stack                          Update the deployed CCME Roles Stack (CRS) named with the -n|--name option
  -um|--update-management-host                      Update the deployed CCME Management Host (CMH) named with the -n|--name option
  -cr|--conf-roles              <crs_conf_file>     Reads a configuration file for the CCME Roles Stack (CRS)
  -cm|--conf-management         <cmh_conf_file>     Reads a configuration file for the CCME Management Host (CMH) and update action
  -n|--name                     <stack_name>        Defines the name of the stack deployed by the script (default is CCME)
  -u|--update                                       Updates the CCME bucket
  -nr|--no-rollback                                 Disables rollback of the CMH creation stack in CloudFormation when the stack fails
  -p|--profile                  <aws_profile>       Use non-default AWS credentials profile with aws commmand
  -h/--help                                         Help

CCME Roles Stack (CRS)

CRS Configuration

The configuration of the “CCME Roles Stack” is described in the roles.ccme.conf file:

#### CONFIGURATION FILE for the CCME Roles Stack (CRS) automatic setup ####

#### IAM Configuration
# iam_path_prefix defines the prefix of the IAM roles created by the stack
# The expected value is a string prefix like "ccme" or "parallelcluster".
# Setting this variable is OPTIONAL.
# Default: parallelcluster
# iam_path_prefix=

# enable_route53_access defines the possibility of Route53 creation and usage
# for each cluster.
# Setting this variable is OPTIONAL.
# Value: true or false
# Default: true
# enable_route53_access=


# enable_fsx_s3_access defines the possibility of FSx to access to s3 buckets
# Setting this variable is OPTIONAL.
# Value: true or false
# Default: false
# enable_fsx_s3_access=


# fsx_s3_buckets defines the prefix of the IAM roles created by the stack
# The expected value in the ARN of an IAM role. Check the documentation for more information about this role.
# Setting this variable is OPTIONAL.
# Default: parallelcluster
# fsx_s3_buckets=

# ccme_bucket_path is the BucketName:BucketKey where BucketName is the name of an existing s3 bucket
# and where CCME machinery will be stored at the "BucketKey" path to be used to manage clusters.
# E.g.: "ccme-bucket:subfolder1/subfolder2/subfolder3/"
# Subfolder must end with a '/'
# Setting this variable is MANDATORY. No default value exists.
ccme_bucket_path=

# ccme_data_bucket is the Name of an existing s3 bucket where common user data is accessed by users on any cluster.
# Setting this variable is MANDATORY.
# No default value exists.
ccme_data_bucket=

# ccme_efs is the ARN of an existing efs
# Setting this variable is OPTIONAL.
# No default value exists.
# E.g.: ccme_efs=arn:aws:elasticfilesystem:eu-west-1:012345678910:file-system/fs-0f114ed124c21fec3
# ccme_efs=

# ccme_admin_sns_topic_arn define the admin SNS topic ARN you want to
# deliver information about the cluster when it is ready to be used.
# For example, you can configure your SNS topic to deliver the information
# by email to the administrator of the platform
# Setting this variable is OPTIONAL.
# No default value exists.
# E.g.: arn:aws:sns:REGION:ACCOUNT:TOPICNAME
ccme_admin_sns_topic_arn=

# ccme_secret_prefix defines the prefix of the secret that will be used
# for the ActiveDirectory ReadOnlyUser and optionally administrator.
# Setting this variable is OPTIONAL.
# Default value is "ccme-".
ccme_secret_prefix=

# ccme_cluster_prefix defines the prefix of the future clusters
# Used to restrict rights to CloudWatch dashboards
# Setting this variable is OPTIONAL.
# Default value is "ccme-".
ccme_cluster_prefix=

#### KMS Encryption
# enable_kms_encryption defines the usage of KMS for storage encryption
# When set to true the EBS, FSx, S3 ... will use the encryption key variables
# Setting this variable is OPTIONAL.
# Value: true or false
# Default: false
# enable_kms_encryption=

# The following variables define the ARN of KMS keys used to encrypt storages or
# communications on multiple services used by CCME.
# Setting these variables is mandatory if enable_kms_encryption=true
# Value: ARN of existing KMS key
# No default value exists.
# CMH EBS encryption
# ccme_kms_cmh=
# Clusters EBS encryption
# ccme_kms_clusters=
# FSx encryption
# ccme_kms_fsx=
# S3 encryption
# ccme_kms_s3=
# SNS encryption
# ccme_kms_sns=
# Secrets Manager encryption
# ccme_kms_secrets=

You can modify and use this example configuration file or you can create a new file based on it. This configuration file is then used to deploy the CCME Management Host through the script named deployCCME.sh.

Deployment

The following command will create all the IAM roles needed for CCME and on a CCME Roles Stack.

$ ./deployCCME.sh --create-roles-stack --conf-roles "my-ccme.roles.ccme.conf" --name "my-ccme"
**** CCME Roles Deployment - START ****
{
    "StackId": "arn:aws:cloudformation:eu-west-1:012345678910:stack/CRS-my-ccme/af11b1e0-9a5a-11ed-ab69-06562ac4f907"
}
**** CCME Roles Deployment - END ****

CCME Management Host (CMH)

CMH Configuration

The configuration of the “CCME Management Host” is described in the deployment.ccme.conf file:

#### CONFIGURATION FILE for the CCME Management Host (CMH) automatic setup ####

#### IAM Configuration
# management_stack_role defines the IAM role ARN that will be attached to the Management Stack.
# The expected value in the ARN of an IAM role. Check the documentation for more information about this role.
# This parameter corresponds to the output "ParallelClusterUserRole" of the CCME Roles Stack (CRS)
# Setting this variable is MANDATORY. No default value exists.
# Update Policy: This setting can be changed during an update.
management_stack_role=

# ccme_management_instance_profile defines the CCME Management Host (CMH) AWS IAM Instance Profile ARN
# This parameter corresponds to the output "ParallelClusterUserInstanceProfile" of the CCME Roles Stack (CRS)
# Setting this variable is MANDATORY. No default value exists.
# Update Policy: This setting can be changed during an update.
ccme_management_instance_profile=

# ccme_management_lambda_role defines the ApplicationLoadBalancer Lambda AWS IAM Role ARN
# This parameter corresponds to the output "ccmeAlbLambdaRole" of the CCME Roles Stack (CRS)
# Setting this variable is MANDATORY. No default value exists.
# Update Policy: This setting can be changed during an update.
ccme_management_lambda_role=

# ccme_cluster_lambda_role defines the ParallelCluster Lambda AWS IAM Role ARN
# This parameter corresponds to the output "CustomLambdaResourcesRoleSlurm" of the CCME Roles Stack (CRS)
# Setting this variable is MANDATORY. No default value exists.
# Update Policy: If this setting is changed, the update is not allowed.
# ccme_cluster_lambda_role=

# ccme_cluster_headnode_instance_profile defines the HeadNode AWS IAM Instance Profile ARN
# This parameter corresponds to the output "HeadNodeInstanceProfileSlurm" of the CCME Roles Stack (CRS)
# Setting this variable is MANDATORY. No default value exists.
# Update Policy: If this setting is changed, the update is not allowed.
# ccme_cluster_headnode_instance_profile=

# ccme_cluster_compute_instance_profile defines the Compute Nodes AWS IAM Instance profile ARN
# This parameter corresponds to the output "ComputeNodeInstanceProfileSlurm" of the CCME Roles Stack (CRS)
# Setting this variable is MANDATORY. No default value exists.
# Update Policy: If this setting is changed, the update is not allowed.
# ccme_cluster_compute_instance_profile=

#### Global Configuration
# debug is a boolean variable. When set to true, the stack does not automatically "rollback" after any failure.
# Default value: false
# Update Policy: This setting can be changed during an update.
debug=false

# timezone is a string variable. It configures the timezone for the CMH and the clusters
# Setting this variable is MANDATORY.
# Default value: 'Europe/Paris'
# Update Policy: This setting can be changed during an update and requires to update the deployed clusters with the updated setting.
timezone='Europe/Paris'

# ccme_tags defines a list of tags associated to the CMH stack
# Default value: no tag
# Pattern: ("Key=your_key,Value=your_value" "Key=your_key,Value=your_value")
# tags=("Key=your_key,Value=your_value" "Key=your_key,Value=your_value")
# Update Policy: This setting can be changed during an update and requires to update the deployed clusters with the updated setting.

# management_host_os defines the operating system of the CCME Management Host (CMH)
# Allowed values for this variable are the following:
# - al2023
# - rhel8
# Default value: al2023
# Update Policy: This setting can be changed during an update. WARNING: A new CMH instance will be created.
management_host_os=al2023

# management_host_ami defines the AMI for the CMH instance
# Setting this variable is OPTIONAL.
# Default value: NONE
# Update Policy: This setting can be changed during an update. WARNING: A new CMH instance will be created.
# management_host_ami=

# management_host_instance_type defines the instance type that will be configured as a CMH
# Only AWS EC2 x86_64 instance type is allowed.
# Default value: t2.micro
# Update Policy: This setting can be changed during an update.
management_host_instance_type=t2.micro

#### Network Configuration
# vpc defines the VPC where the CMH should be setup. The expected value is an existing VPCId.
# Setting this variable is MANDATORY. No default value exists.
# Update Policy: If this setting is changed, the update is not allowed.
vpc=

# public_subnets defines the Primary and Secondary subnets (created in 2 different AZs)
# where the Application Load Balancer should be setup.
# The expected values is a pair of existing SubnetId.
# Setting this variable is MANDATORY. No default value exists.
# Update Policy: This setting can be changed during an update and requires to update the deployed clusters with the updated setting.
public_subnets=( "" "" )

# private_subnets defines the Primary and Secondary subnets (created in 2 different AZs)
# where the Active Directory and CCME Management Host should be setup.
# The expected values is a pair of existing SubnetId.
# Setting this variable is MANDATORY. No default value exists.
# Update Policy: If this setting is changed, the update is not allowed.
private_subnets=( "" "" )

# additional_security_group defines one optional additional security group
# to the CMH and clusters.
# Setting this variable is OPTIONAL.
# Default value: NONE
# Update Policy: This setting can be changed during an update and requires to update the deployed clusters with the updated setting.
additional_security_group=NONE

# management_host_ip defines the address of an Elastic IP (if you already have one to use for your CMH)
# If the default value is used, CCME will request a new Elastic IP to attach it to the CMH.
# Set "NONE" to not assign a public IP.
# Default value: 0.0.0.0
# Update Policy: This setting can be changed during an update.
management_host_ip=0.0.0.0


#### Security Configuration
# alb_ssl_certificate defines the SSL certificate used by the Application Load Balancer being the entry point to CCME.
# The expected value in the ARN of a SSL certificate stored in AWS Certificate Manager (ACM).
# Setting this variable is MANDATORY. No default value exists.
# Update Policy: This setting can be changed during an update.
alb_ssl_certificate=

# application_load_balancer_scheme defines if the Application Load Balancer (ALB) is "internet-facing" or "internal"
# Default value: 'internet-facing'
# Update Policy: If this setting is changed, the update is not allowed.
application_load_balancer_scheme=internet-facing

# application_load_balancer_arn defines the application load balancer used to forward the
# portal and the visualization of CCME to the end users.
# The expected value is the ARN of the choosen Application Load Balancer
# Setting this variable is OPTIONAL.
# Default value: NONE
# Update Policy: This setting can be changed during an update and requires to update the deployed clusters with the updated setting.
application_load_balancer_arn=NONE

# application_load_balancer_ingress_cidr specifies the CIDR-formatted IP range for ingress connections to the ALB.
# Setting this variable is OPTIONAL.
# Default value: 0.0.0.0/0
# Update Policy: This setting can be changed during an update.
# application_load_balancer_ingress_cidr=0.0.0.0/0

# secrets_manager_prefix defines the prefix of the secrets stored in the Secrets Service Manager
# CCME requires AWS Secrets, it requires to register each secret with a name starting by the same prefix.
# Default value: NONE
# E.g.: "ccme-prefix"
# E.g. of secret using this prefix: "arn:aws:secretsmanager:eu-west-1:012345678910:secret:ccme-prefix-adreadonlyuser.password-7dFt5"
# Update Policy: This setting can be changed during an update and requires to update the deployed clusters with the updated setting.
secrets_manager_prefix=

# ccme_proxy defines the optional proxy for CMH and clusters
# Setting this variable is OPTIONAL.
# Default value: NONE
# E.g.: "https://url_of_proxy/"
# Update Policy: This setting can be changed during an update and requires to update the deployed clusters with the updated setting.
ccme_proxy=

# ccme_no_proxy defines the optional list of no_proxy delimited by comma for CMH and clusters
# Setting this variable is OPTIONAL.
# Default value: NONE
# E.g.: "my.domain1,my.domain2"
# Update Policy: This setting can be changed during an update and requires to update the deployed clusters with the updated setting.
ccme_no_proxy=

# ccme_repository_pip defines a private pip repository
# Setting this variable is OPTIONAL.
# Default value: NONE
# E.g.: "https://url_of_pip_repo"
# Update Policy: This setting can be changed during an update and requires to update the deployed clusters with the updated setting.
ccme_repository_pip=

# management_host_keypair is the Name of an existing EC2 key pair that should be used to connect to the CMH with SSH.
# Setting this variable is OPTIONAL. No default value exists.
# If you do not specify a key pair or set the value to NONE, you can't connect to the instance
# unless you choose an AMI providing another way to log in.
# Update Policy: If this setting is changed, the update is not allowed.
management_host_keypair=

# management_host_authorized_group defines the AD groups of users which are allowed to login to your CMH.
# If the default value is used, no filter is applied and everyone in AD is authorized to login.
# Default value: NONE
# Update Policy: This setting can be changed during an update.
management_host_authorized_group=NONE

# management_host_sudo_group defines the AD group of users which is allowed to use sudo on your CMH.
# If the default value is used, no group from the AD is authorized to use sudo.
# Default value: NONE
# Update Policy: This setting can be changed during an update.
management_host_sudo_group=NONE


#### Storage Configuration
# ccme_bucket_path is the BucketName:BucketKey where BucketName is the name of an existing s3 bucket
# and where CCME machinery will be stored at the "BucketKey" path to be used to manage clusters.
# E.g.: "ccme-bucket:subfolder1/subfolder2/subfolder3/"
# Subfolder must end with a '/'
# Setting this variable is MANDATORY. No default value exists.
# Update Policy for BucketName: If this setting is changed, the update is not allowed.
# Update Policy for BucketKey: If this setting is changed, the update is allowed for the CMH stack, but not the clusters.
ccme_bucket_path=

# ccme_data_bucket is the Name of an existing s3 bucket where common user data is accessed by users on any cluster.
# Setting this variable is MANDATORY. No default value exists.
# Update Policy: This setting can be changed during an update.
ccme_data_bucket=


#### Active Directory Configuration
# active_directory_name defines the Domain Name of your Active Directory.
# If the value is NONE, then no Active Directory is deployed
# Default value: ccme.ad
# Update Policy: This setting can be changed during an update only if the previous and the new values are set for internal Active Directory.
active_directory_name=

# active_directory_ips defines the list of IPs of the Active Directory.
# When the default (NONE) value is used, a new AWS-managed Active Directory is created and the CMH is configured to use it.
# Otherwise, the format is a comma separated list of IPS. E.g., 10.0.0.1,10.0.0.2
# Default value: NONE
# Update Policy: This setting can be changed during an update and requires to update the deployed clusters with the updated setting.
active_directory_ips=NONE

# active_directory_readonlyuser_dn defines the distinguished name of ReadOnlyUser account in your AD.
# The expected value in the preexisting ARN is a "cn=string,OU=string,DC=string,DC=string".
# Setting this variable differently than 'ReadOnlyUser' default value is MANDATORY ONLY IF the AD used for CCME is preexisting.
# Default value: ReadOnlyUser
# Update Policy: This setting can be changed during an update and requires to update the deployed clusters with the updated setting.
active_directory_read_only_user_dn='ReadOnlyUser'

# active_directory_read_only_user_password_secret_arn defines the password of ReadOnlyUser accounts in your AD.
# The expected value in the preexisting ARN of a plaintext string stored in AWS Secrets Manager (ASM).
# Setting this variable is MANDATORY. No default value exists.
# E.g.: active_directory_read_only_user_password_secret_arn=arn:aws:secretsmanager:eu-west-1:012345678910:secret:ccme-prefix-adadmin.password-3nSOf
# Update Policy: This setting can be changed during an update and requires to update the deployed clusters with the updated setting.
active_directory_read_only_user_password_secret_arn=

# active_directory_admin_password_secret_arn defines the password of Admin accounts in your AD.
# The expected value in the preexisting ARN of a plaintext string stored in AWS Secrets Manager (ASM).
# Setting this variable is MANDATORY ONLY IF the AD creation is automated
# E.g.: active_directory_admin_password_secret_arn=arn:aws:secretsmanager:eu-west-1:012345678910:secret:ccme-prefix-adreadonlyuser.password-7dFt5
# Update Policy: If this setting is changed, the update is not allowed.
active_directory_admin_password_secret_arn=

#### KMS Encryption
# enable_kms_encryption defines the usage of KMS for storage encryption
# When set to true the EBS, FSx, S3 ... will use the encryption key variables
# Setting this variable is OPTIONAL.
# Value: true or false
# Default: false
# Update Policy: If this setting is changed, the update is not allowed.
# enable_kms_encryption=

# The following variables define the ARN of KMS keys used to encrypt storages or
# communications on multiple services used by CCME.
# Setting these variables is mandatory if enable_kms_encryption=true
# Value: ARN of existing KMS key
# No default value exists.
# CMH EBS encryption
# Update Policy: If this setting is changed, the update is not allowed.
# ccme_kms_cmh=
# Clusters EBS encryption
# Update Policy: If this setting is changed, the update is not allowed.
# ccme_kms_clusters=
# Additional KMS key for CMH and Clusters encryption
# ccme_kms_additional_key=
# FSx encryption
# Update Policy: If this setting is changed, the update is not allowed.
# ccme_kms_fsx=
# S3 encryption
# Update Policy: This setting can be changed during an update if the CCME Role Stack allows access to the resource ARN.
# ccme_kms_s3=
# SNS encryption
# Update Policy: This setting can be changed during an update if the CCME Role Stack allows access to the resource ARN.
# ccme_kms_sns=

You can modify and use this example configuration file or you can create a new file based on it. This configuration file is then used to deploy the CCME Management Host through the script named deployCCME.sh.

Resources

The resources deployed in your AWS account by the CCME Management Host are:

An AWS Lambda with a log group in /ccme/ccme-lambda-alb-${StackId}
An AWS EC2 instance as CCME Management Host
Optional
- An Active Directory
- An Application Load Balancer
- Multiple AWS EC2 security-groups

Those resources are named using ccme as prefix and -Stack_id as suffix. Example for the resource LambdaALB, the resource is named: ccmeLambdaALB-7225f980-0a8a-11ee-81f0-06c4c07a0d95:

ccme as prefix
A variable information related to the resource: LambdaALB in this case
The stack_id of the CMH stack and a suffix preceded by a -: -7225f980-0a8a-11ee-81f0-06c4c07a0d95 in our case

The only exception for this rule is related to the Application Load Balancer (ALB). The ALB is using -Stack_name instead of the -Stack_id as suffix due to restrictions on the length of the name of the ALB

Deployment

The following command will create all the environment needed for CCME and create a Management Host named my_ccme_mgt_host. First it will upload all the source code of CCME to the ccme_bucket_path as specified in the deployment.ccme.conf file, this is mandatory on a first deployment. For further deployments, you can directly reuse the same bucket if no code change has been made.

$ ./deployCCME.sh --update --create-management-host --conf-management "my-ccme.deployment.ccme.conf" --name "my-ccme"
**** CCME Bucket Update - START ****
**** CCME Bucket Update - END ****
**** CCME ManagementHost Deployment - START ****
{
    "StackId": "arn:aws:cloudformation:eu-west-1:012345678910:stack/CMH-my-ccme/bf11b1e0-9a5a-11ed-ab69-06562ac4f907"
}
**** CCME ManagementHost Deployment - END ****

Customization

Custom Ansible Scripts

On top of CCME specific configurations, you can integrate your own custom scripts to CCME Management Host. To deploy a CMH embedding and executing your own custom scripts, you must place them in the management/custom directory and synchronize this directory in the S3 bucket. You can provide your own Ansible playbooks to add specific configurations to the CMH, they must have the following naming convention: management/custom/install-*-management.yaml.

Custom ParallelCluster configuration files

On top of the ParallelCluster example configuration files provided with CCME, you can provide your own Jinja template along with a set of parameters to generate your own configuration files.

To do so, you just need to:

add your templates in management/custom and name them with the following pattern *.pcluster.config.j2,
add any specific configuration variable in the management/custom/pcluster.vars.yaml file (YAML format). All variables in this file will be assigned under the custom variable. This means that if you declare myvar: myval, it will be available as custom.myvar in you *.pcluster.config.j2 files.

management/custom/custom.pcluster.config.j2

Region: '{{ AWS_REGION }}'
CustomS3Bucket: '{{ CCME_CLUSTER_S3BUCKET }}'
{%if CCME_CLUSTER_LAMBDA_ROLE != "NONE"%}
Iam:
  Roles:
    LambdaFunctionsRole: '{{ CCME_CLUSTER_LAMBDA_ROLE }}'
  # If the role associated to the cluster includes a custom IAM path prefix,
  # replace "parallelcluster" by the custom IAM path prefix.
  ResourcePrefix: "parallelcluster"
{% endif %}
Image:
  Os: alinux2
Tags:
{% for key, value in CCME_TAGS.items() %}
{% if "aws:" not in key and key != "Name" %}
  - Key: {{ key }}
    Value: {{ value }}
{% endif %}
{% endfor %}
SharedStorage:
  - Name: shared
    StorageType: Ebs
    MountDir: shared
    EbsSettings:
{% if "NONE" not in CCME_CLUSTER_KMS_EBS %}
      Encrypted: true
      KmsKeyId: {{ CCME_CLUSTER_KMS_EBS }}
{% else %}
      Encrypted: false
{% endif %}
HeadNode:
  InstanceType: {{ custom.headinsttype }}
  Networking:
    SubnetId: '{{ CCME_SUBNET }}'
    SecurityGroups:
      - '{{ CCME_PRIVATE_SG }}'
{%if CCME_PROXY is defined and CCME_PROXY and CCME_PROXY != "NONE"%}
    Proxy:
      HttpProxyAddress: '{{ CCME_PROXY }}'
{% endif %}
  Ssh:
    KeyName: '{{ AWS_KEYNAME }}'
  CustomActions:
    OnNodeStart:
      Script: s3://{{ CCME_SOURCES }}CCME/sbin/pre-install.sh
      Args:
        - CCME_CMH_NAME={{ CCME_CMH_NAME }}
        - CCME_S3FS={{ CCME_DATA_BUCKET }}
        - CCME_JSLOGS_BUCKET={{ CCME_DATA_BUCKET }}
{%if CCME_NO_PROXY is defined and CCME_NO_PROXY and CCME_NO_PROXY != "NONE"%}
        - CCME_NO_PROXY={{ CCME_NO_PROXY }}
{% endif %}
        # - CCME_OIDC=default
        # - CCME_USER_HOME=/shared-filesystem/home/%u
        # - CCME_DNS=NONE
        - CCME_WIN_LAUNCH_TEMPLATE_ID={{ CCME_WIN_LAUNCH_TEMPLATE_ID }}
        # - CCME_WIN_AMI=NONE
        # - CCME_WIN_INSTANCE_TYPE=NONE
        # - CCME_WIN_INACTIVE_SESSION_TIME=600
        # - CCME_WIN_NO_SESSION_TIME=600
        # - CCME_WIN_NO_BROKER_COMMUNICATION_TIME=600
        # - CCME_EF_ADMIN_GROUP=
        # - CCME_EF_ADMIN_PASSWORD=
{%if CCME_REPOSITORY_PIP is defined and CCME_REPOSITORY_PIP and CCME_REPOSITORY_PIP != "NONE"%}
        - CCME_REPOSITORY_PIP={{ CCME_REPOSITORY_PIP }}
{% endif %}
    OnNodeConfigured:
      Script: s3://{{ CCME_SOURCES }}CCME/sbin/post-install.sh
    OnNodeUpdated:
      Script: s3://{{ CCME_SOURCES }}CCME/sbin/update-install.sh
  Iam:
    InstanceProfile: '{{ CCME_CLUSTER_HEADNODE_INSTANCE_PROFILE }}'
Scheduling:
  Scheduler: slurm
  SlurmSettings:
    Dns:
      # If the role associated to the cluster is not authorized to use Route 53,
      # set "DisableManagedDns" to true.
      DisableManagedDns: False
  SlurmQueues:
{% for queue in custom.queues %}
    - Name: {{ queue.Name }}
      CapacityType: ONDEMAND
      ComputeResources:
{% for cr in queue.InstanceTypes %}
        - Name: {{ cr | replace('.', '') }}
          InstanceType: {{ cr }}
          MinCount: 0
          MaxCount: 10
{% endfor %}
      CustomActions:
        OnNodeStart:
          Script: s3://{{ CCME_SOURCES }}CCME/sbin/pre-install.sh
        OnNodeConfigured:
          Script: s3://{{ CCME_SOURCES }}CCME/sbin/post-install.sh
      Iam:
        InstanceProfile: '{{ CCME_CLUSTER_COMPUTE_INSTANCE_PROFILE }}'
      Networking:
        SubnetIds:
          - '{{ CCME_SUBNET }}'
        SecurityGroups:
          - '{{ CCME_COMPUTE_SG }}'
{%if CCME_PROXY is defined and CCME_PROXY and CCME_PROXY != "NONE"%}
        Proxy:
          HttpProxyAddress: '{{ CCME_PROXY }}'
{% endif %}
{% endfor %}
{%if CCME_AD_DIR_NAME != "NONE"%}
DirectoryService:
  DomainName: {{ CCME_AD_DIR_NAME }}
  DomainAddr: {% for adip in CCME_AD_IPS.split(',') %}{% if adip == CCME_AD_IPS.split(',')[0] %}ldap://{{ adip }}{% else %},ldap://{{ adip }}{% endif %}{% endfor %}{{''}}
  PasswordSecretArn: {{ CCME_AD_READ_ONLY_USER_PASSWORD }}
  DomainReadOnlyUser: {% if CCME_AD_ORIGIN == "external" %}{{ CCME_AD_READ_ONLY_USER_DN }}{% elif CCME_AD_ORIGIN == "internal" %}cn=ReadOnlyUser,ou=Users,ou={{ CCME_AD_DIR_NAME.split('.')[0] }}{% for addn in CCME_AD_DIR_NAME.split('.') %},dc={{ addn }}{% endfor %}{% endif %}{{''}}
  LdapTlsReqCert: never
  # LdapAccessFilter
  AdditionalSssdConfigs:
    # debug_level: "0x1ff"
    ldap_auth_disable_tls_never_use_in_production: True
{% endif %}
Imds:
  ImdsSupport: v2.0
DevSettings:
  Timeouts:
    HeadNodeBootstrapTimeout: 2400
    ComputeNodeBootstrapTimeout: 1800

management/custom/pcluster.vars.yaml

################################################################################
# Copyright (c) 2017-2024 UCit SAS
# All Rights Reserved
#
# This software is the confidential and proprietary information
# of UCit SAS ("Confidential Information").
# You shall not disclose such Confidential Information
# and shall use it only in accordance with the terms of
# the license agreement you entered into with UCit.
################################################################################
# This file can be used to set custom variables used to render the *.pcluster.conf.j2 templates
# All variables in this file will be assigned under the "custom" variable.
# This means that if you declare "myvar: myval", it will be available as custom.myvar
headinsttype: t3.xlarge
queues:
  - Name: basic-slurm
    InstanceTypes:
      - c5n.18xlarge
      - c5n.xlarge
  - Name: dcv-gpu
    InstanceTypes:
      - g4dn.xlarge

Connection

There are two network possibilities to connect to the Management Host: - From authorized IP / CIDR with a Public IP attributed to the CCME Management Host - From authorized IP / CIDR with a Private IP, using an instance in the same VPC (e.g.: bastion, proxy/bounce server…)

There are two user possibilities to connect to the Management Host:

CCME Admin user: ec2-user
User from the ActiveDirectory
- Must be in the group authorized to connect to the Management Host, as defined by the variable “management_host_authorized_group” of the configuration file described in the section “Configuration”

The required information are:

Username (ManagementHostUser): available in the CloudFormation output of the CCME ManagementHost stack
- ssh key: For the local user named ec2-user
- password: For user from the Active Directory
IP address: are available in the CloudFormation output of the CCME ManagementHost stack
- Public IP (ManagementHostPublicIP)
- Private IP (ManagementHostPrivateIP)

You can retrieve the output from the Management Host stack from the AWS Console as in this example:

Key	Value	Description
`ManagementHostPrivateIP`	10.0.0.32	Private IP Address of the Management Host
`ManagementHostPublicIP`	54.122.171.48	Public IP address of the ManagementHost
`ManagementHostUser`	ec2-user	User of the Management Host

Management Host

Prerequisites

deployCCME.sh

CCME Roles Stack (CRS)

CRS Configuration

Deployment

CCME Management Host (CMH)

CMH Configuration

Resources

Deployment

Customization

Custom Ansible Scripts

Custom ParallelCluster configuration files

Connection

`deployCCME.sh`