Management Host

Prerequisites

CCME deployment can be done with the basic CCME automation configuration or manually. Even with the automated deployment, you still have the possibility to manually modify the CCME and AWS ParallelCluster configuration of your clusters.

The CCME_x.y.z directory contains the following files and directories:

  • deployCCME.sh: the script to start the Management Host

  • roles.ccme.conf: the configuration file for the CCME Roles Stack (CRS)

  • deployment.ccme.conf: the configuration file for the CCME Management Host (CMH)

  • dependencies.yaml: the version of the dependencies used in CCME

  • CCME/: the scripts to configure a CCME cluster

  • management/: the scripts to deploy the Management Host and associated services

  • doc/: this documentation

You will first need to untar the CCME_x.y.z.tar.gz archive to get access to the code on your local machine (which needs to have access to AWS and have the AWS CLI installed and configured):

mkdir "CCME_x.y.z"
tar xvf "CCME_x.y.z.tar.gz" -C "CCME_x.y.z"
cd "CCME_x.y.z"

deployCCME.sh

This is the script that you need to use to deploy the initial stacks for the CCME roles, the Management Host and the associated services.

$ ../../deployCCME.sh --help
Usage: deployCCME.sh [-h|--help] [-r|--create-roles-stack] [-m|--create-management-stack] [-cr|--conf-roles <crs_conf_file>] [-cm|--conf-management <cmh_conf_file>] [-n|--name <stack_name>] [-u|--update] [-nr|--no-rollback] [-p|--profile <aws_profile>]

deployCCME can be used to update the CCME bucket set in <conf_file> and/or to deploy a CCME Management Host

Options :
  -r|--create-roles-stack                           Create CCME Roles Stack (CRS) with a name built with the prefix 'CRS-' and the selected -n|--name option
  -m|--create-management-host                       Create CCME Management Host (CMH) with a name built with the the prefix 'CMH-' and the -n|--name option
  -cr|--conf-roles              <crs_conf_file>     Reads a configuration file for the CCME Roles Stack (CRS)
  -cm|--conf-management         <cmh_conf_file>     Reads a configuration file for the CCME Management Host (CMH) and update action
  -n|--name                     <stack_name>        Defines the name of the stack deployed by the script (default is CCME)
  -u|--update                                       Updates the CCME bucket
  -nr|--no-rollback                                 Disables rollback of the CMH creation stack in CloudFormation when the stack fails
  -p|--profile                  <aws_profile>       Use non-default AWS credentials profile with aws commmand
  -h/--help                                         Help

CCME Roles Stack (CRS)

CRS Configuration

The configuration of the “CCME Roles Stack” is described in the roles.ccme.conf file:

 1#### CONFIGURATION FILE for the CCME Roles Stack (CRS) automatic setup ####
 2
 3#### IAM Configuration
 4# iam_path_prefix defines the prefix of the IAM roles created by the stack
 5# The expected value is a string prefix like "ccme" or "parallelcluster".
 6# Setting this variable is OPTIONAL.
 7# Default: parallelcluster
 8# iam_path_prefix=
 9
10# enable_route53_access defines the possibility of Route53 creation and usage
11# for each cluster.
12# Setting this variable is OPTIONAL.
13# Value: true or false
14# Default: true
15# enable_route53_access=
16
17
18# enable_fsx_s3_access defines the possibility of FSx to access to s3 buckets
19# Setting this variable is OPTIONAL.
20# Value: true or false
21# Default: false
22# enable_fsx_s3_access=
23
24
25# fsx_s3_buckets defines the prefix of the IAM roles created by the stack
26# The expected value in the ARN of an IAM role. Check the documentation for more information about this role.
27# Setting this variable is OPTIONAL.
28# Default: parallelcluster
29# fsx_s3_buckets=
30
31# ccme_bucket_path is the BucketName:BucketKey where BacketName is the name of an existing s3 bucket
32# and where CCME machinery will be stored at the "BucketKey" path to be used to manage clusters.
33# E.g.: "ccme-bucket:subfolder1/subfolder2/subfolder3/"
34# Subfolder must end with a '/'
35# Setting this variable is MANDATORY. No default value exists.
36ccme_bucket_path=
37
38# ccme_data_bucket is the Name of an existing s3 bucket where common user data is accessed by users on any cluster.
39# Setting this variable is MANDATORY.
40# No default value exists.
41ccme_data_bucket=
42
43# ccme_efs is the ARN of an existing efs
44# Setting this variable is OPTIONAL.
45# No default value exists.
46# E.g.: ccme_efs=arn:aws:elasticfilesystem:eu-west-1:012345678910:file-system/fs-0f114ed124c21fec3
47# ccme_efs=
48
49# ccme_admin_sns_topic_arn define the admin SNS topic ARN you want to
50# deliver information about the cluster when it is ready to be used.
51# For example, you can configure your SNS topic to deliver the information
52# by email to the administrator of the platform
53# Setting this variable is OPTIONAL.
54# No default value exists.
55# E.g.: arn:aws:sns:REGION:ACCOUNT:TOPICNAME
56ccme_admin_sns_topic_arn=
57
58# ccme_secret_prefix defines the prefix of the secret that will be used
59# for the ActiveDirectory ReadOnlyUser and optionally administrator.
60# Setting this variable is OPTIONAL.
61# Default value is "ccme-".
62ccme_secret_prefix=
63
64# ccme_cluster_prefix defines the prefix of the future clusters
65# Used to restrict rights to CloudWatch dashboards
66# Setting this variable is OPTIONAL.
67# Default value is "ccme-".
68ccme_cluster_prefix=
69
70#### KMS Encryption
71# enable_kms_encryption defines the usage of KMS for storage encryption
72# When set to true the EBS, FSx, S3 ... will use the encryption key variables
73# Setting this variable is OPTIONAL.
74# Value: true or false
75# Default: false
76# enable_kms_encryption=
77
78# The following variables define the ARN of KMS keys used to encrypt storages or
79# communications on multiple services used by CCME.
80# Setting these variables is mandatory if enable_kms_encryption=true
81# Value: ARN of existing KMS key
82# No default value exists.
83# CMH EBS encryption
84# ccme_kms_cmh=
85# Clusters EBS encryption
86# ccme_kms_clusters=
87# FSx encryption
88# ccme_kms_fsx=
89# S3 encryption
90# ccme_kms_s3=
91# SNS encryption
92# ccme_kms_sns=
93# Secrets Manager encryption
94# ccme_kms_secrets=

You can modify and use this example configuration file or you can create a new file based on it. This configuration file is then used to deploy the CCME Management Host through the script named deployCCME.sh.

Deployment

The following command will create all the IAM roles needed for CCME and on a CCME Roles Stack.

$ ./deployCCME.sh --create-roles-stack --conf-roles "my-ccme.roles.ccme.conf" --name "my-ccme"
**** CCME Roles Deployment - START ****
{
    "StackId": "arn:aws:cloudformation:eu-west-1:012345678910:stack/CRS-my-ccme/af11b1e0-9a5a-11ed-ab69-06562ac4f907"
}
**** CCME Roles Deployment - END ****

CCME Management Host (CMH)

CMH Configuration

The configuration of the “CCME Management Host” is described in the deployment.ccme.conf file:

  1#### CONFIGURATION FILE for the CCME Management Host (CMH) automatic setup ####
  2
  3#### IAM Configuration
  4# management_stack_role defines the IAM role ARN that will be attached to the Management Stack.
  5# The expected value in the ARN of an IAM role. Check the documentation for more information about this role.
  6# This parameter corresponds to the output "ParallelClusterUserRole" of the CCME Roles Stack (CRS)
  7# Setting this variable is MANDATORY. No default value exists.
  8management_stack_role=
  9
 10# ccme_management_instance_profile defines the CCME Management Host (CMH) AWS IAM Instance Profile ARN
 11# This parameter corresponds to the output "ParallelClusterUserInstanceProfile" of the CCME Roles Stack (CRS)
 12# Setting this variable is MANDATORY. No default value exists.
 13ccme_management_instance_profile=
 14
 15# ccme_management_lambda_role defines the ApplicationLoadBalancer Lambda AWS IAM Role ARN
 16# This parameter corresponds to the output "ccmeAlbLambdaRole" of the CCME Roles Stack (CRS)
 17# Setting this variable is MANDATORY. No default value exists.
 18ccme_management_lambda_role=
 19
 20# ccme_cluster_lambda_role defines the ParallelCluster Lambda AWS IAM Role ARN
 21# This parameter corresponds to the output "CustomLambdaResourcesRoleSlurm" of the CCME Roles Stack (CRS)
 22# Setting this variable is OPTIONAL.
 23# Default: NONE
 24# ccme_cluster_lambda_role=
 25
 26# ccme_cluster_headnode_instance_profile defines the HeadNode AWS IAM Instance Profile ARN
 27# This parameter corresponds to the output "HeadNodeInstanceProfileSlurm" of the CCME Roles Stack (CRS)
 28# Setting this variable is OPTIONAL.
 29# Default: NONE
 30# ccme_cluster_headnode_instance_profile=
 31
 32# ccme_cluster_compute_instance_profile defines the Compute Nodes AWS IAM Instance profile ARN
 33# This parameter corresponds to the output "ComputeNodeInstanceProfileSlurm" of the CCME Roles Stack (CRS)
 34# Setting this variable is OPTIONAL. No default value exists.
 35# Default: NONE
 36# ccme_cluster_compute_instance_profile=
 37
 38#### Global Configuration
 39# debug is a boolean variable. When set to true, the stack does not automatically "rollback" after any failure.
 40# Default value: false
 41debug=false
 42
 43# timezone is a string variable. It configures the timezone for the CMH and the clusters
 44# Setting this variable is MANDATORY.
 45# Default value: 'Europe/Paris'
 46timezone='Europe/Paris'
 47
 48# ccme_tags defines a list of tags associated to the CMH stack
 49# Default value: no tag
 50# Pattern: ("Key=your_key,Value=your_value" "Key=your_key,Value=your_value")
 51# tags=("Key=your_key,Value=your_value" "Key=your_key,Value=your_value")
 52
 53# management_host_os defines the operating system of the CCME Management Host (CMH)
 54# Allowed values for this variable are the following:
 55# - al2023
 56# - rhel8
 57# Default value: al2023
 58management_host_os=al2023
 59
 60# management_host_ami defines the AMI for the CMH instance
 61# Setting this variable is OPTIONAL.
 62# Default value: NONE
 63# management_host_ami=
 64
 65# management_host_instance_type defines the instance type that will be configured as a CMH
 66# Allowed values for this variable are the following:
 67# - t2.micro
 68# - t2.small
 69# - t2.medium
 70# - t2.large
 71# - t3.small
 72# - t3.medium
 73# - t3.large
 74# - t3.xlarge
 75# - m5.large
 76# - m5.xlarge
 77# - m5.2xlarge
 78# - m5.4xlarge
 79# Default value: t2.micro
 80management_host_instance_type=t2.micro
 81
 82#### Network Configuration
 83# vpc defines the VPC where the CMH should be setup. The expected value is an existing VPCId.
 84# Setting this variable is MANDATORY. No default value exists.
 85vpc=
 86
 87# public_subnets defines the Primary and Secondary subnets (created in 2 different AZs)
 88# where the Application Load Balancer should be setup.
 89# The expected values is a pair of existing SubnetId.
 90# Setting this variable is MANDATORY. No default value exists.
 91public_subnets=( "" "" )
 92
 93# private_subnets defines the Primary and Secondary subnets (created in 2 different AZs)
 94# where the Active Directory and CCME Management Host should be setup.
 95# The expected values is a pair of existing SubnetId.
 96# Setting this variable is MANDATORY. No default value exists.
 97private_subnets=( "" "" )
 98
 99# additional_security_group defines one optional additional security group
100# to the CMH and clusters.
101# Setting this variable is OPTIONAL.
102# Default value: NONE
103additional_security_group=NONE
104
105# management_host_ip defines the address of an Elastic IP (if you already have one to use for your CMH)
106# If the default value is used, CCME will request a new Elastic IP to attach it to the CMH.
107# Set "NONE" to not assign a public IP.
108# Default value: 0.0.0.0
109management_host_ip=0.0.0.0
110
111
112#### Security Configuration
113# alb_ssl_certificate defines the SSL certificate used by the Application Load Balancer being the entry point to CCME.
114# The expected value in the ARN of a SSL certificate stored in AWS Certificate Manager (ACM).
115# Setting this variable is MANDATORY. No default value exists.
116alb_ssl_certificate=
117
118# application_load_balancer_scheme defines if the Application Load Balancer (ALB) is "internet-facing" or "internal"
119# Default value: 'internet-facing'
120application_load_balancer_scheme=internet-facing
121
122# application_load_balancer_arn defines the application load balancer used to forward the
123# portal and the visualization of CCME to the end users.
124# The expected value is the ARN of the choosen Application Load Balancer
125# Setting this variable is OPTIONAL.
126# Default value: NONE
127application_load_balancer_arn=NONE
128
129# secrets_manager_prefix defines the prefix of the secrets stored in the Secrets Service Manager
130# CCME requires AWS Secrets, it requires to register each secret with a name starting by the same prefix.
131# Default value: NONE
132# E.g.: "ccme-prefix"
133# E.g. of secret using this prefix: "arn:aws:secretsmanager:eu-west-1:012345678910:secret:ccme-prefix-adreadonlyuser.password-7dFt5"
134secrets_manager_prefix=
135
136# ccme_proxy defines the optional proxy for CMH and clusters
137# Setting this variable is OPTIONAL.
138# Default value: NONE
139# E.g.: "https://url_of_proxy/"
140ccme_proxy=
141
142# ccme_no_proxy defines the optional list of no_proxy delimited by comma for CMH and clusters
143# Setting this variable is OPTIONAL.
144# Default value: NONE
145# E.g.: "my.domain1,my.domain2"
146ccme_no_proxy=
147
148# ccme_repository_pip defines a private pip repository
149# Setting this variable is OPTIONAL.
150# Default value: NONE
151# E.g.: "https://url_of_pip_repo"
152ccme_repository_pip=
153
154# management_host_keypair is the Name of an existing EC2 key pair that should be used to connect to the CMH with SSH.
155# Setting this variable is MANDATORY. No default value exists.
156management_host_keypair=
157
158# management_host_authorized_group defines the AD groups of users which are allowed to login to your CMH.
159# If the default value is used, no filter is applied and everyone in AD is authorized to login.
160# Default value: NONE
161management_host_authorized_group=NONE
162
163
164#### Storage Configuration
165# ccme_bucket_path is the BucketName:BucketKey where BacketName is the name of an existing s3 bucket
166# and where CCME machinery will be stored at the "BucketKey" path to be used to manage clusters.
167# E.g.: "ccme-bucket:subfolder1/subfolder2/subfolder3/"
168# Subfolder must end with a '/'
169# Setting this variable is MANDATORY. No default value exists.
170ccme_bucket_path=
171
172# ccme_data_bucket is the Name of an existing s3 bucket where common user data is accessed by users on any cluster.
173# Setting this variable is MANDATORY. No default value exists.
174ccme_data_bucket=
175
176
177#### Active Directory Configuration
178# active_directory_name defines the Domain Name of your Active Directory.
179# If the value is NONE, then no Active Directory is deployed
180# Default value: ccme.ad
181active_directory_name=
182
183# active_directory_ips defines the list of IPs of the Active Directory.
184# When the default (NONE) value is used, a new AWS-managed Active Directory is created and the CMH is configured to use it.
185# Otherwise, the format is a comma separated list of IPS. E.g., 10.0.0.1,10.0.0.2
186# Default value: NONE
187active_directory_ips=NONE
188
189# active_directory_readonlyuser_dn defines the distinguished name of ReadOnlyUser account in your AD.
190# The expected value in the preexisting ARN is a "cn=string,OU=string,DC=string,DC=string".
191# Setting this variable differently than 'ReadOnlyUser' default value is MANDATORY ONLY IF the AD used for CCME is preexisting.
192# Default value: ReadOnlyUser
193active_directory_read_only_user_dn='ReadOnlyUser'
194
195# active_directory_read_only_user_password_secret_arn defines the password of ReadOnlyUser accounts in your AD.
196# The expected value in the preexisting ARN of a plaintext string stored in AWS Secrets Manager (ASM).
197# Setting this variable is MANDATORY. No default value exists.
198# E.g.: active_directory_read_only_user_password_secret_arn=arn:aws:secretsmanager:eu-west-1:012345678910:secret:ccme-prefix-adadmin.password-3nSOf
199active_directory_read_only_user_password_secret_arn=
200
201# active_directory_admin_password_secret_arn defines the password of Admin accounts in your AD.
202# The expected value in the preexisting ARN of a plaintext string stored in AWS Secrets Manager (ASM).
203# Setting this variable is MANDATORY ONLY IF the AD creation is automated
204# E.g.: active_directory_admin_password_secret_arn=arn:aws:secretsmanager:eu-west-1:012345678910:secret:ccme-prefix-adreadonlyuser.password-7dFt5
205active_directory_admin_password_secret_arn=
206
207#### KMS Encryption
208# enable_kms_encryption defines the usage of KMS for storage encryption
209# When set to true the EBS, FSx, S3 ... will use the encryption key variables
210# Setting this variable is OPTIONAL.
211# Value: true or false
212# Default: false
213# enable_kms_encryption=
214
215# The following variables define the ARN of KMS keys used to encrypt storages or
216# communications on multiple services used by CCME.
217# Setting these variables is mandatory if enable_kms_encryption=true
218# Value: ARN of existing KMS key
219# No default value exists.
220# CMH EBS encryption
221# ccme_kms_cmh=
222# Clusters EBS encryption
223# ccme_kms_clusters=
224# FSx encryption
225# ccme_kms_fsx=
226# S3 encryption
227# ccme_kms_s3=
228# SNS encryption
229# ccme_kms_sns=

You can modify and use this example configuration file or you can create a new file based on it. This configuration file is then used to deploy the CCME Management Host through the script named deployCCME.sh.

Resources

The resources deployed in your AWS account by the CCME Management Host are:

  • An AWS Lambda with a log group

  • An AWS EC2 instance as CCME Management Host

  • Optional

    • An Active Directory

    • An Application Load Balancer

    • Multiple AWS EC2 security-groups

Those resources are named using ccme as prefix and -Stack_id as suffix. Example for the resource LambdaALB, the resource is named: ccmeLambdaALB-7225f980-0a8a-11ee-81f0-06c4c07a0d95:

  • ccme as prefix

  • A variable information related to the resource: LambdaALB in this case

  • The stack_id of the CMH stack and a suffix preceded by a -: -7225f980-0a8a-11ee-81f0-06c4c07a0d95 in our case

The only exception for this rule is related to the Application Load Balancer (ALB). The ALB is using -Stack_name instead of the -Stack_id as suffix due to restrictions on the length of the name of the ALB

Deployment

The following command will create all the environment needed for CCME and create a Management Host named my_ccme_mgt_host. First it will upload all the source code of CCME to the ccme_bucket_path as specified in the deployment.ccme.conf file, this is mandatory on a first deployment. For further deployments, you can directly reuse the same bucket if no code change has been made.

$ ./deployCCME.sh --update --create-management-host --conf-management "my-ccme.deployment.ccme.conf" --name "my-ccme"
**** CCME Bucket Update - START ****
**** CCME Bucket Update - END ****
**** CCME ManagementHost Deployment - START ****
{
    "StackId": "arn:aws:cloudformation:eu-west-1:012345678910:stack/CMH-my-ccme/bf11b1e0-9a5a-11ed-ab69-06562ac4f907"
}
**** CCME ManagementHost Deployment - END ****

Customization

Custom Ansible Scripts

On top of CCME specific configurations, you can integrate your own custom scripts to CCME Management Host. To deploy a CMH embedding and executing your own custom scripts, you must place them in the management/custom directory and synchronize this directory in the S3 bucket. You can provide your own Ansible playbooks to add specific configurations to the CMH, they must have the following naming convention: management/custom/install-*-management.yaml.

Custom ParallelCluster configuration files

On top of the ParallelCluster example configuration files provided with CCME, you can provide your own Jinja template along with a set of parameters to generate your own configuration files.

To do so, you just need to:

  • add your templates in management/custom and name them with the following pattern *.pcluster.config.j2,

  • add any specific configuration variable in the management/custom/pcluster.vars.yaml file (YAML format). All variables in this file will be assigned under the custom variable. This means that if you declare myvar: myval, it will be available as custom.myvar in you *.pcluster.config.j2 files.

management/custom/custom.pcluster.config.j2
  1Region: '{{ AWS_REGION }}'
  2CustomS3Bucket: '{{ CCME_CLUSTER_S3BUCKET }}'
  3{%if CCME_CLUSTER_LAMBDA_ROLE != "NONE"%}
  4Iam:
  5  Roles:
  6    LambdaFunctionsRole: '{{ CCME_CLUSTER_LAMBDA_ROLE }}'
  7  # If the role associated to the cluster includes a custom IAM path prefix,
  8  # replace "parallelcluster" by the custom IAM path prefix.
  9  ResourcePrefix: "parallelcluster"
 10{% endif %}
 11Image:
 12  Os: alinux2
 13Tags:
 14{% for key, value in CCME_TAGS.items() %}
 15{% if "aws:" not in key and key != "Name" %}
 16  - Key: {{ key }}
 17    Value: {{ value }}
 18{% endif %}
 19{% endfor %}
 20SharedStorage:
 21  - Name: shared
 22    StorageType: Ebs
 23    MountDir: shared
 24    EbsSettings:
 25{% if "NONE" not in CCME_CLUSTER_KMS_EBS %}
 26      Encrypted: true
 27      KmsKeyId: {{ CCME_CLUSTER_KMS_EBS }}
 28{% else %}
 29      Encrypted: false
 30{% endif %}
 31HeadNode:
 32  InstanceType: {{ custom.headinsttype }}
 33  Networking:
 34    SubnetId: '{{ CCME_SUBNET }}'
 35    SecurityGroups:
 36      - '{{ CCME_PRIVATE_SG }}'
 37{%if CCME_PROXY is defined and CCME_PROXY and CCME_PROXY != "NONE"%}
 38    Proxy:
 39      HttpProxyAddress: '{{ CCME_PROXY }}'
 40{% endif %}
 41  Ssh:
 42    KeyName: '{{ AWS_KEYNAME }}'
 43  CustomActions:
 44    OnNodeStart:
 45      Script: s3://{{ CCME_SOURCES }}CCME/sbin/pre-install.sh
 46      Args:
 47        - CCME_CMH_NAME={{ CCME_CMH_NAME }}
 48        - CCME_S3FS={{ CCME_DATA_BUCKET }}
 49        - CCME_JSLOGS_BUCKET={{ CCME_DATA_BUCKET }}
 50{%if CCME_NO_PROXY is defined and CCME_NO_PROXY and CCME_NO_PROXY != "NONE"%}
 51        - CCME_NO_PROXY={{ CCME_NO_PROXY }}
 52{% endif %}
 53        # - CCME_OIDC=default
 54        # - CCME_USER_HOME=/shared-filesystem/home/%u
 55        # - CCME_DNS=NONE
 56        - CCME_WIN_LAUNCH_TEMPLATE_ID={{ CCME_WIN_LAUNCH_TEMPLATE_ID }}
 57        # - CCME_WIN_AMI=NONE
 58        # - CCME_WIN_INSTANCE_TYPE=NONE
 59        # - CCME_WIN_INACTIVE_SESSION_TIME=600
 60        # - CCME_WIN_NO_SESSION_TIME=600
 61        # - CCME_WIN_NO_BROKER_COMMUNICATION_TIME=600
 62        # - CCME_EF_ADMIN_GROUP=
 63        # - CCME_EF_ADMIN_PASSWORD=
 64{%if CCME_REPOSITORY_PIP is defined and CCME_REPOSITORY_PIP and CCME_REPOSITORY_PIP != "NONE"%}
 65        - CCME_REPOSITORY_PIP={{ CCME_REPOSITORY_PIP }}
 66{% endif %}
 67    OnNodeConfigured:
 68      Script: s3://{{ CCME_SOURCES }}CCME/sbin/post-install.sh
 69    OnNodeUpdated:
 70      Script: s3://{{ CCME_SOURCES }}CCME/sbin/update-install.sh
 71  Iam:
 72{%if CCME_CLUSTER_HEADNODE_INSTANCE_PROFILE != "NONE" %}
 73    InstanceProfile: '{{ CCME_CLUSTER_HEADNODE_INSTANCE_PROFILE }}'
 74{% else %}
 75    S3Access:
 76      - BucketName: '{{ CCME_BUCKET }}'
 77      - EnableWriteAccess: true
 78        BucketName: '{{ CCME_DATA_BUCKET }}'
 79    AdditionalIamPolicies:
 80    - Policy: '{{ CCME_CLUSTER_POLICY_ALB }}'
 81    - Policy: '{{ CCME_CLUSTER_POLICY_DCV }}'
 82    - Policy: '{{ CCME_CLUSTER_POLICY_EF }}'
 83    - Policy: '{{ CCME_CLUSTER_POLICY_JOB_COSTS }}'
 84    - Policy: '{{ CCME_CLUSTER_POLICY_SNS }}'
 85    - Policy: '{{ CCME_CLUSTER_POLICY_SSM }}'
 86    - Policy: 'arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore'
 87{% endif %}
 88Scheduling:
 89  Scheduler: slurm
 90  SlurmSettings:
 91    Dns:
 92      # If the role associated to the cluster is not authorized to use Route 53,
 93      # set "DisableManagedDns" to true.
 94      DisableManagedDns: False
 95  SlurmQueues:
 96{% for queue in custom.queues %}
 97    - Name: {{ queue.Name }}
 98      CapacityType: ONDEMAND
 99      ComputeResources:
100{% for cr in queue.InstanceTypes %}
101        - Name: {{ cr | replace('.', '') }}
102          InstanceType: {{ cr }}
103          MinCount: 0
104          MaxCount: 10
105{% endfor %}
106      CustomActions:
107        OnNodeStart:
108          Script: s3://{{ CCME_SOURCES }}CCME/sbin/pre-install.sh
109        OnNodeConfigured:
110          Script: s3://{{ CCME_SOURCES }}CCME/sbin/post-install.sh
111      Iam:
112{% if CCME_CLUSTER_COMPUTE_INSTANCE_PROFILE != "NONE" %}
113        InstanceProfile: '{{ CCME_CLUSTER_COMPUTE_INSTANCE_PROFILE }}'
114{% else %}
115        S3Access:
116          - BucketName: '{{ CCME_BUCKET }}'
117          - EnableWriteAccess: true
118            BucketName: '{{ CCME_DATA_BUCKET }}'
119        AdditionalIamPolicies:
120        - Policy: '{{ CCME_CLUSTER_POLICY_ALB }}'
121        - Policy: '{{ CCME_CLUSTER_POLICY_DCV }}'
122        - Policy: '{{ CCME_CLUSTER_POLICY_JOB_COSTS }}'
123        - Policy: '{{ CCME_CLUSTER_POLICY_SNS }}'
124        - Policy: '{{ CCME_CLUSTER_POLICY_SSM }}'
125        - Policy: 'arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore'
126{% endif%}
127      Networking:
128        SubnetIds:
129          - '{{ CCME_SUBNET }}'
130        SecurityGroups:
131          - '{{ CCME_COMPUTE_SG }}'
132{%if CCME_PROXY is defined and CCME_PROXY and CCME_PROXY != "NONE"%}
133        Proxy:
134          HttpProxyAddress: '{{ CCME_PROXY }}'
135{% endif %}
136{% endfor %}
137{%if CCME_AD_DIR_NAME != "NONE"%}
138DirectoryService:
139  DomainName: {{ CCME_AD_DIR_NAME }}
140  DomainAddr: {% for adip in CCME_AD_IPS.split(',') %}{% if adip == CCME_AD_IPS.split(',')[0] %}ldap://{{ adip }}{% else %},ldap://{{ adip }}{% endif %}{% endfor %}{{''}}
141  PasswordSecretArn: {{ CCME_AD_READ_ONLY_USER_PASSWORD }}
142  DomainReadOnlyUser: {% if CCME_AD_ORIGIN == "external" %}{{ CCME_AD_READ_ONLY_USER_DN }}{% elif CCME_AD_ORIGIN == "internal" %}cn=ReadOnlyUser,ou=Users,ou={{ CCME_AD_DIR_NAME.split('.')[0] }}{% for addn in CCME_AD_DIR_NAME.split('.') %},dc={{ addn }}{% endfor %}{% endif %}{{''}}
143  LdapTlsReqCert: never
144  # LdapAccessFilter
145  AdditionalSssdConfigs:
146    # debug_level: "0x1ff"
147    ldap_auth_disable_tls_never_use_in_production: True
148{% endif %}
149Imds:
150  ImdsSupport: v2.0
151DevSettings:
152  Timeouts:
153    HeadNodeBootstrapTimeout: 2400
154    ComputeNodeBootstrapTimeout: 1800
management/custom/pcluster.vars.yaml
 1################################################################################
 2# Copyright (c) 2017-2023 UCit SAS
 3# All Rights Reserved
 4#
 5# This software is the confidential and proprietary information
 6# of UCit SAS ("Confidential Information").
 7# You shall not disclose such Confidential Information
 8# and shall use it only in accordance with the terms of
 9# the license agreement you entered into with UCit.
10################################################################################
11# This file can be used to set custom variables used to render the *.pcluster.conf.j2 templates
12# All variables in this file will be assigned under the "custom" variable.
13# This means that if you declare "myvar: myval", it will be available as custom.myvar
14headinsttype: t3.xlarge
15queues:
16  - Name: basic-slurm
17    InstanceTypes:
18      - c5n.18xlarge
19      - c5n.xlarge
20  - Name: dcv-gpu
21    InstanceTypes:
22      - g4dn.xlarge

Connection

There are two network possibilities to connect to the Management Host: - From authorized IP / CIDR with a Public IP attributed to the CCME Management Host - From authorized IP / CIDR with a Private IP, using an instance in the same VPC (e.g.: bastion, proxy/bounce server…)

There are two user possibilities to connect to the Management Host:

  • CCME Admin user: ec2-user

  • User from the ActiveDirectory

    • Must be in the group authorized to connect to the Management Host, as defined by the variable “management_host_authorized_group” of the configuration file described in the section “Configuration”

The required information are:

  • Username (ManagementHostUser): available in the CloudFormation output of the CCME ManagementHost stack

    • ssh key: For the local user named ec2-user

    • password: For user from the Active Directory

  • IP address: are available in the CloudFormation output of the CCME ManagementHost stack

    • Public IP (ManagementHostPublicIP)

    • Private IP (ManagementHostPrivateIP)

You can retrieve the output from the Management Host stack from the AWS Console as in this example:

Key

Value

Description

ManagementHostPrivateIP

10.0.0.32

Private IP Address of the Management Host

ManagementHostPublicIP

54.122.171.48

Public IP address of the ManagementHost

ManagementHostUser

ec2-user

User of the Management Host