Management

Management Host update policies

In CCME, AWS CloudFormation analyzes the settings used to create the current CCME Management Host stack and the new settings for issues. If any issues are discovered, they are reported through AWS CloudFormation. This issue is reported when it is discovered. If no blocking issues are discovered, update process is started and the changes are reported.

Update Policy: definitions

This setting can be changed during an update.

After changing this setting, the CCME Management Host stack can be updated.

This setting can be changed during an update and requires to update the deployed clusters with the updated setting.

After changing this setting, the CCME Management Host stack can be updated. After the update of the CCME Management Host stack you must report the new settings to the deployed clusters.

This setting can be changed during an update if the CCME Role Stack allows access to the resource ARN.

After changing this setting, the CCME Management Host stack can be updated. The CCME Role Stack must allows access to the resource in order to avoid unauthorized access to the target AWS resource.

This setting can be changed during an update only if the previous and the new values are set for internal Active Directory.

After changing this setting is possible at the exception of the next case: the current CCME Management Host stack has internal AD deployed and the new settings is not NONE or external ActiveDirectory.

If this setting is changed, the update is not allowed.

After changing this setting, the CCME Management Host stack can’t be updated. You must revert the settings for the original Management Host and create a new CCME Management Host stack with the updated settings. You can delete the original CCME Management Host stack at a later date.

Cluster update policies

Update Policy: settings

Setting	Update Policy
CCME_UPDATE_DATE	This setting can be changed in the “OnNodeUpdated/Args” section during an update.
CCME_CMH_NAME	If this setting is changed, the update is not allowed.
CCME_SOURCES	If this setting is changed, the update is not allowed.
CCME_CLUSTER_S3BUCKET	This setting can be changed during an update if the CCME Role Stack allows access to the resource ARN.
CCME_CLUSTER_CLUSTER_LAMBDA_ROLE	This setting can be changed during an update if the CCME Role Stack allows access to the resource ARN.
CCME_S3FS	This setting can be changed during an update if the CCME Role Stack allows access to the resource ARN.
CCME_JSLOGS_BUCKET	If this setting is changed, the update is not allowed.
CCME_NO_PROXY	If this setting is changed, the update is not allowed.
CCME_EFADMIN_PASSWORD	This setting can be changed during an update if the CCME Role Stack allows access to the resource ARN.
CCME_OIDC	This setting can be changed in the “OnNodeUpdated/Args” section during an update.
CCME_USER_HOME	This setting can be changed in the “OnNodeUpdated/Args” section during an update.
CCME_DNS	This setting can be changed in the “OnNodeUpdated/Args” section during an update.
CCME_EF_ADMIN_GROUP	This setting can be changed in the “OnNodeUpdated/Args” section during an update.
CCME_REPOSITORY_PIP	This setting can be changed in the “OnNodeUpdated/Args” section during an update.
CCME_WIN_AMI	This setting can be changed in the “OnNodeUpdated/Args” section during an update.
CCME_WIN_INSTANCE_TYPE	This setting can be changed in the “OnNodeUpdated/Args” section during an update.
CCME_WIN_INACTIVE_SESSION_TIME	This setting can be changed in the “OnNodeUpdated/Args” section during an update.
CCME_WIN_NO_SESSION_TIME	This setting can be changed in the “OnNodeUpdated/Args” section during an update.
CCME_WIN_NO_BROKER_COMMUNICATION_TIME	This setting can be changed in the “OnNodeUpdated/Args” section during an update.
CCME_WIN_TAGS	This setting can be changed in the “OnNodeUpdated/Args” section during an update.
CCME_AWS_SSM	This setting can be changed in the “OnNodeUpdated/Args” section during an update.
CCME_CUSTOM_SLURMDBD_SETTINGS	If this setting is changed, the update is not allowed.

Update Policy: definitions

Update Policy: This setting can be changed in the “OnNodeUpdated/Args” section during an update.

After changing this setting, the CCME cluster can be updated.

Update Policy: This setting can be changed during an update if the CCME Role Stack allows access to the resource ARN.

After changing this setting, the CCME cluster can be updated. The CCME Role Stack must allows access to the resource in order to avoid unauthorized access to the target AWS resource.

Update Policy: If this setting is changed, the update is not allowed.

After changing this setting, the CCME cluster can’t be updated. You must revert the settings for the original cluster and create a new CCME cluster with the updated settings. You can delete the original CCME cluster stack at a later date.

Update Cluster: process

List of actions to update the configuration of a cluster:

• On the HeadNode of the cluster:

  • Stop the batch jobs with scancel <job_id>

  • Stop the VDI Windows if any with: connect to EnginFrame as an administrator, click on Admin portal/All Sessions, and terminate all active sessions.

• On the the CCME Management Host, you will need to perform the following steps. On each step, you can check the Compute Fleet status with pcluster describe-compute-fleet -n <cluster_name> -r <aws_region>:

  1. Stop the Compute Fleet with pcluster update-compute-fleet --status STOP_REQUESTED -n <cluster_name> -r <aws_region>, and wait for the Compute Fleet status to be stopped

  2. Update the cluster with pcluster update-cluster -c <configuration_file> -n <cluster_name> -r <aws_region>, and wait for the end of the cluster update

  3. Start again the Compute Fleet with pcluster update-compute-fleet --status START_REQUESTED -n <cluster_name> -r <aws_region>, and wait for the Compute Fleet status to be running

  4. Your cluster is now updated and functional

For more information about pcluster update-cluster command, refer to: https://docs.aws.amazon.com/parallelcluster/latest/ug/pcluster.update-cluster-v3.html

Updating CCME dependencies

Warning

Updating CCME dependencies in dependencies.yaml can break CCME. First test any changes in a sandbox environment where you don’t risk to break your production cluster.

In some cases, it can be necessary to update CCME dependencies. One of such cases is for example when a CVE has been issued for one of the components of CCME.

If you need to do so, update the version of the impacted CCME dependencies in CCME/dependencies.yaml and/or management/dependencies.yaml

For example, if you want to update Slurm to a newer version, you can update the slurm: "YY.MM.V" parameter.

Then, start a new CCME cluster, or update an currently running CCME cluster to apply the changes.