Support customer-managed KMS keys (CMK) for AWS Secrets Manager pipeline secrets
acknowledged
Rob Syme
When Platform launches a run on an AWS Batch compute environment, it creates a temporary AWS Secrets Manager secret for each selected workspace or user secret and removes it when the run completes. These secrets are created with the default AWS-managed key, and there is currently no option to have Platform encrypt them with a customer-managed KMS key (CMK).
This prevents workspace secrets from working in any AWS account that enforces CMK encryption on Secrets Manager. We have hit this at an enterprise customer whose account runs an automated governance policy that deletes any secret not encrypted with a CMK. The policy removes Platform's secret within seconds of it being created, before the Nextflow head job reads it, so every run that uses a secret fails with Unable to find a secret with name '<x>' — with no indication that the secret was created and then deleted by a third party.
The request is to add an option to specify a customer-managed KMS key for these secrets, applied at creation time (the KmsKeyId parameter of the Secrets Manager CreateSecret API). A per-compute-environment setting fits best, since the key is account- and region-specific, with an optional global default. The equivalent capability for GCP Secret Manager would be valuable too. The compute roles that read these secrets already require kms:Decrypt, so no read-path changes should be needed.
The only current workaround is for the customer to exempt Platform's secret name prefix (tower-*) from their CMK-enforcement policy. Security teams in regulated environments often will not grant that exemption, which leaves affected customers unable to use workspace secrets on AWS at all.
M
Michael Tansini
updated the status to
acknowledged