Seqera Platform Feature Requests

When using FSx for NetApp ONTAP to store raw sequencing data, we'll need to be able to easily leverage this storage solution from within Seqera Platform.

To easily automate cleanup of (large) process work directories without losing important tower metadata files, like the nf-.. files used for Reports, it would be good to be able to store them in a different path (e.g. a different prefix on S3). Right now they are both directly in workDir , making it impossible to use simple AWS lifecycle rules to cleanup process work folders without also deleting the nf-* files and breaking the Report links from the tower front-end

Our NextFlow pipelines generate very large interim files - sometimes this can be 6-8X the size of the input data. Storing all this data can be a burden, especially in resource constrained HPC systems - our pipelines sometimes fail just because they run out of scratch space. This could be (partially) address if we could clean the scratch directories _during pipeline execution_.

We use GCP to run our NextFlow pipelines from Seqera platform and generate many PB of data a year. To clean up this data, we have to perform manual cleanups or implement features outside of Seqera Platform. It would be useful to get Seqera Platform to attach Lifecycle Policies to the data generated by a pipeline so that it is automatically cleaned up.

We would like the ability to use multiple Azure Blob Storage Accounts in Nextflow with the az: syntax. Storage accounts within Azure each have their own unique credentials that are required by the Seqera Platform to access the storage account. Additionally, the Platform can only currently support one credential per compute environment, setting a hard limit on 1 CE only being able to access a single storage account based on the credential.

Many AWS EC2 instances provide out-of-the-box additional storage capacity without the need to mount additional EBS volumes and incur additional changes. The limitations of this storage type are: Each different instance type has a different amount of instance storage capacity and volumes. A list of instance types is available . The instance storage is required to be "formatted" and mounted in the host instance before it can be used. Seqera Platform/Tower should streamline the usage of EC2 instance storage used by AWS Batch jobs by automatically discovering, formating and mounting instance storage volumes provided by the computing node, and make it available in the Batch jobs as scratch storage for the local task computation.