Copy data from public bucket to organisation bucket
acknowledged
Adam Talbot
There are many cases where you want to use public data but it's not practical or feasible to pull it from the hosted region. In those cases, it would be better to copy the data into your own bucket which will put it alongside your compute environment, saving you costs and time in the long run.
An example use case would be copying certain files from the igenomes S3 bucket to your own AWS region or cloud provider for use with nf-core pipelines.
Rob Newman
acknowledged
Rob Newman
Adam Talbot Is there a reason for doing this in Data Explorer instead of using an existing dedicated service such as AWS DataSync?
Adam Talbot
Of course, there's a million ways of doing this but all:
- Are limited in scope (e.g. few data sources supported)
- Require permissions (e.g. admin rights to AWS)
- Require extensive customisation or code
A platform native solution would be preferable to these.