Seqera Platform to provide fallback for log upload when Nextflow fails
acknowledged
D
Dramatic Possum
It is possible for Nextflow to fail early which prevents the cleanup and log upload which would ordinarily happen at the end of a run. Without the logs on cloud storage, it becomes difficult to understand the failure.
It is possible to force the upload by adding in a post-run script such as:
echo "Copying log files:" for log in nf-*.log*; do
aws s3 cp $log s3://my-bucket-name/debug-logs/
done
It is possible to add this sort of fallback log upload as a built-in feature in Seqera Platform?
D
Dramatic Possum
To clarify, the log files fail to upload when a plugin (in my case, nf-quilt) fails execution prior to a task running. It seems that Tower lacks the handling to recover from the plugin error and upload to remote storage the log files that are crucial for diagnosing errors.
Thanks,
James
Rob Newman
acknowledged