Chaining of Pipelines
evaluating
C
Charcoal Mandrill
I think this has been discussed several times for Nextflow in various places, including on the Nextflow GitHub Issues pages. Would be great to have this kind of feature. I think its been mentioned a few times at this point but CWL actually does a great job of enabling this by serializing the pipeline output metadata into a JSON format that can be used as input to the next pipeline. Very similar to what Nextflow's nf-prov already does, and similar to Nextflow's newer "named outputs" defined in the workflow; the main thing missing really is connecting a JSON serialized output definition to the input declaration of the next Nextflow pipeline.
Implementing it within Nextflow itself and making it available via Seqera Platform would be two separate things I think though, are there details available about the proposed methods to be used? Ideally you could have this enabled directly from Nextflow, so its usable from the standard interactive cli pipeline run, and then Platform would have extra support to facilitate it.
Rob Newman
evaluating
Analyzing how pipeline output formats can be used as a trigger for new pipeline runs.
Rob Newman
acknowledged
Y
Yinmn blue Orangutan
This is a use case that has come up many times in my org. Would be very powerful for making feature-rich E2E pipelines.