Summary

nf-fl86 is a Nextflow plugin that prevents a fresh workflow run from using an output directory that already exists. It uses the legacy params.outdir value when one is provided and otherwise falls back to Nextflow's session.outputDir. The check is skipped only when Nextflow is actually resuming a previous run.

At startup, the plugin also copies the CSV referenced by params.sample_sheet to <outdir>/pipeline_info/sample_sheet.csv. If params.sample_sheet is not set, it falls back to params.input.

If params.benchling is supplied, its JSON file is also copied unchanged to <outdir>/pipeline_info/benchling.json at startup. For example, pass --benchling ./benchling.json. The file must exist and be a regular file; resumed runs replace the saved copy. If the parameter is omitted, no copy is made.

Before tasks begin, the plugin saves the resolved workflow parameters to <outdir>/pipeline_info/params.json as pretty-printed JSON with top-level keys sorted alphabetically. Resumed runs replace the file with the current parameters.

When params.mproj is provided, the plugin sets Fusion tags to [*](MatrixProject=${params.mproj}). If it is missing or blank, Fusion tagging defaults to disabled unless fusion.tags is explicitly configured. Fusion must be enabled separately. This tag gets set in all s3 files created by the workflow.

S3-to-S3 publishDir copies (mode: 'copy') now preserve source object tags, including Fusion tags on large files copied using multipart transfers.

The plugin also sets dynamic process.resourceLabels with the Platform run ID, process name, task tag and hash, pipeline run and manifest names, and MatrixProject from params.mproj. Missing values are omitted.

After a successful run, the plugin writes workflow metadata to <outdir>/pipeline_info/workflow.json and creates pipeline.complete in the Nextflow work directory.

An optional seqera_action {} block can launch a follow-up Seqera Action after a successful run, using an access token resolved from AWS Secrets Manager.

The plugin can also send non-fatal lifecycle updates to an HTTP endpoint for a Notion-backed run tracker. It sends one update when the workflow begins and a second update only after successful completion.

Get Started

nf-fl86 requires Nextflow 25.10.0 or later. Enable a published release in your pipeline's nextflow.config:

plugins {
    id 'nf-fl86@0.5.0'
}

Run the pipeline with a new output directory. The optional sample sheet is copied into the pipeline metadata directory:

nextflow run hello \
    --outdir 's3://scratch/TEST' \
    --sample_sheet ./sample_sheet.csv

If the output directory belongs to an earlier run that should be resumed, use Nextflow's -resume option. The plugin permits an existing output directory only when the Nextflow session is running in resume mode.

Examples

S3 metadata output

Use an S3 output prefix and optionally S3 input files:

nextflow run your-pipeline \
    --outdir 's3://example-results/runs/run-123' \
    --sample_sheet 's3://example-inputs/samples.csv' \
    --benchling 's3://example-inputs/benchling.json'

The plugin writes sample_sheet.csv, benchling.json, params.json, and workflow.json under s3://example-results/runs/run-123/pipeline_info/, following the timing and optional-input rules described above. params.input can also reference an S3 sample sheet when params.sample_sheet is omitted.

The Nextflow launch environment needs AWS access to read the input objects and write to the output prefix. These metadata operations use Nextflow's S3 filesystem support through nf-amazon; Fusion is not required. Use a new prefix for a fresh run, or -resume when reusing an existing run's output directory.

Fusion project tags

Pass --mproj PROJECT_ID to set Fusion tags automatically at startup:

nextflow run hello --outdir results --mproj PROJECT_ID

This sets fusion.tags = "[*](MatrixProject=PROJECT_ID)", replacing any configured tag pattern. When mproj is missing or blank, tagging defaults to disabled (fusion.tags = false); an explicitly configured fusion.tags value is preserved. Other Fusion settings are preserved, and Fusion must be enabled separately.

Process resource labels

The plugin automatically configures process.resourceLabels with these values, evaluated separately for each task:

Label	Source
uniqueRunId	TOWER_WORKFLOW_ID environment variable
pipelineProcess	task.process
pipelineTag	task.tag
taskHash	task.hash
pipelineRunName	workflow.runName
pipelineName	workflow.manifest.name
MatrixProject	params.mproj

Values are converted to strings; missing or blank values are omitted. These labels are independent of Fusion and apply to compute resources on executors that support resource labels.

Existing labels in the global process.resourceLabels map or closure are merged with these defaults, with explicit values taking precedence. Process definitions and withName/withLabel selectors can override the entire directive using Nextflow's usual precedence rules. Other process settings are preserved.

Follow-up Seqera Action

Configure a follow-up workflow launch directly in nextflow.config. The plugin resolves the access token from AWS Secrets Manager and launches the specified Seqera Action only after the current workflow succeeds and its run name contains an uppercase NGSRUN or QENGSRUN identifier followed by one or more digits (for example, NGSRUN123 or QENGSRUN123):

seqera_action {
    aws_token_name = 'nextflow/secret'
    aws_region = 'us-east-1'
    run_name_suffix = '_quilt'
    action = '123456'
    workspace = '123456'
    params = '{"input":"s3://example-bucket/results"}'
}

No workflow.onComplete handler or script-level plugin import is required. The follow-up run name is the completed workflow's run name plus run_name_suffix. Set run_name instead when a fixed name is required.

params must be a JSON object string and is passed to Seqera as the follow-up workflow's parameters. If it is omitted, the plugin defaults to [input_dir: params.outdir]; if params.outdir is not set, it uses Nextflow's resolved output directory. For example:

seqera_action {
    // Required settings omitted for brevity.
    params = '{"input":"s3://example-bucket/results","priority":"high"}'
}

The optional api_base setting selects an HTTPS Seqera Enterprise API origin. Missing configuration, AWS failures, and Seqera API failures are logged as warnings and do not change a successful workflow into a failed one. The token, secret value, and API response body are not logged.

The AWS secret may contain either the raw access token or a JSON object with a TOWER_ACCESS_TOKEN field.

Notion lifecycle updates

Configure endpoint templates in nextflow.config to update a Notion-backed run tracker automatically:

notion {
    beginning_endpoint = 'https://example.execute-api.us-east-2.amazonaws.com/prod/update-status-to-mapping-underway?ngsrun={NGSRUN}'
    complete_endpoint = 'https://example.execute-api.us-east-2.amazonaws.com/prod/update-status-to-mapped?ngsrun={NGSRUN}&seqeraPlatformUrl={seqeraPlatformUrl}'
}

Use single-quoted endpoint templates so the plugin replaces placeholders at runtime:

• {NGSRUN} is the first NGSRUN\d+ value in workflow.runName.
• {seqeraPlatformUrl} is the nf-tower watch URL, or empty when monitoring is unavailable. It is a Notion endpoint placeholder, not an exported workflow function.

Placeholder values are URL-encoded. Each endpoint is optional; the beginning endpoint runs at startup, while the completion endpoint runs only after a successful workflow. Requests use POST with X-HTTP-Method-Override: PATCH. Invalid configuration, absent run IDs, timeouts, connection errors, and non-2xx responses are logged without failing the workflow.

Command-line plugin selection

The plugin can also be selected on the command line instead of in nextflow.config:

nextflow run hello \
    -plugins nf-fl86@0.3.0 \
    --outdir 's3://scratch/TEST' \
    --sample_sheet ./sample_sheet.csv

License

nf-fl86 is licensed under the Apache License 2.0.