Summary

nf-gc is a Nextflow plugin that provides lifecycle-aware garbage collection for work artifacts. It reclaims eligible intermediate outputs from a Nextflow work directory while the workflow is still running. The plugin tracks ownership, output provenance, downstream demand, publication, workflow-output retention, and deletion per concrete artifact path. It offers two garbage collection modes: artifact (default, fine-grained tracking) and process (coarser process-level boundaries).

Get Started

Enable the plugin in nextflow.config:

plugins {
    id 'nf-gc@0.3.0'
}

No nf-gc-specific configuration is required. The default configuration is:

nfGc {
    gc_mode = 'artifact'
}

To use the coarser process clock explicitly:

nfGc {
    gc_mode = 'process'
}

Run a pipeline normally with the plugin configured, or select it from the command line:

nextflow run main.nf -plugins nf-gc@0.3.0

For lowest peak disk usage, prefer publishDir mode: 'link' when work and results paths are on the same filesystem, as hard-link publication does not duplicate the payload and gives nf-gc synchronous publication evidence.

To enable lifecycle tracing for testing and field runs:

env.NF_GC_TEST_TRACE = 'true'

Examples

Basic plugin configuration with default artifact mode:

plugins {
    id 'nf-gc@0.3.0'
}

nfGc {
    gc_mode = 'artifact'
}

Using process mode for coarser garbage collection:

plugins {
    id 'nf-gc@0.3.0'
}

nfGc {
    gc_mode = 'process'
}

Publishing with link mode for optimal disk usage:

process {
    publishDir = [path: { "${params.outdir}/${task.process.replaceAll(/:/, '_')}" }, mode: 'link']
}

Running from the command line with the plugin:

nextflow run main.nf -plugins nf-gc@0.3.0

License

Licensed under the Apache License 2.0. See LICENSE for the full license text.