Summary

This plugin implements a Nextflow executor for Hashicorp Nomad, enabling Nextflow pipelines to submit and manage tasks on Nomad clusters. It supports distributed workdirs via SPI for clusters without shared filesystems, custom job naming, process-level Nomad options, and comprehensive task lifecycle management.

Get Started

To use this plugin in your Nextflow pipeline, add the following to your nextflow.config file:

plugins {
    id 'nf-nomad'
}

process {
    executor = 'nomad'
}

nomad {
    client {
        address = 'http://your-nomad-server:4646'
    }
}

Testing and Debugging

To run and test the plugin in a development environment, configure a local Nextflow build with the following steps:

1. Clone the Nextflow repository in your computer into a sibling directory:

   git clone --depth 1 https://github.com/nextflow-io/nextflow _resources/nextflow
   

2. Generate the nextflow class path

   cd _resources/nextflow && ./gradlew exportClasspath
   

3. Compile the plugin alongside the Nextflow code:

   cd ../../ && ./gradlew compileGroovy
   

4. Run Nextflow with the plugin, using ./launch.sh as a drop-in replacement for the nextflow command, and adding the option -plugins nf-nomad to load the plugin:

   ./launch.sh run main.nf -plugins nf-nomad
   

Unit Testing

Run the following command in the project root directory (ie. where the file settings.gradle is located):

./gradlew check

The Gradle test task supports selecting the target environment with -PtestEnv:

• ./gradlew test → unit tests only
• ./gradlew test -PtestEnv=mock → unit + mock tests
• ./gradlew test -PtestEnv=local → unit + local Nomad integration tests
• ./gradlew test -PtestEnv=oci → unit + OCI Nomad integration tests

For local integration tests, the default Nomad address is http://localhost:4646. If you need a different endpoint, override explicitly:

./gradlew test -PtestEnv=local -PnomadAddr=http://<host>:4646

Examples

The following example demonstrates how to configure a Nextflow pipeline to target a specific process to run on Nomad with customized datacenters and resource limits:

// nextflow.config
plugins {
  id 'nf-nomad'
}

process {
  executor = 'nomad'
  
  withName: runAlignment {
    nomadOptions = [
      datacenters: ['dc-analytics'],
      priority: 'high',
      resources: [
        cores: 8,
        memoryMax: '32 GB'
      ]
    ]
  }
}

Process-level Nomad Options

Process directives support both legacy keys and the preferred map-based nomadOptions directive:

process {
  withName: sayHello {
    nomadOptions = [
      datacenters: ['dc1', 'dc2'],
      namespace: 'bio',
      constraints: { node { unique = [name: params.RUN_IN_NODE] } },
      affinity: [attribute: '${meta.workload}', operator: '=', value: 'batch', weight: 25],
      meta: [owner: 'team-x', step: 'align'],
      shutdownDelay: '15s',
      failures: [
        restart: [attempts: 1, delay: '5s', mode: 'fail'],
        reschedule: [attempts: 2, delay: '10s']
      ],
      volumes: [[type: 'host', name: 'ref-data', path: '/ref', readOnly: true]],
      secretsPath: 'secret/projects/team-x',
      secrets: ['MY_ACCESS_KEY', 'MY_SECRET_KEY'],
      spread: [name: 'node.datacenter', weight: 50, targets: ['us-east1': 70, 'us-east2': 30]],
      priority: 'high',
      resources: [memoryMax: '64 GB', cores: 4, device: [[name: 'nvidia/gpu', count: 1]]]
    ]
  }
}

Distributed Workdir via SPI

nf-nomad ships a DistributedWorkdirProvider SPI for clusters without a shared filesystem between head node and workers. The nf-nomad-s5cmd provider plugin offers S3-backed work-dir via s5cmd. Activate with:

nomad {
  s5cmd.enabled = true
  s5cmd.workDir.enabled = true
  s5cmd.workDir.bucket = 's3://…'
}

Custom Nomad Job Naming

If you wish to customize the Nomad job-id naming, set NF_NOMAD_RUN_TAG in the environment:

export NF_NOMAD_RUN_TAG=my-run

Runtime Security and Resource Configuration

Nomad Docker jobs are not privileged by default. To opt in:

nomad {
  jobs {
    privileged = true
  }
}

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.