Summary

nf-python is a Nextflow plugin enabling seamless integration between Nextflow and Python scripts. Python scripts can be invoked as part of your Nextflow workflow, and arguments and outputs will be serialized back-and-forth automatically, making them available as native pythonic objects.

Get Started

To use the plugin in your Nextflow data pipelines, simply include it by writing include { pyFunction } from 'plugin/nf-python' and it will be downloaded and installed. For all setup options, please refer to the Nextflow plugins documentation.

nf-python requires a working python installation in the execution environment. By default, the python in path will be used. It is also possible to specify a path to a specific python executable or a conda environment. Either in the configuration file:

nf_python {
  // Option 1
  executable = '/usr/bin/python'
  // Option 2
  conda_env = ''
}

The supported values for the conda_env option are the same ones supported by Nextflow's native conda support, and is using the same configuration options. Here are a few options:

1. A list of required packages (e.g. conda_env = 'numpy biopython')
2. A conda configuration file (e.g. conda_env = '/opt/task-env.yml')
3. A path to an existing conda environment (e.g. conda_env = '/home/user/.conda/envs/my-env')

It is also possible to specify different environments on a per-function basis:

pyFunction(script: "", x: 1, y: 2, _executable: "/usr/bin/python")
pyFunction(script: "", x: 1, y: 2, _conda_env: "matplotlib")

Examples

To use a python script as part of your Nextflow pipeline, import pyFunction from the nf-python plugin:

include { pyFunction } from 'plugin/nf-python'

pyFunction('my_other_script.py', foo: 'too')

Usage inside python is easy:

• Arguments (like foo) are passed as nf_python.nf.args.
• To return results to nextflow, use nf_python.nf.output(...)
• All major native data types (lists, dicts, etc.) are supported.

# example_script.py
from nf_python import nf

# Access arguments and options
print(nf.args)
print(nf.opts)

# Assign output
nf.output(result=nf.args[0] + 1)

Inline Code Support

pyFunction also support inline code. Here is an example:

result = pyFunction("""
    # Code is automatically de-indented correctly
    # `nf_python.nf` is accessible as `nf`
    nf.output(arg1=123, arg2='abc')
""")
assert result == [arg1: 123, arg2: 'abc']

result = pyFunction("""
    nf.output(arg1=123, arg2='abc', arg3=nf.args['foo'])
""", foo: 1)
assert result == [arg1: 123, arg2: 'abc', arg3: 1]

License

See COPYING for license information. The layout of the project is based on the nf-hello project, licensed under the Apache License 2.0.