Amazon Web Services plugin for Nextflow

Summary

The Amazon Web Services (AWS) plugin provides support for AWS, including AWS Batch as a compute executor, S3 as a file system, and Fusion file system for high-performance data operations.

Get Started

To use this plugin, add it to your nextflow.config:

plugins {
    id 'nf-amazon'
}

Configure your AWS credentials using environment variables, AWS CLI profiles, or IAM roles. Then set up the executor and work directory:

process.executor = 'awsbatch'
process.queue = '<YOUR BATCH QUEUE>'
workDir = 's3://<YOUR BUCKET>/work'

aws {
    region = 'us-east-1'
    batch {
        cliPath = '/home/ec2-user/miniconda/bin/aws'
    }
}

Examples

Basic AWS Batch configuration

plugins {
    id 'nf-amazon'
}

process.executor = 'awsbatch'
process.queue = 'my-batch-queue'
workDir = 's3://my-bucket/work'

aws {
    region = 'eu-west-1'
    batch {
        cliPath = '/home/ec2-user/miniconda/bin/aws'
        jobRole = 'arn:aws:iam::123456789:role/MyBatchJobRole'
    }
}

Using Fusion file system

fusion {
    enabled = true
}

wave {
    enabled = true
}

process.executor = 'awsbatch'
workDir = 's3://my-bucket/work'

S3 storage options

aws {
    client {
        maxConnections = 20
        connectionTimeout = 10000
        storageEncryption = 'AES256'
    }
    region = 'us-east-1'
}

License

Apache License 2.0