Warning: This document is for the development version of Pydra: A simple dataflow engine with scalable semantics. The main version is master.

pydra.engine.environments module

class pydra.engine.environments.Container(image, tag='latest', root='/mnt/pydra', xargs=None)

Bases: Environment

Base class for container environments used by Docker and Singularity.

Parameters:

• image (str) – Name of the container image

• tag (str) – Tag of the container image

• root (str) – Base path for mounting host directories into the container

• xargs (Union[str, List[str]]) – Extra arguments to be passed to the container

bind(loc, mode='ro')

class pydra.engine.environments.Docker(image, tag='latest', root='/mnt/pydra', xargs=None)

Bases: Container

Docker environment.

execute(task)

Execute the task in the environment.

Parameters:

task (TaskBase) – the task to execute

Returns:

Output of the task.

Return type:

output

class pydra.engine.environments.Environment

Bases: object

Base class for environments that are used to execute tasks. Right now it is assumed that the environment, including container images, are available and are not removed at the end TODO: add setup and teardown methods

execute(task)

Execute the task in the environment.

Parameters:

task (TaskBase) – the task to execute

Returns:

Output of the task.

Return type:

output

setup()

teardown()

class pydra.engine.environments.Native

Bases: Environment

Native environment, i.e. the tasks are executed in the current python environment.

execute(task)

Execute the task in the environment.

Parameters:

task (TaskBase) – the task to execute

Returns:

Output of the task.

Return type:

output

class pydra.engine.environments.Singularity(image, tag='latest', root='/mnt/pydra', xargs=None)

Bases: Container

Singularity environment.

execute(task)

Execute the task in the environment.

Parameters:

task (TaskBase) – the task to execute

Returns:

Output of the task.

Return type:

output