Warning: This document is for an old version of Pydra: A simple dataflow engine with scalable semantics. The main version is master.

pydra.engine.task module

Implement processing nodes.

Notes:

• Environment specs

  1. neurodocker json

  2. singularity file+hash

  3. docker hash

  4. conda env

  5. niceman config

  6. environment variables

• Monitors/Audit

  1. internal monitor

  2. external monitor

  3. callbacks

• Resuming

  1. internal tracking

  2. external tracking (DMTCP)

• Provenance

  1. Local fragments

  2. Remote server

• Isolation

  1. Working directory

  2. File (copy to local on write)

  3. read only file system

• Original implementation

class pydra.engine.task.FunctionTask(func: Callable, audit_flags: AuditFlag = AuditFlag.NONE, cache_dir=None, cache_locations=None, input_spec: SpecInfo | BaseSpec | None = None, cont_dim=None, messenger_args=None, messengers=None, name=None, output_spec: SpecInfo | BaseSpec | None = None, rerun=False, **kwargs)

Bases: TaskBase

Wrap a Python callable as a task element.

class pydra.engine.task.ShellCommandTask(audit_flags: ~pydra.utils.messenger.AuditFlag = AuditFlag.NONE, cache_dir=None, input_spec: ~pydra.engine.specs.SpecInfo | None = None, cont_dim=None, messenger_args=None, messengers=None, name=None, output_spec: ~pydra.engine.specs.SpecInfo | None = None, rerun=False, strip=False, environment=<pydra.engine.environments.Native object>, **kwargs)

Bases: TaskBase

Wrap a shell command as a task element.

DEFAULT_COPY_COLLATION = 2

property cmdline

Get the actual command line that will be submitted Returns a list if the task has a state.

command_args(root=None)

Get command line arguments

get_bindings(root: str | None = None) → dict[str, tuple[str, str]]

Return bindings necessary to run task in an alternative root.

This is primarily intended for contexts when a task is going to be run in a container with mounted volumes.

Parameters:

root (str)

Returns:

bindings – Mapping from paths in the host environment to the target environment

Return type:

dict

input_spec = None

output_spec = None

pydra.engine.task.split_cmd(cmd: str)

Splits a shell command line into separate arguments respecting quotes

Parameters:

cmd (str) – Command line string or part thereof

Returns:

the command line string split into process args

Return type:

str