pydra.engine.task module

Implement processing nodes.

Notes:

  • Environment specs

    1. neurodocker json

    2. singularity file+hash

    3. docker hash

    4. conda env

    5. niceman config

    6. environment variables

  • Monitors/Audit

    1. internal monitor

    2. external monitor

    3. callbacks

  • Resuming

    1. internal tracking

    2. external tracking (DMTCP)

  • Provenance

    1. Local fragments

    2. Remote server

  • Isolation

    1. Working directory

    2. File (copy to local on write)

    3. read only file system

  • Original implementation

class pydra.engine.task.FunctionTask(func: Callable, audit_flags: AuditFlag = AuditFlag.NONE, cache_dir=None, cache_locations=None, input_spec: SpecInfo | BaseSpec | None = None, cont_dim=None, messenger_args=None, messengers=None, name=None, output_spec: SpecInfo | BaseSpec | None = None, rerun=False, **kwargs)

Bases: TaskBase

Wrap a Python callable as a task element.

class pydra.engine.task.ShellCommandTask(audit_flags: ~pydra.utils.messenger.AuditFlag = AuditFlag.NONE, cache_dir=None, input_spec: ~pydra.engine.specs.SpecInfo | None = None, cont_dim=None, messenger_args=None, messengers=None, name=None, output_spec: ~pydra.engine.specs.SpecInfo | None = None, rerun=False, strip=False, environment=<pydra.engine.environments.Native object>, **kwargs)

Bases: TaskBase

Wrap a shell command as a task element.

DEFAULT_COPY_COLLATION = 2
property cmdline

Get the actual command line that will be submitted Returns a list if the task has a state.

command_args(root=None)

Get command line arguments

get_bindings(root: str | None = None) → dict[str, tuple[str, str]]

Return bindings necessary to run task in an alternative root.

This is primarily intended for contexts when a task is going to be run in a container with mounted volumes.

Parameters:

root (str)

Returns:

bindings – Mapping from paths in the host environment to the target environment

Return type:

dict

input_spec = None
output_spec = None
pydra.engine.task.split_cmd(cmd: str)

Splits a shell command line into separate arguments respecting quotes

Parameters:

cmd (str) – Command line string or part thereof

Returns:

the command line string split into process args

Return type:

str