pydra.engine.submitter module

Handle execution backends.

class pydra.engine.submitter.Submitter(plugin: str | Type[Worker] = 'cf', **kwargs)

Bases: object

Send a task to the execution backend.

close()

Close submitter.

Do not close previously running loop.

async expand_runnable(runnable, wait=False, rerun=False)

This coroutine handles state expansion.

Removes any states from runnable. If wait is set to False (default), aggregates all worker execution coroutines and returns them. If wait is True, waits for all coroutines to complete / error and returns None.

Parameters:

runnable (pydra Task) – Task instance (Task, Workflow)
wait (bool (False)) – Await all futures before completing

Returns:

futures – Coroutines for TaskBase execution.

Return type:

set or None

async expand_workflow(wf, rerun=False)

Expand and execute a stateless Workflow. This method is only reached by Workflow._run_task.

Parameters:: wf (Workflow) – Workflow Task object
Returns:: wf – The computed workflow
Return type:: pydra.engine.core.Workflow

async submit_from_call(runnable, rerun, environment)

This coroutine should only be called once per Submitter call, and serves as the bridge between sync/async lands.

There are 4 potential paths based on the type of runnable: 0) Workflow has a different plugin than a submitter 1) Workflow without State 2) Task without State 3) (Workflow or Task) with State

Once Python 3.10 is the minimum, this should probably be refactored into using structural pattern matching.

pydra.engine.submitter.get_runnable_tasks(graph): Parse a graph and return all runnable tasks.

pydra.engine.submitter.is_runnable(graph, obj): Check if a task within a graph is runnable.

async pydra.engine.submitter.prepare_runnable_with_state(runnable)