pydra.engine.state module

Keeping track of mapping and reduce operations over tasks.

class pydra.engine.state.State(name, splitter=None, combiner=None, other_states=None)

Bases: object

A class that specifies a State of all tasks.

It’s only used when a task have a splitter.

It contains all information about splitter, combiner, final splitter, and input values for specific task states (specified by the splitter and the input).

It also contains information about the final groups and the final splitter if combiner is available.

name

name of the state that is the same as a name of the task

Type:: str

splitter

can be a str (name of a single input), tuple for scalar splitter, or list for outer splitter

Type:: str, tuple, list

splitter_rpn_compact

splitter in RPN, using a compact notation for splitter from previous states, e.g. _NA

Type:: list

splitter_rpn

splitter represented in RPN, unwrapping splitters from previous states

Type:: list

combiner

list of fields that should be combined (order is not important)

Type:: list

splitter_final: final splitter that includes the combining process

other_states

used to create connections with previous states:

{
  name of a previous state:
    (previous state, input from current state needed the connection)
}

Type:: dict

inner_inputs

used to create connections with previous states {"{self.name}.input name for current inp": previous state}

Type:: dict

states_ind

dictionary for every state that contains indices for all state inputs (i.e. inputs that are part of the splitter)

Type:: list of dict

states_val

dictionary for every state that contains values for all state inputs (i.e. inputs that are part of the splitter)

Type:: list of dict

inputs_ind

dictionary for every state that contains indices for all task inputs (i.e. inputs that are relevant for current task, can be outputs from previous nodes)

Type:: list of dict

group_for_inputs

specifying groups (axes) for each input field (depends on the splitter)

Type:: dict

group_for_inputs_final

specifying final groups (axes) for each input field (depends on the splitter and combiner)

Type:: dict

groups_stack_final

specify stack of groups/axes (used to determine which field could be combined)

Type:: list

final_combined_ind_mapping

mapping between final indices after combining and partial indices of the results

Type:: dict

property combiner: the combiner associated to the state.

combiner_validation(): validating if the combiner is correct (after all states are connected)

property current_combiner: the current part of the combiner, i.e. the part that is related to the current task’s state only (doesn’t include fields propagated from the previous tasks)

property current_combiner_all: the current part of the combiner including all the fields that should be combined (i.e. not only the fields that are explicitly set, but also the fields that re in the same group/axis and had to be combined together, e.g., if splitter is (a, b) a and b has to be combined together)

property current_splitter: the current part of the splitter, i.e. the part that is related to the current task’s state only (doesn’t include fields propagated from the previous tasks)

property current_splitter_rpn: the current part of the splitter using RPN

property inner_inputs: specifies connections between fields from the current state with the specific state from the previous states, uses dictionary {input name for current state: the previous state}

property other_states: specifies the connections with previous states, uses dictionary: {name of a previous state: (previous state, input field from current state)}

prepare_inputs()

Preparing inputs indices, merges input from previous states.

Includes indices for fields from inner splitters (removes elements connected to the inner splitters fields).

prepare_states(inputs, cont_dim=None)

Prepare a full list of state indices and state values.

State Indices: number of elements depends on the splitter
State Values: specific elements from inputs that can be used running interfaces

Parameters:

inputs (dict) – inputs of the task
cont_dim (dict or None) – container’s dimensions for a specific input’s fields

prepare_states_combined_ind(elements_to_remove_comb)

Prepare the final list of dictionaries with indices after combiner.

Parameters:: elements_to_remove_comb (list) – elements of the splitter that should be removed due to the combining

prepare_states_ind()

Calculate a list of dictionaries with state indices.

Uses hlpst.splits.

prepare_states_val(): Evaluate states values having states indices.

property prev_state_combiner: the prev-state part of the combiner, i.e. the part that comes from the previous tasks’ states

property prev_state_combiner_all: the prev-state part of the combiner including all the fields that should be combined (i.e. not only the fields that are explicitly set, but also the fields that re in the same group/axis and had to be combined together, e.g., if splitter is (a, b) a and b has to be combined together)

property prev_state_splitter: the prev-state part of the splitter, i.e. the part that comes from the previous tasks’ states

property prev_state_splitter_rpn: the prev-state art of the splitter using RPN

property prev_state_splitter_rpn_compact: the prev-state part of the splitter using RPN in a compact form, (without unwrapping the states from previous nodes), e.g. [_NA, _NB, *]

set_input_groups(state_fields=True)

Evaluates groups, especially the final groups that address the combiner.

Parameters:: state_fields (bool) – if False the splitter from the previous states are unwrapped

splits(splitter_rpn)

Splits input variable as specified by splitter

Parameters:

splitter_rpn (list) – splitter in RPN notation

Returns:

splitter (list) – each element contains indices for input variables
keys (list) – names of input variables

property splitter: Get the splitter of the state.

property splitter_final: the final splitter, after removing the combined fields

property splitter_rpn: splitter in RPN

property splitter_rpn_compact: splitter in RPN with a compact representation of the prev-state part (i.e. without unwrapping the part that comes from the previous states), e.g., [_NA, _NB, *]

property splitter_rpn_final

splitter_validation(): validating if the splitter is correct (after all states are connected)

update_connections(new_other_states=None, new_combiner=None)

updating connections, can use a new other_states and combiner

Parameters:

new_other_states (dict, optional) – dictionary with new other_states, will be set before updating connections
new_combiner (str, or list, optional) – new combiner