Pipeline¶

Module contents¶

Definition of the Node, Pipe, and Pipeline models.

Submodules¶

django_analyses.models.pipeline.node module¶

Definition of the Node class.

class django_analyses.models.pipeline.node.Node(*args, **kwargs)¶

Bases: django_extensions.db.models.TimeStampedModel

A Model representing a single pipeline node in the database. A node is simply a reference to some distinct configuration of a particular analysis version. Nodes are the building blocks of a Pipeline, and Pipe instances are used to attach them to one another.

analysis_version¶: The analysis version this node holds a configuration for.

configuration¶: The configuration of the analysis version ran when executing this node.

get_configuration() → dict¶

Undo any changes made to the node’s configuration for serialization before passing them on to the interface.

Returns:	Node’s analysis version configuration
Return type:	dict

get_full_configuration(inputs: dict = None) → dict¶

Returns a “full” input configuration, combining any provided inputs with this node’s configuration and any default values configured in this analysis version’s input specification.

Parameters:	inputs (dict) – User provided inputs for the execution of the node
Returns:	Full configuration to pass to the interface
Return type:	dict

get_required_nodes(pipeline=None, run_index: int = None) → django.db.models.query.QuerySet¶

Returns a queryset of Node instances that are a previous step in some Pipeline instance (i.e. there is a pipe in which the retuned nodes are the source and this node is the destination).

Parameters:	pipeline (`Pipeline`) – A pipeline instance to filter the pipe set with, optional run_index (int) – If this node is executed more than once in a given pipeline, filter by the index of the node’s run, optional
Returns:	Required nodes
Return type:	models.QuerySet

get_requiring_nodes(pipeline=None, run_index: int = None) → django.db.models.query.QuerySet¶

Returns a queryset of Node instances that are the next step in some Pipeline instance (i.e. there is a pipe in which the retuned nodes are the destination and this node is the source).

Parameters:	pipeline (`Pipeline`) – A pipeline instance to filter the pipe set with, optional run_index (int) – If this node is executed more than once in a given pipeline, filter by the index of the node’s run, optional
Returns:	Requiring nodes
Return type:	models.QuerySet

get_run_set() → django.db.models.query.QuerySet¶

Returns all the existing Run instances that match this node’s configuration value.

Returns:	Existing node runs
Return type:	models.QuerySet

is_entry_node(pipeline) → bool¶

Determines whether this node is an entry point of the specified pipeline by checking if the first run of this node has any dependencies (i.e. has any pipes leading to it).

Parameters:	pipeline (`Pipeline`) – The pipeline to check
Returns:	Whether this node is an entry point of the given pipeline or not
Return type:	bool

objects = <django.db.models.manager.Manager object>¶

pipe_destination_set¶

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

pipe_source_set¶

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

required_nodes¶

Returns the queryset of required nodes returned by calling get_required_nodes(), or None if it is empty.

Returns:	Required nodes
Return type:	models.QuerySet

See also

get_required_nodes()

requiring_nodes¶

Returns the queryset of requiring nodes returned by calling get_requiring_nodes(), or None if it is empty.

Returns:	Requiring nodes
Return type:	models.QuerySet

See also

get_requiring_nodes()

run(inputs: Union[Dict[str, Any], List[Dict[str, Any]], Tuple[Dict[str, Any]]], user: django.contrib.auth.models.User = None, return_created: bool = False) → Union[django_analyses.models.run.Run, List[django_analyses.models.run.Run], Tuple[django_analyses.models.run.Run, bool], List[Tuple[django_analyses.models.run.Run, bool]]]¶

Run this node (the interface associated with this node’s analysis_version) with the given inputs.

Parameters:	inputs (dict) – Any user-provided inputs to pass to the interface user (User, optional) – The user creating this run, by default None return_created (bool) – Whether to also return a boolean indicating if the run already existed in the database or created, defaults to False
Returns:	The created or retreived run instance/s
Return type:	Union[Run, List[Run], Tuple[Run, bool], List[Tuple[Run, bool]]]

save(*args, **kwargs)¶: Overrides the model’s save() method to provide custom validation.

Hint

For more information, see Django’s documentation on overriding model methods.

validate() → None¶: Validates that the assigned configuration matches the chosen analysis version’s input specification.

django_analyses.models.pipeline.pipe module¶

Definition of the Pipe class.

class django_analyses.models.pipeline.pipe.Pipe(*args, **kwargs)¶

Bases: django.db.models.base.Model

A Model representing a directed association between one Node’s output and another Node’s input within the context of a particular Pipeline.

base_destination_port¶

An input definition of the destination node that will be provided some input by the source node.

Note

This field holds the reference to the base InputDefinition instance.

See also

destination_port

base_source_port¶

An output definition of the source node that will provide some input of the destination node.

Note

This field holds the reference to the base OutputDefinition instance.

See also

source_port

destination¶: The destination Node, i.e. a node that will be provided an input from the source node.

destination_port¶

Returns the InputDefinition subclass of the assigned base_destination_port.

Returns:	The destination input definition
Return type:	InputDefinition

destination_run_index¶: If the destination node has multiple executions within the pipline, this attribute determines the index of the execution that will be used.

index¶: The index field is used to listify arguments in transit between nodes. An integer indicates expected input’s index in the destination ListInput, and None indicates the index doesn’t matter.

objects = <django_analyses.models.managers.pipe.PipeManager object>¶

pipeline¶: The Pipeline instance to which this pipe belongs.

source¶: The source Node, i.e. a node that will provide some input for the destination node.

source_port¶

Returns the OutputDefinition subclass of the assigned base_source_port.

Returns:	The source output definition
Return type:	OutputDefinition

source_run_index¶: If the source node has multiple executions within the pipline, this attribute determines the index of the execution that will be used.

django_analyses.models.pipeline.pipeline module¶

Definition of the Pipeline class.

class django_analyses.models.pipeline.pipeline.Pipeline(*args, **kwargs)¶

Bases: django_extensions.db.models.TitleDescriptionModel, django_extensions.db.models.TimeStampedModel

A pipeline essentially represents a set of Pipe instances constituting a distinct analysis procedure.

class Meat¶

Bases: object

ordering = ('title',)¶

count_node_runs(node: django_analyses.models.pipeline.node.Node) → int¶

Returns the number of times a particular node is meant to run during the execution of this pipeline.

Parameters:	node (Node) – Node to count
Returns:	Number of separate runs of node within this pipeline
Return type:	int

entry_nodes¶

Returns the “entry” node/s of this pipeline.

Returns:	Entry nodes
Return type:	list

See also

get_entry_nodes()

get_entry_nodes() → list¶

Returns the “entry” node/s of this pipeline, i.e. nodes that are a source of some Pipe but not a destination in any.

Returns:	List of `Node` instances
Return type:	list

get_node_set() → django.db.models.query.QuerySet¶

Returns all Node instances used in this pipeline.

Returns:	Pipeline nodes
Return type:	`QuerySet`

node_set¶

Returns all Node instances used in this pipeline.

Returns:	Pipeline nodes
Return type:	QuerySet

See also

get_node_set()

objects = <django_analyses.models.managers.pipeline.PipelineManager object>¶

pipe_set¶

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.