Pipelines are used to define graphs of tasks which are executed as a single
unit. Once a pipeline is defined, it can be instantiated either when a trigger
is activated or when a command is called.
Pipeline data contain the following fields:
- True if multiple instances of the pipelines can be executed at the same time, or
false else. (optional, defaults to false).
- The number of days after which pipeline execution data are automatically
deleted (optional). Note that pipeline retention must be lower or equal to
the retention of the organization
- The set of tasks which are part of the pipeline.
Each task is an object containing the following fields:
- The name of the task in this pipeline (optional). This is only necessary
when this task must be referenced in a dependency list of another task. If
not set, the name will be automatically generated.
- A description of the task (optional). This field is used in the web
interface and can help document the actual use of the task.
- The name of the task resource.
- The set of parameters used for the task. See task parameters for more information (optional).
- The list of names of the tasks which must be executed before this one in the
- The behavior of the pipeline if the task does not succeed. Can be either
continue (optional, defaults to
- The number of times to instantiate the task in the pipeline (optional,
defaults to 1). Must be lower or equal to 10. This can be used to parallelize
multiple similar tasks.
- The number of times to retry a task if it fails (optional, defaults to 0).
Must be lower or equal to 10.
- The number of seconds to wait before scheduling a task during a retry
(optional, defaults to 0). Must be lower or equal to 600 (10 minutes).
- name: "export-primary"
- name: "export-marketing"
- name: "generate"
In this example, we want to export statistics from two database and generate a
global report. Therefore the
generate task, based on the
generate-db-report task resource, depends on both
export-marketing. These two tasks are based on the same
task but with different parameters.