Eventline is now open source and available on GitHub !

If you are still using the old Eventline platform, contact us to migrate to the new service, and head to the new documentation website for more information.

Pipelines

Introduction

Pipelines are used to define graphs of tasks which are executed as a single unit. Once a pipeline is defined, it can be instantiated either when a trigger is activated or when a command is called.

Data schema

Pipeline data contain the following fields:

concurrent (boolean)
True if multiple instances of the pipelines can be executed at the same time, or false else. (optional, defaults to false).
retention (integer)
The number of days after which pipeline execution data are automatically deleted (optional). Note that pipeline retention must be lower or equal to the retention of the organization plan.
tasks (array)
The set of tasks which are part of the pipeline.

Each task is an object containing the following fields:

name (string)
The name of the task in this pipeline (optional). This is only necessary when this task must be referenced in a dependency list of another task. If not set, the name will be automatically generated.
label (string)
A description of the task (optional). This field is used in the web interface and can help document the actual use of the task.
task (string)
The name of the task resource.
parameters (object)
The set of parameters used for the task. See task parameters for more information (optional).
dependencies (array)
The list of names of the tasks which must be executed before this one in the pipeline (optional).
on_failure (string)
The behavior of the pipeline if the task does not succeed. Can be either abort or continue (optional, defaults to abort).
nb_instances (integer)
The number of times to instantiate the task in the pipeline (optional, defaults to 1). Must be lower or equal to 10. This can be used to parallelize multiple similar tasks.
nb_retries (integer)
The number of times to retry a task if it fails (optional, defaults to 0). Must be lower or equal to 10.
retry_delay (integer)
The number of seconds to wait before scheduling a task during a retry (optional, defaults to 0). Must be lower or equal to 600 (10 minutes).

Example

type: "pipeline"
version: 1
name: "generate-db-report"
data:
  tasks:
    - name: "export-primary"
      task: "export-db-stats"
      parameters:
        database: "primary"
    - name: "export-marketing"
      task: "export-db-stats"
      parameters:
        database: "marketing"
    - name: "generate"
      task: "generate-db-report"
      dependencies:
        - "export-primary"
        - "export-marketing"

In this example, we want to export statistics from two database and generate a global report. Therefore the generate task, based on the generate-db-report task resource, depends on both export-primary and export-marketing. These two tasks are based on the same export-db-stats task but with different parameters.

Table of content