Eventline is now open source and available on GitHub !

If you are still using the old Eventline platform, contact us to migrate to the new service, and head to the new documentation website for more information.

Tasks

Introduction

Tasks are instantiated at the same time as their pipeline. As soon as they are created, the Eventline scheduler will determine which tasks can be executed, and will start them in the appropriate runtime environment.

Lifecycle

A task can have one of the following statuses:

created
The task has just been created and is waiting for scheduling.
started
The task is currently being executed.
aborted
The task was stopped before completion, usually by a user aborting the pipeline.
successful
The task has completed successfully.
failed
The task has failed, usually because one of the steps failed.

Retries

It is possible to configure tasks to automatically retry when they fail. This is done in the pipeline configuration. For example:

---
type: "pipeline"
version: 1
name: "generate-report"
data:
  tasks:
    - task: "generate-report"
      nb_retries: 3
      retry_delay: 60

Here, the generate-report task will retry up to three times, with a 60 seconds delay between each attempt.

See the pipeline documentation for more information on retry settings.

When a task is retried, it is restarted from the first step, and all previous log data are erased.

This feature is useful when executing tasks which are expected to fail occasionally, for example if they interact with an external platform which is regularly down.

Steps

After a task has been started, the Eventline platform will execute tasks in order.

A step is considered failed if the status code of the program or code fragment executed is different from 0. When that happens, the task is marked as failed and is immediately interrupted, meaning that remaining steps will not be executed.

During execution, the output of each step is collected by the platform. You can visualize them on the web interface.

Runtime context

When Eventline start a task, it injects various data into the runtime environment. These data are bundled in a structure called the runtime context, and are available as a file during task execution.

The file is located at /eventline/task/context. It contains a single JSON object with the following fields:

event (object)
The event associated with the current pipeline. See the API documentation for more information.
task_parameters (object)
The set of parameters passed to the task in the pipeline resource definition, associating the name of the parameter to its value.
instance_id (integer)
When the task has multiple instances, the identifier of the instance, from 1 to the number of instances. If the task has a single instance, the identifier is set to 1. See the nb_instances field in the pipeline resource documentation for more information.
identities (object)
For each identity listed in the task resource, the set of data associated with it. These data depend on the connector and type of each identity. See the documentation of the identity on the connector page for more information.

Event

During execution, all event data fields are made available as files in /eventline/event/<field>, where <field> is the name of the data field.

For example, for a push event from the github connector, the revision field is available in /eventline/event/revision.

Identities

During execution, all identities data fields are also made available as files in /eventline/identities/<name>/<field>, where <name> is the name of the identity in the task resource, and <field> is the name of the data field.

For example, if a task references an identity named database of type generic.password, the password field of this identity will be available as file located at /eventline/identities/database/password.

Example

{
  "event": {
    "connector": "github",
    "creation_time": "2021-10-20T17:24:48Z",
    "data": {
      "branch": "master",
      "new_revision": "41d71893cfe266d82c28dcc395e7bf6e0407b3cf",
      "old_revision": "e72c9f45b666ce8cce3df67536b63ce7867f8152",
      "organization": "example",
      "repository": "my-project"
    },
    "event_time": "2021-10-20T17:24:47Z",
    "id": "1zmQGafbFRej9ThnY8BBVgh1DoE",
    "name": "push",
    "org_id": "1zY1y6offsPNwvhFxgpteVO0GvG",
    "trigger_id": "1zY3VcZXaI36Nfi0XSMyW2GL7Ya"
  },
  "identities": {
    "github": {
      "access_token": "gho_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
      "username": "alice"
    }
  },
  "instance_id": 1,
  "task_parameters": {
    "my-value": 42
  }
}

This example shows the structure of a runtime context for a fictional github.push event. A single identity named github was referenced in the task resource, and a single parameter, my-value, was set to the value 42.

The event contains information associated with the push.

Environment

Eventline injects several environment variables during the execution of each task:

  • EVENTLINE: always set to true, indicating that the task is being executed in the Eventline platform.
  • EVENTLINE_PROJECT_ID: the identifier of the current project.
  • EVENTLINE_PROJECT_NAME: the name of the current project.
  • EVENTLINE_PIPELINE_ID: the identifier of the current pipeline.
  • EVENTLINE_TASK_ID: the identifier of the current task.
  • EVENTLINE_CONTEXT_PATH: the path of the context file. The value will always be /eventline/task/context; the environment variable is provided to make it easier to test scripts using a mock context file out of the platform.
Info
The EVENTLINE environment variable is ideal when testing task scripts: you can behave differently depending on the presence or absence of this variable.