Eventline is now open source and available on GitHub !
Pipelines are instantiated either when triggers are activated or when commands are executed. In this section, we will examine how it happens and how we can interact with the pipelines.
Triggers are activated on new events. For each event, each pipeline defined in the trigger resource is instantiated.
Instantiating a pipeline is simply instantiating every one of its task. Once created, the Eventline scheduler will use the list of dependencies in the pipeline resource to determine which tasks can be executed.
When all tasks have been either terminated (successfully or not) or aborted, the pipeline will also be considered terminated.
Commands are an alternative way to execute commands. Users can execute them at any moment, using the web interface or the evcli command line tool.
Pipelines associated with a command are immediately instantiated when the command is executed, and they are then handled the exact same way as they are for triggers.
Pipelines instantiated from triggers are associated with the event which
activated the trigger. For commands, we generate an event of type
command.execution containing the set of parameters used among other things.
Pipelines instantiated from the command will have access to this event during
A pipeline can have one of the following statuses:
Created or started pipelines can be manually aborted, using either the web interface of the evcli command line tool. When that happens, any running task is interrupted, and remaining task are immediately marked as aborted.
Note that due to the concurrent nature of task execution, it is possible for a task to complete (successfully or not) after you act to abort a pipeline, but before the pipeline is aborted. In that case, the abortion could fail.
Any successful, aborted or failed pipeline can be manually restarted, using
the web interface or the evcli command line tool. On restart, the state of the
pipeline and of all of its tasks are reset, and they are all marked as
created. Once scheduled, each task will use the exact same resource data as
the previous execution(s), even if the associated resources have been modified
in the mean time.
Note that identities are not resources: each identity used by a task is read at the moment the task is executed. Therefore a pipeline running a second time after a restart could still act differently if identities have been modified.
Eventline is a scheduling platform: tasks execute code which usually affect external systems. Restarting a pipeline could have unexpected consequences regarding these systems.
In general, writing tasks in an idempotent way will help a lot in keeping your technical processes robust.
In addition to the normal restart procedure, aborted and failed pipelines can
restart from failure. Tasks which where not successful and tasks which depend
on them will be reset, and will therefore be marked as
tasks will not be affected.
The scratchpad is a data storage system made available to each instantiated pipeline. Its main purpose is to allow data sharing between tasks.
A scratchpad contains multiple entries, each one identified by a key. The key itself is an arbitrary character string (with a limit of 100 bytes). Each entry contains a value, which is also a character string of up to 4000 bytes.
A scratchpad is created when a pipeline is instantiated, and initially does not contain any entry.
Users can interact with the scratchpad either through the API or using the evcli client. It is possible to create new entries, update existing ones, and delete them at any moment.
When a pipeline is restarted from the start, all entries in the scratchpad are deleted.
When a pipeline is retarted from failure, the scratchpad is kept as it was when the pipeline failed.