How to Read Pipelines
A pipeline is a representations of a workflow. Oxford Dictionary defines workflow as the sequence of industrial, administrative, or other processes through which a piece of work passes from initiation to completion.
Stuff that makes up a pipeline workflow.
Concourse CI uses only a handful of abstract concepts to help you implement the most advanced workflows. Grab a piece of paper and draw the following relationships between concepts to absorb its mental model once and for all.
- The highest-level concept and a container for a workflow is a pipeline.
- A pipeline is made up out of a set of jobs.
- Each job contains one or more steps. Steps are typically sequential, but can be executed in parallel.
- Steps may accept multiple inputs or produce multiple outputs. Anything that can be represented on a filesystem as a directory with files in it can serve as input or output of a step.
- When inputs are passed between steps within a job they can remain just that: inputs/outputs. For passing inputs/outputs between jobs, you must use resources. A resource is an input/output set whose state is retrieved/stored externally by a job, e.g. a git repo or an S3 object.
- A good way to think about a resource is as a feed of versions. Resource Versions are instantiations of the resource. For example, a path to a Github repo could be defined as a resource, but a git repo checked out at a specifc commit is a resource version
How to read a pipeline
Visually, a pipeline is a set of linked jobs (green rectangles) with progress of work flowing through resources (black rectangles) from left to right (build -> test -> deploy),
- A solid line leading from a resource into the job means that resource triggers the execution of the job. For example, a job will trigger on every push to the git repo.
- A dashed line from a resource into a job means that a resource might be used by some of the steps but it's not triggering the job.
- If a resource links two jobs, the job on the right requires the job on the left to successfully complete execution, before it can proceed with a given version.
Click through to the build
job:
Controls, from the top right:
- Encircled plus sign = start a new build of this job the latest inputs.
Clicking this button will result in an integer build number in the bottom
left corner, e.g.
2
- Semi-encircled dotted plus = start a new build of this job without checking
for new inputs. Use inputs from the recently completed build. Clicking
this button results in a subscripted build number, e.g.
2.1