Skip to content

GitLab CI/CD pipelines: from first job to production

GitLab CI/CD runs pipelines defined in a .gitlab-ci.yml file at the root of a repository. Each push, merge request, tag or schedule creates a pipeline; GitLab Runner executes its jobs, usually in containers, and reports status back to the commit and merge request. This page takes one invented application, OrdersPortal (an ASP.NET Core API and a Vite front end), from a first pipeline to a reusable, scheduled, failure-tolerant setup with notifications.

The facts on this page were checked against the GitLab documentation source (gitlab-org/gitlab, doc/ci/) in November 2026, when GitLab 19.x was current. The YAML samples were checked for YAML syntax only; they were not run on a GitLab instance. In a real project, validate them with the pipeline editor or Build > Pipeline editor > Validate, which uses the CI Lint API.

Topic What it answers
Pipeline basics Pipelines, stages, jobs, runners and a first .gitlab-ci.yml.
Rules and workflow Which pipelines run, and which jobs run in them.
Variables artifacts and cache Passing configuration, files and dependencies between jobs.
Templates and reuse default, extends, anchors, !reference, include and CI/CD components.
Scheduled pipelines Running pipelines on a cron schedule, and what they run as.
Failure handling retry, allow_failure, when: on_failure, after_script, timeouts and auto-cancel.
Notifications Email, Slack and webhook notifications for pipeline events.

A pipeline is a set of jobs grouped into stages. By default, jobs in the same stage run in parallel, and a stage starts only when the previous stage succeeded. A job is a script plus the settings to run it: an image, variables, rules, artifacts. Runners pick up jobs; GitLab.com provides hosted runners, and self-managed instances register their own with tags that jobs can select.

needs turns the stage order into a directed acyclic graph: a job with needs: [build_api] starts as soon as build_api finishes, without waiting for the rest of its stage. It is the main tool for shortening pipelines.

.gitlab-ci.yml
stages: [build, test, package, deploy]
default:
image: mcr.microsoft.com/dotnet/sdk:10.0
interruptible: true
build_api:
stage: build
script:
- dotnet publish src/Orders.Api -c Release -o publish
artifacts:
paths: [publish/]
expire_in: 1 day
test_api:
stage: test
needs: [build_api]
script:
- dotnet test --logger "junit;LogFilePath=test-results/{assembly}.xml"
artifacts:
when: always
reports:
junit: test-results/*.xml
build_web:
stage: build
image: node:24-alpine
script:
- npm ci
- npm run build
artifacts:
paths: [dist/]
package_image:
stage: package
image: docker:29
services: [docker:29-dind]
needs: [test_api]
script:
- echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" --password-stdin "$CI_REGISTRY"
- docker build -t "$CI_REGISTRY_IMAGE/orders-api:$CI_COMMIT_SHORT_SHA" .
- docker push "$CI_REGISTRY_IMAGE/orders-api:$CI_COMMIT_SHORT_SHA"
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
deploy_staging:
stage: deploy
needs: [package_image]
environment:
name: staging
url: https://staging.orders.example.test
script:
- ./deploy.sh staging "$CI_COMMIT_SHORT_SHA"
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

The JUnit logger used by test_api comes from the JunitXml.TestLogger NuGet package referenced by the test project. artifacts:reports:junit makes test results appear in the merge request widget, and environment records deployments so the Operate > Environments page shows what is running where. The package_image job uses Docker-in-Docker, which needs a runner configured for privileged containers; Kaniko-style or BuildKit rootless builders are alternatives where privileged runners are not allowed.

Pitfalls:

  • Everything in one long stage chain. Without needs, a slow front-end build delays API tests that do not depend on it.
  • Unpinned images. image: node changes under you. Pin a version, as with node:24-alpine.

rules decides whether a job is added to a pipeline. GitLab evaluates the rules in order and uses the first match; each rule can test variables (if), changed files (changes) or file existence (exists), and set when (on_success, manual, delayed, always, never), allow_failure or variables. If no rule matches, the job is not added. The older only and except keywords are deprecated in favor of rules, and the two cannot be mixed in one job.

workflow:rules applies the same logic to the whole pipeline. The documentation’s recommended pattern runs merge request pipelines for merge requests and branch pipelines only for the default branch, which avoids duplicate pipelines when a branch has an open merge request:

workflow:
rules:
- if: $CI_COMMIT_TITLE =~ /-draft$/
when: never
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
- if: $CI_PIPELINE_SOURCE == "schedule"
deploy_production:
stage: deploy
script:
- ./deploy.sh production "$CI_COMMIT_SHORT_SHA"
environment: production
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: manual

A manual job defined through rules has allow_failure: false by default, so the pipeline waits for someone to run it. Combine this with protected environments so only approved roles can deploy to production.

rules:changes is useful in monorepos (changes: [src/web/**/*] builds the front end only when it changed), but on branch pipelines without a merge request it compares against the previous push, not the target branch, so the result can surprise you. Use rules:changes:compare_to when you need a fixed comparison point.

Pitfalls:

  • Duplicate pipelines. Rules that match both push and merge_request_event for the same branch create two pipelines per commit. Use workflow:rules as above.
  • A rule list with no fallback. If the last rule is conditional and none match, the job silently disappears. Read the rules top to bottom as an if/else if chain.

CI/CD variables carry configuration into jobs. GitLab predefines many (CI_COMMIT_SHA, CI_DEFAULT_BRANCH, CI_REGISTRY_IMAGE, CI_PIPELINE_SOURCE). You define others in .gitlab-ci.yml (variables: at the top level or per job) for non-secret values, and in Settings > CI/CD > Variables at project, group or instance level for secrets. Mark secrets as masked (hidden in job logs) and protected (available only to pipelines on protected branches and tags), so a feature branch cannot print the production password. For cloud credentials, prefer id_tokens with OIDC federation over stored keys. GitLab also recommends spec:inputs over variables for configuring reusable configuration and scheduled pipelines, because inputs are typed and validated.

Artifacts are files a job produces and GitLab stores: build output passed to later jobs, test reports, coverage. Jobs download artifacts from earlier stages automatically, or only from the jobs listed in needs. Set expire_in so storage does not grow without bound.

Cache is for speeding up jobs by reusing dependencies between pipelines, such as the npm or NuGet download cache. It is best-effort: a job must still work when the cache is empty. Key it on the lock file so it changes only when dependencies change:

build_web:
stage: build
image: node:24-alpine
cache:
key:
files: [package-lock.json]
paths: [.npm/]
script:
- npm ci --cache .npm --prefer-offline
- npm run build
artifacts:
paths: [dist/]
expire_in: 1 week

Caching .npm/ (the download cache) rather than node_modules/ keeps npm ci correct while avoiding most downloads.

Pitfalls:

  • Using cache to pass build output between jobs. Cache may be missing or come from another runner; artifacts are guaranteed. Use artifacts for outputs.
  • Secrets in .gitlab-ci.yml. The file is visible to everyone with repository access. Keep secrets in protected, masked variables or an external secrets manager.

Pipelines grow, and copy-paste between jobs and projects is how they rot. GitLab offers reuse at several levels.

Within one file, default sets keywords such as image, before_script, retry and interruptible for every job. Hidden jobs (names starting with a dot) are templates that never run; extends merges a template into a job, and !reference pulls one section of another job into a list, for example a shared login step:

.dotnet_job:
image: mcr.microsoft.com/dotnet/sdk:10.0
before_script:
- dotnet --info
.registry_login:
script:
- echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" --password-stdin "$CI_REGISTRY"
lint_api:
extends: .dotnet_job
stage: test
script:
- dotnet format --verify-no-changes
push_image:
stage: package
image: docker:29
services: [docker:29-dind]
script:
- !reference [.registry_login, script]
- docker push "$CI_REGISTRY_IMAGE/orders-api:$CI_COMMIT_SHORT_SHA"

extends merges hashes deeply but replaces arrays, so a job that sets its own script replaces the template’s. YAML anchors (&name and <<: *name) also work, but only within one file; extends and !reference work across included files.

Across files and projects, include pulls in configuration: include:local for files in the same repository, include:project (with a ref you should pin) for a shared template repository, include:remote for a URL, and include:template for GitLab’s own templates. CI/CD components are the modern form of shared templates: versioned units with declared inputs, published in a component project and discoverable in the CI/CD Catalog.

include:
- component: $CI_SERVER_FQDN/example-org/ci-components/dotnet-test@2.1.0
inputs:
stage: test
project_path: src/Orders.Api.Tests

Inside the component, spec:inputs declares typed inputs and the template references them as $[[ inputs.stage ]]. Pin a released version (or a commit SHA) rather than a moving branch so a change in the shared component cannot break every pipeline at once.

Pitfalls:

  • Including shared templates from main. A bad commit to the template project breaks every consumer. Pin a tag, a SHA or a component version.
  • Expecting extends to append to script. Arrays are replaced; use !reference to combine lists.

Pipeline schedules run a pipeline for a branch or tag at intervals given in cron notation, independent of code changes. Typical uses for OrdersPortal are a nightly full test suite with integration tests, a weekly dependency and container image scan, and periodic cleanup of review environments. Create them in Build > Pipeline schedules: choose an interval pattern, a cron time zone, the target branch, and optional inputs or variables that apply only to scheduled runs. An instance-wide setting limits how often schedules can actually run, whatever the cron says.

Inside the pipeline, $CI_PIPELINE_SOURCE == "schedule" identifies scheduled runs, so jobs can opt in or out:

nightly_integration_tests:
stage: test
script:
- dotnet test tests/Orders.IntegrationTests
rules:
- if: $CI_PIPELINE_SOURCE == "schedule" && $SUITE == "nightly"
deploy_staging:
stage: deploy
script:
- ./deploy.sh staging "$CI_COMMIT_SHORT_SHA"
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: never
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

A schedule runs with the permissions of its owner, the user who created it or last took ownership. That matters for protected branches and environments, and it means a schedule stops working (becomes inactive) when its owner leaves or loses access. Maintainers can take ownership. Schedules also stop when the project is marked for deletion.

Pitfalls:

  • Cron time zone assumptions. Each schedule has its own cron time zone; a “2 AM” job may run during business hours elsewhere. Set it explicitly.
  • Schedules owned by individuals. When the person leaves, the nightly pipeline quietly stops. Assign ownership to a maintainer account policy and review schedules regularly.

Not every failure is the same, and GitLab lets you respond to each kind differently.

  • retry re-runs a failed job up to two times. With retry:when you restrict retries to infrastructure problems, such as runner_system_failure, api_failure or the newer, more specific reasons introduced in GitLab 19.1 (runner_external_dependency_failure, runner_interrupted, stuck_pending_with_matching_runners and others), and with retry:exit_codes to specific exit codes. Retrying script_failure hides flaky tests instead of fixing them.
  • allow_failure: true lets the pipeline continue and pass with a warning when the job fails. allow_failure:exit_codes allows only specific exit codes, for example a scanner that exits 2 for “findings below the threshold”.
  • when: on_failure runs a job only if an earlier stage failed: a place to collect diagnostics or roll back.
  • after_script runs after script, even when the script failed with a script failure or, since GitLab 17.0, when the job was canceled. Use it for cleanup, not for steps that must succeed.
  • timeout caps a job’s run time (it can exceed the project timeout but not the runner’s), so a hung job fails fast instead of holding a runner for an hour.
  • interruptible: true together with the project’s Auto-cancel redundant pipelines setting cancels outdated pipelines on the same ref when a new commit arrives. workflow:auto_cancel:on_new_commit and on_job_failure tune this.
default:
interruptible: true
retry:
max: 2
when: [runner_system_failure, api_failure, runner_external_dependency_failure]
dependency_scan:
stage: test
script:
- ./scan-dependencies.sh
allow_failure:
exit_codes: [2]
e2e_tests:
stage: test
timeout: 20 minutes
script:
- npm run test:e2e
after_script:
- ./collect-browser-logs.sh
rollback_staging:
stage: deploy
script:
- ./deploy.sh staging previous
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: on_failure
deploy_production:
stage: deploy
interruptible: false
script:
- ./deploy.sh production "$CI_COMMIT_SHORT_SHA"
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: manual

rollback_staging sets when: on_failure inside its rule (rules:when), the place to set when for jobs that use rules. Deployment jobs set interruptible: false so a new commit never cancels a deployment halfway through. The documentation also notes that a job with allow_failure: true is always considered successful, so later when: on_failure jobs do not start when it fails.

Pitfalls:

  • retry: 2 on everything. Real test failures take three times as long to report. Retry only infrastructure failure types.
  • allow_failure: true on security scans forever. Everyone learns to ignore the orange warning. Allow specific exit codes and track the findings.

GitLab can tell people about pipeline results in several ways:

  • Personal notifications. Users get email for failed pipelines they triggered, depending on their notification settings.
  • Pipeline status emails integration (Settings > Integrations) sends pipeline results for a project or group to a list of addresses, with options to notify only for broken pipelines, only when the status changes, for child pipelines, and for selected branch types such as the default and protected branches.
  • GitLab for Slack app posts notifications for chosen events, including failed and successful pipelines and deployments, to Slack channels, and offers slash commands. The older Slack notifications integration is deprecated in its favor. Microsoft Teams and other chat tools have their own integrations.
  • Webhooks (Settings > Webhooks) send pipeline, job and deployment events as JSON to any HTTPS endpoint, for custom dashboards or incident tooling.

For OrdersPortal, a sensible setup is pipeline status emails to the team for broken default-branch pipelines only, Slack notifications for deployments and failed pipelines on the default branch, and no notifications for feature branches, where the merge request already shows the status.

Pitfalls:

  • Notifying on every pipeline. People mute the channel. Notify on failures and status changes.
  • Notifications instead of rules. A failing deployment needs a blocked pipeline and a protected environment, not just a Slack message.
You want to Use
Run jobs as soon as their inputs are ready needs
Run the right pipelines without duplicates workflow:rules for merge requests and the default branch
Pass build output to later jobs Artifacts
Speed up dependency installs Cache keyed on the lock file
Share pipeline logic across projects Versioned CI/CD components or pinned include:project
Run nightly or weekly tasks Pipeline schedules plus $CI_PIPELINE_SOURCE == "schedule" rules
Survive flaky infrastructure but not flaky tests retry:when with infrastructure failure types only

What is the difference between artifacts and cache? Artifacts are job outputs stored by GitLab and reliably passed to later jobs and kept for download; cache is a best-effort speed-up shared between pipelines, typically for dependencies, and may be missing.

How do stages and needs interact? Without needs, a job waits for all jobs in earlier stages. With needs, it waits only for the listed jobs, which forms a DAG and lets independent work run in parallel across stages.

How do you avoid duplicate branch and merge request pipelines? Use workflow:rules that create merge request pipelines for merge_request_event and branch pipelines only for the default branch (or only when no merge request is open).

What does a scheduled pipeline run as, and why does it matter? It runs with the permissions of the schedule owner. Access to protected branches, environments and variables depends on that user, and the schedule becomes inactive if the owner loses access.

Senior scenario: OrdersPortal has 30 microservices, each with its own 400-line .gitlab-ci.yml, pipelines take 40 minutes and flaky jobs are retried everywhere. What do you do? Extract the common build, test, scan and deploy logic into versioned CI/CD components with typed inputs, and migrate services to them gradually, pinning versions. Shorten pipelines with needs, lock-file-keyed caches, rules:changes in monorepos and interruptible jobs with auto-cancel. Restrict retry to infrastructure failure types, quarantine and fix flaky tests, and track pipeline duration and failure reasons so the improvements are measured rather than assumed.