GitLab CI pipeline for spatial wheels

This page answers one question: how do you express the same GDAL/PROJ wheel matrix in .gitlab-ci.yml — with parallel:matrix for the OS/arch grid, a Docker executor for the manylinux build, cache:key:files for the native dependencies, and a fan-in stage — when GitLab has no cibuildwheel action and macOS/Windows need shell runners? It sits inside the CI Matrix Recipes for Spatial Wheels section of the Modern Python Build Tooling & Wheel Configuration reference, and gives you the complete pipeline plus the GitLab-specific gotchas.

Two-stage GitLab pipeline: parallel build matrix then a collect stage A build stage runs a parallel matrix over OS and architecture, each job using a cache keyed on the native lock file and producing wheel artifacts. A collect stage depends on the build stage, gathers all artifacts, runs twine check, and emits the distribution set. stage: build linux · x86_64 linux · aarch64 macos · arm64 windows · amd64 cache:key:files native.lock stage: collect needs build artifacts twine check → dist/

Context & Root Cause

GitLab CI models a pipeline as ordered stages of jobs, and a matrix is written with parallel:matrix, which expands one job template across variable combinations. The structural difference from GitHub Actions is that GitLab has no first-party cibuildwheel step, so each job runs cibuildwheel (or a raw build) as a script inside a chosen executor — a Docker executor for Linux, and shell runners for macOS and Windows because those cannot run inside a Linux container. Caching uses cache:key:files, which hashes named files to build the key, giving the same “invalidate when GDAL changes” behaviour as GitHub’s hashFiles.

The failure modes are GitLab-flavoured: forgetting needs: makes the collect stage wait for the entire build stage instead of streaming, Docker-in-Docker is needed for the manylinux image unless you run the build directly in a manylinux-based job image, and artifacts expire by default and vanish before the release job runs. The recipe below addresses each. It is the GitLab counterpart to GitHub Actions matrix for GDAL wheel builds.

Solution / Fix

This targets GitLab 16+, a Docker executor for Linux, registered macOS/Windows shell runners, and GDAL 3.8.x.

1. The Linux build job (runs inside a manylinux image) 

stages: [build, collect]

.build-linux:
  stage: build
  image: quay.io/pypa/manylinux_2_28_x86_64   # build directly in the manylinux image
  parallel:
    matrix:
      - ARCH: [x86_64, aarch64]
  cache:
    key:
      files: [ci/native.lock]                 # invalidate when GDAL/PROJ change
    paths: [.cache/native-deps]
  script:
    - pipx run cibuildwheel --platform linux --output-dir wheelhouse
  artifacts:
    paths: [wheelhouse/*.whl]
    expire_in: 1 week                          # long enough for the release job

build:linux:
  extends: .build-linux
  tags: [docker]

2. The macOS and Windows jobs (shell runners) 

build:macos:
  stage: build
  tags: [saas-macos-medium-m1]                 # a registered macOS runner
  script:
    - pipx run cibuildwheel --platform macos --output-dir wheelhouse
  artifacts: { paths: [wheelhouse/*.whl], expire_in: 1 week }

build:windows:
  stage: build
  tags: [shared-windows]
  script:
    - pipx run cibuildwheel --platform windows --output-dir wheelhouse
  artifacts: { paths: [wheelhouse/*.whl], expire_in: 1 week }

3. The collect stage 

collect:
  stage: collect
  image: python:3.12-slim
  needs: [build:linux, build:macos, build:windows]   # stream, don't wait for the stage
  script:
    - mkdir -p dist && cp wheelhouse/*.whl dist/ 2>/dev/null || true
    - pip install twine && twine check dist/*.whl
  artifacts: { paths: [dist/], expire_in: 1 month }

Verification 

# 1. Validate the pipeline definition before pushing (GitLab CLI)
glab ci lint            # expected: "Configuration is valid"

# 2. Reproduce the Linux cell locally in the same image
docker run --rm -v "$PWD:/w" -w /w quay.io/pypa/manylinux_2_28_x86_64 \
  bash -c "pipx run cibuildwheel --platform linux"
# expected: cp39-abi3 manylinux + musllinux wheels in wheelhouse/

# 3. The collect artifact spans every platform family
ls dist/ | grep -oE '(manylinux|musllinux|macosx|win)' | sort -u
# expected: all four present

The Same Grid in GitLab’s Vocabulary

Almost every construct in a spatial wheel pipeline has a direct GitLab equivalent, and the translation is mechanical once the vocabulary lines up. Where the two systems genuinely differ is in how jobs find each other’s artifacts and how the runner executor is chosen.

GitHub Actions constructs mapped to their GitLab CI equivalents A translation table. Strategy matrix include maps to parallel matrix. Runs-on maps to tags selecting a runner. Actions cache maps to the cache keyword with a files key. Upload artifact maps to the artifacts paths keyword. Needs with download artifact maps to needs with artifacts true. If startsWith refs tags maps to a rules clause on the tag variable. Permissions id-token write maps to an id tokens block. Two rows are marked as the real differences: artifact passing is implicit in GitLab, and the container executor is chosen per job by an image keyword. GitHub Actions GitLab CI note strategy.matrix.include parallel:matrix same product semantics runs-on: ubuntu-latest tags: [docker] runner selection by tag actions/cache · key cache:key:files hash of listed files actions/upload-artifact artifacts:paths no action needed needs + download-artifact needs: [{job, artifacts: true}] artifacts arrive automatically if: startsWith(ref, tags) rules: if $CI_COMMIT_TAG same intent permissions: id-token id_tokens: OIDC for publishing container: image image: manylinux_2_28 per-job executor — simpler for manylinux

The two highlighted rows are where GitLab is genuinely more convenient for this kind of build. Artifacts flow along needs edges without an explicit download step, which removes a class of mistake where a fan-in job forgets to fetch one cell’s output. And because the Docker executor selects the image per job, running the whole build inside a manylinux_2_28 container is a one-line declaration rather than a container action with its own argument surface.

Where the Pipeline Needs Care

Three details are specific enough to GitLab that they are worth stating explicitly, because each has produced a broken spatial release somewhere.

Three GitLab-specific pitfalls in a spatial wheel pipeline Three pitfalls with their symptoms and fixes. Artifact expiry defaults can delete wheels before the publish stage runs on a delayed pipeline; set expire_in explicitly on the build jobs. Cache keys default to the branch name, so a release built from a tag misses every cached native build; key on the native lock file instead. Non-Linux targets require shell runners rather than the Docker executor, so macOS and Windows jobs need their own runner tags and their own repair tooling installed. artifact expiry symptom: the publish job finds an empty dist/ on a pipeline that waited for a manual approval fix: set expire_in explicitly on every build job — the default is shorter than a release cycle cache key defaults symptom: tag pipelines take fifteen minutes per cell while branch pipelines take ninety seconds fix: key the cache on the native lock file, not on the default $CI_COMMIT_REF_SLUG non-Linux runners symptom: macOS and Windows cells cannot start, because the Docker executor cannot host them fix: shell runners with their own tags, and delocate/delvewheel installed on the runner image

The cache-key default is the one that costs the most and is hardest to notice, because branch pipelines — the ones developers watch — behave perfectly while the tag pipeline that actually produces releases recompiles everything from scratch.

Pitfalls & Alternatives

Artifacts expiring before release. GitLab expires job artifacts (default 30 days, often less on self-hosted). A tag pipeline that builds today and releases after review can find the wheels gone. Set expire_in explicitly on every build job.

Omitting needs:. Without needs:, the collect stage waits for the slowest build job’s whole stage and cannot start early; worse, it may run even if a build job you did not list failed. List explicit needs: so the DAG is correct.

Running manylinux via Docker-in-Docker unnecessarily. Building inside a manylinux job image: avoids DinD entirely — only reach for docker:dind if you must build a custom image in-pipeline. For that custom-image path, see cibuildwheel vs manual Docker matrix for GDAL wheels.

Frequently Asked Questions

Which executor should the Linux build jobs use?

The Docker executor, with the manylinux image named directly in the job’s image keyword. That gives you the curated toolchain and the auditwheel policy files without any container plumbing of your own, and it makes the image a reviewable, pinnable line in the pipeline file. Shell executors are for the platforms Docker cannot host — macOS and Windows — where the runner itself has to carry the toolchain.

How do artifacts move between stages?

Automatically along needs edges when artifacts: true is set, which is the main ergonomic advantage over a workflow where each consumer downloads explicitly. The corollary is that artifact expiry becomes load-bearing: if a pipeline waits for a manual approval longer than the expiry window, the publish job finds an empty directory. Set expire_in explicitly on every job that produces wheels rather than relying on the instance default.

What is the GitLab equivalent of trusted publishing?

An id_tokens block that requests a JSON Web Token with the appropriate audience, which the publish job then presents to PyPI in place of an API token. The registered publisher names the GitLab instance, the project path and the ref, and each of those is compared literally — so a project moved between groups needs its publisher updated, which is the usual cause of a first failure after a reorganisation.

Can parallel:matrix express the same grid as a GitHub matrix include?

Yes, with one difference worth knowing: parallel:matrix produces the full product of the lists you give it, so excluding specific combinations means either restructuring the lists or adding a rules clause that skips the unwanted cells. Generating the matrix from the shared platform file avoids the question entirely, because the file lists the cells you want rather than the axes to multiply.

Why is the tag pipeline slower than the branch pipeline?

Almost always the cache key. The default key is derived from the ref, so a tag pipeline looks for entries that were written under a branch name and finds nothing, recompiling GDAL from scratch in every cell. Keying the cache on the native lock file instead makes the two pipelines share entries, and printing the resolved key in the job log makes the mismatch obvious the first time it happens.

How should a self-hosted runner fleet be organised for this?

By tag, matching the platform list: a docker tag for the Linux cells, a macos-arm64 tag for the macOS runner, a windows tag for the Windows one. Keep the repair tooling — delocate, delvewheel — installed on the runner images rather than installing it per job, and pin their versions alongside the native library pins so a runner rebuild does not silently change how wheels are repaired.

Should the pipeline publish from GitLab if the project also builds on GitHub?

Build in both if you want redundancy, but publish from exactly one. Two pipelines uploading the same version race, and the loser fails with an error that reads like a bug. Record which system is authoritative in the repository so nobody has to reconstruct it during a release.

Where should the platform list live in a GitLab-only project?

Still in its own file rather than inline in the pipeline. Even without a second CI system to keep in step, a machine-readable list lets the validation job iterate the same cells the build produced, and lets a script assert that the collected artifact set matches the declared platforms — the check that catches a silently missing wheel.