CI Matrix Recipes for Spatial Wheels

A geospatial wheel matrix has more axes than a pure-Python one — interpreter ABI, operating system, CPU architecture, and libc all multiply — and the same conceptual grid must be expressed twice if you support both GitHub Actions and GitLab CI. This guide sits under the Modern Python Build Tooling & Wheel Configuration reference and gives platform teams side-by-side, copy-pasteable matrix definitions for building GDAL/PROJ/GEOS wheels, plus the caching keys and fan-in step that turn a grid of jobs into one publishable set of artifacts. It targets cibuildwheel 3.0+, GitHub Actions and GitLab CI, GDAL 3.8.x, and the abi3 build that collapses the interpreter axis established in C-API vs CPython ABI compatibility.

The spatial wheel build matrix fanning out and folding back in One build trigger fans out into a matrix of jobs across operating system and architecture: Linux x86_64, Linux aarch64, macOS arm64, and Windows amd64. Each job builds and repairs an abi3 wheel. The jobs fold back into a single collect step that gathers all wheels into one artifact set ready for publishing. push / tag one trigger linux · x86_64 linux · aarch64 macos · arm64 windows · amd64 collect wheels one artifact set

Prerequisites & Environment

  • A wheel that builds under cibuildwheel locally, configured per configuring cibuildwheel in pyproject.toml for GDAL — the CI file should be a thin driver around it, not a second source of build logic.
  • Runner access for each target: GitHub-hosted ubuntu-latest, macos-14, windows-latest; GitLab needs a Docker executor for Linux and, for macOS/Windows, shell runners.
  • A cache backend: GitHub actions/cache, GitLab cache: keyed on the native-dependency lock.
# The one command both CI systems ultimately run
pipx run cibuildwheel --output-dir wheelhouse

Core Configuration

The invariant across both systems is: define the OS/arch grid, run the identical cibuildwheel step, key the cache on the native versions, and upload per-job artifacts that a final job collects. Only the YAML dialect differs.

Concern GitHub Actions GitLab CI
Grid definition strategy.matrix.include parallel:matrix
Per-job artifact actions/upload-artifact artifacts:paths
Cache actions/cache keyed on lock hash cache:key:files
Fan-in a needs: job with download-artifact a needs: job in a later stage
aarch64 QEMU or native ARM runner Docker --platform or ARM runner

The two full recipes are GitHub Actions matrix for GDAL wheel builds and GitLab CI pipeline for spatial wheels.

Step-by-Step Implementation

  1. Collapse the interpreter axis first. Build one abi3 wheel per platform (build = "cp39-*"), not one per Python version — this alone cuts the matrix by 4–5×.

  2. Enumerate only the OS/arch cells you ship, and mark fail-fast: false (GitHub) or allow_failure selectively so one arch failing does not cancel the rest.

  3. Key the cache on native versions, not Python. The expensive artifact is compiled GDAL, so the key is gdal3.8-proj9.3-${hashFiles('native.lock')} — the strategy in async build execution and cache strategies.

  4. Fan in. A final job gathers every per-job wheel into one directory and runs a single twine check before publishing.

Verification 

# 1. Enumerate what the matrix will build — no surprises
pipx run cibuildwheel --print-build-identifiers
# expected: one cp39-abi3 identifier per OS/arch cell you declared

# 2. After the fan-in job, the collected set covers every platform
ls dist/ | sed -E 's/.*-(manylinux|musllinux|macosx|win).*/\1/' | sort -u
# expected: manylinux, musllinux, macosx, win — every promised platform present

# 3. Metadata is publishable
python -m twine check dist/*
# expected: PASSED for every wheel

Optimization & Edge Cases

  • Emulated aarch64 dominates wall-clock. A QEMU aarch64 cell can be 10× the native cells; either use a native ARM runner or cross-compile per building aarch64 GDAL wheels without QEMU.
  • Prune before you optimize. Drop cells nobody installs (32-bit, PyPy) rather than speeding them up. Read your download stats first.
  • Cache write only on the default branch so feature branches cannot poison the shared native-build cache.

Troubleshooting

One arch fails and cancels the whole matrix. fail-fast defaults to true on GitHub; set it false so a musllinux break does not discard finished manylinux wheels.

No space left on device on the runner. GDAL’s native build plus Docker layers fills the default runner disk. Prune Docker between steps or use a larger runner; this bites the emulated Linux cells first.

Artifacts collide on upload. Two jobs uploading wheelhouse/*.whl under the same artifact name overwrite each other. Name artifacts per cell (wheels-${os}-${arch}) and merge in the fan-in job.

Counting the Grid Before You Write It

Matrix design is an arithmetic problem before it is a YAML problem. Four axes are available — interpreter, operating system, architecture, and libc — and the naive product of all four is where most spatial pipelines end up by accident. Five interpreters times three operating systems times two architectures times two libc variants is sixty jobs, each of which compiles GDAL from source. At fifteen minutes a job that is fifteen hours of runner time for a release that ships perhaps twelve useful artifacts.

The abi3 decision collapses the interpreter axis to one, which is the single largest saving available and costs you only the CPython-version-specific optimisations that a geospatial binding almost never uses. What remains is a grid of platforms, and platforms are not interchangeable in cost: a native x86_64 Linux job and an emulated aarch64 Linux job differ by an order of magnitude, not a percentage.

Job count and runner cost for four matrix strategies Four strategies compared. The full product of interpreter, operating system, architecture and libc produces sixty jobs and about fifteen hours of runner time. Collapsing the interpreter axis with abi3 gives twelve jobs and about three hours. Replacing emulated aarch64 with a native ARM runner keeps twelve jobs but cuts runner time to about one hour twenty. Pruning unused platforms leaves eight jobs and about fifty-five minutes. runner time per release, by matrix strategy full product 60 jobs ≈ 15 h abi3 collapses interpreters 12 jobs ≈ 3 h native ARM runner 12 jobs ≈ 1 h 20 prune unused platforms 8 jobs ≈ 55 min the emulation axis, not the job count, dominates a spatial build

The order of those interventions is deliberate. Collapsing the interpreter axis is free and reversible. Moving aarch64 off emulation costs either money (a hosted ARM runner) or engineering time (cross-compilation, described in building aarch64 GDAL wheels without QEMU), but it removes the largest single block of wall-clock time. Pruning platforms is last because it is the only step that takes something away from users, and it should be justified with download statistics rather than intuition.

One caveat about pruning: download counts under-report platforms that are broken. If a musllinux wheel has never worked, nobody installs it, and the statistics will tell you to remove it — a self-fulfilling measurement. Fix a platform before you measure whether anyone wants it.

Caching Native Builds Without Poisoning Them

Caching is the second-largest lever and the one most likely to produce a wrong artifact if it is done carelessly. The expensive object in a spatial build is compiled GDAL, PROJ and GEOS — not the Python wheel, which takes seconds. That means the cache key must describe the native inputs exactly: the library versions, the compiler and its flags, the base image digest, and the target architecture. A key built from hashFiles('pyproject.toml') is wrong in the direction that matters, because it will happily hand a job compiled artifacts from a different GDAL.

# A key that names every input the cached objects depend on
- uses: actions/cache@v4
  with:
    path: ~/.cache/native-build
    key: native-${{ matrix.os }}-${{ matrix.arch }}-gdal3.8.4-proj9.3.1-geos3.12.1-${{ hashFiles('ci/native-versions.lock', 'ci/build-native.sh') }}
    restore-keys: |
      native-${{ matrix.os }}-${{ matrix.arch }}-gdal3.8.4-proj9.3.1-

The restore-keys prefix is what makes the cache useful rather than merely correct: an exact miss still restores the closest previous build, so ccache or sccache can reuse most object files even when one dependency moved. That distinction is the difference between a fifteen-minute build and a ninety-second one after a routine version bump.

What belongs in a native build cache key, and what must never Two columns. On the left, inputs that must be part of the cache key: GDAL, PROJ and GEOS versions, the base image digest, the target architecture, the compiler identity and flags, and the native build script itself. On the right, inputs that must not be part of it: the Python version when building abi3, the package version, the git commit, and the branch name. Below, a warning that a key missing a native input serves objects compiled against a different library. must be in the key GDAL / PROJ / GEOS versions base image digest (not tag) target architecture compiler id + optimisation flags the native build script's hash libc variant (glibc / musl) change any of these and the objects are not reusable must not be in the key CPython version (abi3 build) your package version git commit sha branch name build timestamp runner instance id including these means the cache never hits a key that omits a native input is worse than no cache: it serves objects built against a different library

Two operational rules keep a shared cache honest. Write to it only from the default branch, so a feature branch experimenting with a different PROJ cannot publish objects that a release build will later restore. And treat the base image by digest rather than tag: manylinux_2_28_x86_64:latest moves under you, and a cache keyed on the tag will pair yesterday’s objects with today’s toolchain, producing link errors that appear random because the inputs that changed are not visible in the workflow file.

Cross-system portability is worth a note as well. The GitHub and GitLab caches are not interchangeable — different key semantics, different eviction, different size limits — so a project that supports both should keep the cache-key computation in a small script that both CI files call, rather than duplicating an expression in two YAML dialects that will inevitably drift.

Keeping Two CI Dialects in Agreement

A project that supports both GitHub Actions and GitLab CI has two files describing one grid, and the failure mode is never a syntax error — it is drift. Someone adds musllinux to the Actions matrix during a release crunch, the GitLab file keeps building four platforms while the Actions file builds five, and six weeks later a release cut from the GitLab pipeline is missing a wheel that users have started depending on. Nothing fails; the artifact set is simply smaller, and the first person to notice is a user whose pip install starts compiling from source.

The remedy is to stop treating the CI file as the source of truth for the grid. Put the platform list in a single machine-readable file — a small JSON or TOML document listing the operating system, architecture, libc and runner label for each cell — and have both CI systems read it. GitHub Actions can consume it directly through a setup job that emits the matrix as an output, and GitLab can generate a child pipeline from the same file. What lives in the YAML is then only the dialect-specific plumbing: how a job is declared, how artifacts are named, how the fan-in step gathers them.

{
  "platforms": [
    { "os": "linux",   "arch": "x86_64",  "libc": "glibc", "runner": "ubuntu-latest" },
    { "os": "linux",   "arch": "aarch64", "libc": "glibc", "runner": "ubuntu-24.04-arm" },
    { "os": "linux",   "arch": "x86_64",  "libc": "musl",  "runner": "ubuntu-latest" },
    { "os": "macos",   "arch": "arm64",   "libc": "libc",  "runner": "macos-14" },
    { "os": "windows", "arch": "amd64",   "libc": "msvc",  "runner": "windows-latest" }
  ]
}

# GitHub: read the grid instead of restating it
jobs:
  setup:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.load.outputs.matrix }}
    steps:
      - uses: actions/checkout@v4
      - id: load
        run: echo "matrix=$(jq -c .platforms ci/platforms.json)" >> "$GITHUB_OUTPUT"
  build:
    needs: setup
    strategy:
      fail-fast: false
      matrix:
        include: ${{ fromJSON(needs.setup.outputs.matrix) }}
    runs-on: ${{ matrix.runner }}

The second source of drift is the build step itself. Anything the CI file knows about compiling GDAL — configure flags, vendored versions, strip settings — is knowledge that should live in a script the CI file calls, for exactly the same reason. A single ci/build-native.sh invoked identically from both systems means a change to the build is one edit rather than two, and means a developer can reproduce a CI failure locally by running the same script. It also makes the cache key honest, because the script’s own hash becomes part of the key and any change to the build invalidates the objects it produced.

Artifact naming deserves the same discipline. The two systems have different defaults and different collision behaviour: GitHub overwrites same-named artifacts within a run, GitLab merges paths across jobs. Naming every artifact after the cell that produced it — wheels-linux-aarch64-glibc — removes the ambiguity in both, and makes the fan-in step’s job obvious. When a release is missing a platform, the missing artifact name tells you which cell failed without opening a log.

Finally, decide which system is authoritative for releases and say so in the repository. Running the full grid on both is defensible for redundancy, but publishing from both is not: two pipelines uploading the same version race each other, and the loser fails with File already exists in a way that looks like a bug. Build everywhere if you like; publish from one place.

What to Measure Once the Grid Works

A matrix that has stopped failing is not finished, because its cost and its coverage both drift. Three numbers are worth recording per release, and all three are cheap to emit from the pipeline itself.

The first is wall-clock time per cell. Recording it turns “CI feels slow” into a ranked list, and in spatial builds that list is almost always topped by an emulated architecture or an uncached native build. It also exposes the cell that quietly stopped hitting its cache — a job whose time jumps from ninety seconds to fifteen minutes and stays there is a cache key that no longer matches, which no test will ever report.

The second is the artifact inventory: the exact set of wheel filenames the fan-in step collected. Diffing it against the previous release is the only reliable way to notice that a platform silently stopped being produced, and it takes one line in the release job. Store it next to the wheels so the record survives log retention.

The third is the resolved native versions per cell. A matrix where one runner picked up GDAL 3.8.4 and another 3.8.5 — because a base image moved under a floating tag — produces wheels that behave differently on different platforms, which is the hardest class of user report to diagnose. Printing gdal-config --version and proj --version in every job, and asserting they agree in the fan-in step, catches it before publication.

None of this requires a dashboard. A JSON file written by the fan-in job, committed to the release artifacts, gives you a diffable history that answers “what changed?” in the minutes after a regression report rather than the hours.

Frequently Asked Questions

Should the matrix build one wheel per interpreter or one abi3 wheel?

One abi3 wheel per platform, unless you have a measured reason not to. Geospatial bindings spend nearly all their time inside GDAL and PROJ, so the version-specific CPython optimisations you give up are unmeasurable, while the matrix shrinks by a factor of four or five and every subsequent Python release stops requiring a rebuild.

How should the matrix handle a platform that is allowed to fail?

Give it continue-on-error: true (or GitLab’s allow_failure: true) and exclude its artifacts from the fan-in step, rather than letting a half-published set reach the release job. An experimental platform that blocks releases gets deleted within a month; one that is visibly allowed to fail can stay until it is ready.

Is fail-fast: false always right?

For a release matrix, yes: cancelling twelve nearly-finished jobs because one architecture broke wastes the compute that would have told you whether anything else was also broken. For a pull-request matrix the opposite is often better, since the first failure is usually the only signal a reviewer needs and fast feedback matters more than completeness.

Can the same matrix build both release wheels and pull-request wheels?

It can, and it should, with one difference: pull requests build a reduced grid. Running the two most representative cells on every pull request catches build breakage within minutes, while the full grid runs on the release tag and on a nightly schedule. Keeping the reduced grid a strict subset of the full one — same script, same cache keys, fewer cells — means a pull request that passes is genuine evidence about the release build rather than a different pipeline that happens to look similar.

Where should the version pins for GDAL and PROJ live?

In a single file that both CI systems and the local build read — a small lock file or shell script listing the versions and source URLs. Pins duplicated across a Dockerfile, two CI configurations and a README drift within two releases, and the resulting mismatch is invisible until a cache key silently stops matching.

Does a scheduled build add anything if every release is tagged?

It adds the one signal a tag-driven pipeline cannot produce: whether the build still works when nothing in your repository changed. Base images move, runner images are replaced, conda-forge rebuilds packages. A nightly run of the full grid turns “the release build broke” into “the environment changed on the eleventh”, which is a far cheaper thing to diagnose than the same breakage discovered under release pressure.

Further Reading

  • cibuildwheel CI examples (cibuildwheel.readthedocs.io/en/stable/setup/).