cibuildwheel vs manual Docker matrix for GDAL wheels

This page answers one question: should you build your GDAL wheels with cibuildwheel’s managed lifecycle or hand-roll a Docker matrix that drives manylinux containers yourself — and where does each break down for a heavy native stack like GDAL/PROJ/GEOS? It sits inside the Manylinux and Manyarm Docker Base Images section of the Modern Python Build Tooling & Wheel Configuration reference, and gives you a decision table, the crossover point, and a hybrid pattern that keeps cibuildwheel’s repair while owning the native build.

Where cibuildwheel and a manual Docker matrix differ across the build lifecycle Across five lifecycle concerns — image selection, native dependency build, wheel repair, cross-platform coverage, and maintenance — cibuildwheel manages image, repair, and platform coverage automatically while leaving the native build to before-all hooks, whereas a manual Docker matrix gives full control of every stage at the cost of writing and maintaining each one. cibuildwheel manual Docker matrix image native build repair multi-platform maintenance managed before-all hook automatic one config low you pick full control you script per-arch job high

Context & Root Cause

Both tools produce the same thing — a repaired manylinux wheel — but they draw the control boundary in different places. cibuildwheel owns the lifecycle: it pulls the manylinux base image, builds the wheel, runs auditwheel repair, and tests in isolation, exposing only hooks (before-all, environment, repair-wheel-command). A manual Docker matrix inverts this: you write the docker run invocations, install GDAL yourself, call the compiler yourself, and invoke auditwheel yourself, gaining total control and paying for every line.

For a pure-Python-plus-thin-C package the choice is easy — cibuildwheel wins outright. GDAL complicates it because the expensive, fiddly part is building the native stack (GDAL depends on PROJ, GEOS, libtiff, libsqlite3, libcurl), and that work lives in a before-all hook regardless of tool. The real question is whether cibuildwheel’s managed image/repair/test layer is worth more than the control a raw matrix gives you over caching, layer reuse, and unusual targets. This page is the decision framework, sitting alongside the image-selection detail in manylinux2014 vs musllinux for spatial libs.

The decision

The two approaches map cleanly onto project maturity and target exotica:

Concern cibuildwheel Manual Docker matrix
Image selection Pinned by one config key You choose and maintain the tag
Native GDAL build In a before-all hook (same work) In your Dockerfile (same work)
Wheel repair Automatic auditwheel/delocate/delvewheel You script each per platform
macOS + Windows Same config, one runner each Separate non-Docker jobs entirely
Native-build caching Coarser (hook-level) Fine-grained Docker layer cache
Exotic targets (custom libc, odd arch) Limited to supported images Anything you can containerize
Maintenance burden Low High

Choose cibuildwheel when your targets are the standard manylinux/musllinux/macOS/Windows grid and you want the matrix, repair, and test isolation for free — which is most projects. Choose a manual matrix only when you need something cibuildwheel cannot express: a bespoke base image, exhaustive Docker-layer caching of a 40-minute GDAL build, or an architecture outside the supported set. The crossover is not project size but target exotica — a large project on standard targets still wants cibuildwheel.

The hybrid that usually wins

You rarely have to pick cleanly. Let cibuildwheel own the lifecycle but hand it a prebuilt GDAL so the slow native compile is cached as a Docker image, not rebuilt every run:

[tool.cibuildwheel]
build = "cp39-*"
manylinux-x86_64-image = "ghcr.io/myorg/manylinux_2_28-gdal:3.8.5"   # your image, GDAL baked in
before-all = "echo 'GDAL already in the image'"

# Dockerfile.build — built and pushed once, reused by every wheel run
FROM quay.io/pypa/manylinux_2_28_x86_64
RUN bash /ci/build_gdal_proj_geos.sh    # the 40-minute compile, cached as a layer

This keeps cibuildwheel’s repair, test isolation, and multi-platform config while capturing the Docker-layer caching that is the manual matrix’s one decisive advantage. The before-all becomes a no-op because the image already carries GDAL.

Verification 

# 1. Confirm the custom image is used and GDAL is present pre-build
docker run --rm ghcr.io/myorg/manylinux_2_28-gdal:3.8.5 gdal-config --version
# expected: 3.8.5

# 2. cibuildwheel still produces a correctly tagged, repaired wheel
pipx run cibuildwheel --platform linux && auditwheel show wheelhouse/*.whl | grep -i tag
# expected: manylinux_2_28_x86_64

A version printed straight from the image and a versioned platform tag confirm the hybrid works: the slow build is cached, the repair still runs. If before-all is still compiling GDAL, the custom image key did not take effect.

What You Take On With Each Choice

The decision is usually framed as convenience versus control, which understates it. What actually differs is the list of responsibilities you accept, and that list is worth seeing in full before committing a team to the hand-rolled path.

Responsibilities carried by cibuildwheel versus a hand-rolled Docker matrix Seven responsibilities compared. Selecting build identifiers, choosing and pinning base images, invoking the correct repair tool per platform, computing platform tags, running per-platform tests, handling macOS and Windows, and tracking upstream policy changes. With cibuildwheel all seven are handled by the tool and configured declaratively. With a hand-rolled matrix all seven become code you write and maintain, and the last two are the ones teams most often underestimate. cibuildwheel hand-rolled select build identifiers declarative your loop choose and pin base images one setting your Dockerfiles the right repair tool per OS automatic three code paths compute the platform tag automatic yours to get right per-platform test runs test-command your scripts macOS and Windows same config, no containers needed Docker does not help here at all upstream policy changes arrive with an upgrade you track them yourself the last two rows are what teams underestimate: containers solve one platform of three, and the policy moves without asking

The macOS and Windows row is decisive for most projects. A Docker matrix is a Linux tool; the other two platforms still need their own runners, their own repair tooling and their own scripts, so the hand-rolled path does not replace cibuildwheel — it replaces one third of it while leaving the rest to be written twice.

The Hybrid in Practice

The arrangement that works well for GDAL-heavy projects keeps cibuildwheel as the driver and moves only the expensive, project-specific part — building the native stack — into a container you control.

A hybrid where a custom image supplies GDAL and cibuildwheel does the rest A separate, infrequently rebuilt pipeline produces a custom manylinux image containing compiled GDAL, PROJ and GEOS, published by digest to a registry. The wheel pipeline references that digest, and cibuildwheel then performs identifier selection, the wheel build, repair, tagging and testing exactly as it would with a stock image. The native build therefore happens once per dependency bump rather than once per wheel build. runs on a dependency bump — weekly at most stock manylinux pinned by digest + GDAL/PROJ/GEOS compiled once push to registry referenced by digest runs on every commit and tag cibuildwheel manylinux-image = ours build + repair seconds, not minutes tag + test unchanged behaviour wheels the expensive work moves out of the per-commit path without giving up any of the tooling that makes the wheel correct

This keeps the responsibilities table above firmly on the left-hand column while removing the one genuine advantage of the hand-rolled approach: control over exactly how the native stack is compiled, and the ability to compile it once rather than on every build.

Pitfalls & Alternatives

Hand-rolling a matrix to “save time.” Teams underestimate the repair and test-isolation logic cibuildwheel provides. Reimplementing auditwheel invocation, platform tagging, and clean-import testing per platform is weeks of maintenance for parity with a tool you could configure in an afternoon.

Rebuilding GDAL every CI run under cibuildwheel. The default before-all recompiles the native stack on every job — the single biggest waste in a spatial pipeline. Bake GDAL into a custom image (the hybrid) or cache the archives per the async build execution and cache strategies guide.

Choosing manual for macOS/Windows. Docker does not build macOS or Windows wheels; a “manual Docker matrix” still needs separate non-container jobs for those, whereas cibuildwheel spans all three from one config. Do not let the Linux decision dictate the whole matrix.

Frequently Asked Questions

When is a hand-rolled Docker matrix genuinely the right answer?

When the native build is unusual enough that the standard flow cannot express it — a patched GDAL, a proprietary driver, an internal mirror that must be configured before anything downloads — and the project ships Linux only. Once macOS or Windows are in scope, the container approach covers one platform of three and the other two need writing anyway, which usually tips the decision back.

Does the hybrid approach lose any of the tool’s benefits?

No, which is why it is the recommended shape. Pointing the tool at your own image changes where GDAL comes from and nothing else: identifier selection, repair, tagging and testing all behave exactly as they do with a stock image. What you gain is a native build that happens on a dependency bump rather than on every commit.

How do I keep a custom base image from drifting?

Build it in its own pipeline, tag it by digest, and reference the digest from the wheel pipeline. Bump the digest deliberately through a pull request, ideally opened automatically on a schedule. The failure this prevents is a floating tag that moves under a cache key, pairing yesterday’s objects with today’s toolchain.

Is it harder to onboard contributors with a custom image?

Slightly, and it is worth mitigating. Publish the image publicly if the project is open source, and provide a single command that reproduces a build locally using it. A contributor who can run the same image the pipeline uses is in a much better position than one who has to assemble a GDAL toolchain by hand, so a custom image can actually lower the barrier rather than raise it.

What about docker buildx for multi-architecture images?

It is the right tool for producing the custom base image across architectures, and it does not replace anything in the wheel pipeline. Build the image for x86_64 and aarch64, push both under one manifest, and let the wheel jobs pull whichever matches their runner — which keeps the platform logic in one place rather than duplicated per cell.

How much of this decision is reversible?

More than it feels like. Moving from a hand-rolled matrix to the tool is mostly deleting scripts, because the tool’s defaults cover what those scripts were doing. Moving the other way is harder, since you take on the tag computation and three repair paths at once. If you are undecided, start with the tool: it is the cheaper direction to leave later. In practice most teams that start hand-rolled end up adopting the hybrid within a release or two, because the repair and test layers are the parts nobody enjoys rewriting and the parts that quietly decide whether a wheel installs.