Modern Python Build Tooling & Wheel Configuration

Turning a geospatial Python package into a binary wheel is a compilation problem disguised as a packaging problem. Every pip install rasterio or pip install pyproj ultimately resolves to a .so, .pyd, or .dylib that was compiled against GDAL, PROJ, and GEOS on a machine the end user will never see — and the manifest that drives that build is the single artifact deciding whether the result is reproducible or a one-off accident. This reference covers the build-frontend-to-registry half of the geospatial CI/CD library; its companion, Geospatial C-Extension Fundamentals & ABI Architecture, governs the binary-interface contracts the compiled object must honour. Start from the geospatial CI/CD engineering reference for the full map. The three chapters that carry the most weight here are mastering pyproject.toml for spatial wheels, integrating CMake with scikit-build-core, and the manylinux and manyarm Docker base images that anchor glibc compliance.

Modern tooling turns a declarative manifest into a tagged, repaired wheel through a deterministic pipeline:

The contract that holds this pipeline together is build isolation: the frontend constructs a throwaway virtual environment, installs exactly the backend and compile-time dependencies the manifest declares, and runs the backend with no access to whatever happens to be installed on the host. Break isolation and you get builds that pass on a maintainer’s laptop and fail on a clean runner; honour it and the same pyproject.toml produces byte-identical wheels on GitHub Actions, GitLab CI, and a developer workstation alike.

Core Concept: The Build-Isolation Contract #

Contemporary Python packaging mandates a strict separation between build configuration and runtime metadata, codified by PEP 517 (the backend interface) and PEP 518 (the [build-system] table). The pyproject.toml file is the single source of truth: it names the build backend, pins the exact compile-time requirements, and carries static project metadata. For geospatial packages that compile C and C++ extensions, the non-negotiable rule is that the [build-system].requires list must pin every tool that participates in compilation — backend, cmake, ninja, Cython — to a closed version range, because the frontend will install them into a fresh, network-isolated environment and any unpinned drift silently changes the compiler invocation.

[build-system]
requires = [
    "scikit-build-core>=0.9.0,<0.11",
    "cython>=3.0.0,<3.1",
    "numpy>=1.24.0",
    "setuptools-scm>=8.0.0",
]
build-backend = "scikit_build_core.build"

[project]
name = "spatial-core"
dynamic = ["version"]
requires-python = ">=3.9"
dependencies = ["numpy>=1.24.0", "pyproj>=3.5.0"]

[tool.scikit-build]
cmake.version = ">=3.26"
ninja.version = ">=1.11"
wheel.py-api = "cp39"          # emit a single abi3 wheel for 3.9+
build-dir = "build/{wheel_tag}"

Two clauses in that manifest do the heavy lifting. build-backend = "scikit_build_core.build" hands compilation to the scikit-build-core backend that translates pyproject.toml into CMake invocations, replacing the imperative setup.py model where arbitrary Python ran at build time and mutated the environment. wheel.py-api = "cp39" requests a Stable-ABI (abi3) wheel so one binary serves every interpreter from 3.9 up — a strategy that only works if the C source enforces Py_LIMITED_API=0x03090000 at compile time. The full mapping of spatial metadata, dynamic versioning, and backend selection is the subject of mastering pyproject.toml for spatial wheels.

Because the frontend builds in isolation, the compiler also needs the geospatial system libraries resolved before the build starts. CMake is the bridge: the backend translates the [tool.scikit-build] table into CMake cache variables and runs an out-of-source build that discovers headers, links libraries, and patches load paths.

cmake_minimum_required(VERSION 3.26)
project(spatial_core LANGUAGES C CXX)

# Locate the interpreter and the limited-API headers the abi3 wheel needs
find_package(Python COMPONENTS Interpreter Development.Module REQUIRED)

# Resolve geospatial system libraries provisioned by the CI image or pixi env
find_package(GDAL CONFIG REQUIRED)
find_package(PROJ CONFIG REQUIRED)

python_add_library(_spatial_core MODULE
    src/spatial_core.c
    src/proj_wrapper.cpp
    WITH_SOABI
)

target_link_libraries(_spatial_core PRIVATE GDAL::GDAL PROJ::proj)

# Hidden visibility keeps the abi3 surface small and the wheel portable
set_target_properties(_spatial_core PROPERTIES
    C_VISIBILITY_PRESET hidden
    CXX_VISIBILITY_PRESET hidden
    POSITION_INDEPENDENT_CODE ON
)

install(TARGETS _spatial_core LIBRARY DESTINATION spatial_core)

This eliminates the brittle setup.py compiler-flag hacks that hard-coded include directories per machine, and it lets CMake — not handwritten logic — manage the RPATH/RUNPATH entries that the repair step later rewrites for distribution.

Dependency Topology: System Libraries, Static vs Dynamic #

The hard part of a geospatial wheel is not Python — it is GDAL pulling in PROJ, GEOS, libtiff, libgeotiff, libsqlite3, libcurl, and a dozen format drivers, each with its own SONAME and ABI. How those native libraries reach the final artifact is the central design decision, and it trades portability against wheel size and patch cadence.

Vendoring is the default for public wheels because it is the only way to guarantee an import succeeds on a host you do not control, but it duplicates a PROJ datum grid and a full GDAL driver set into every wheel — the size and de-duplication trade-offs are dissected in why vendoring PROJ causes wheel bloat, and the broader vendoring PROJ and GDAL vs system libraries decision lives in the ABI reference. The dynamic strategy keeps wheels lean but pushes the SONAME-matching burden onto runtime, where it surfaces as a load-time failure governed by shared library path resolution. The hybrid path resolves both build- and run-time libraries from a single locked solver and is the model behind environment isolation with Pixi and Conda.

Whichever topology you pick, the system libraries must be present and ABI-consistent before CMake runs find_package. On a manylinux base image that means compiling GDAL/PROJ into the image or installing them from a vendored prefix; in a pixi environment it means a lock file that pins gdal, proj, and the C/C++ compilers to exact builds so header paths and ABI flags stay identical across macOS, Linux, and Windows runners.

CI/CD Integration: The Build Matrix #

Wheel compilation is computationally expensive, so the pipeline fans out across interpreter versions, platforms, and architectures and caches everything that can be reused. The matrix below builds with cibuildwheel, which drives the build inside the correct manylinux container per row and applies the repair step automatically; the env vars carry the GDAL/PROJ configuration into each isolated build, and the cache key is derived from the lock files so a dependency change invalidates it deterministically.

name: build-wheels
on: [push, workflow_dispatch]

jobs:
  wheels:
    name: ${{ matrix.os }} · ${{ matrix.arch }}
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        include:
          - { os: ubuntu-latest,  arch: x86_64,  image: manylinux_2_28 }
          - { os: ubuntu-latest,  arch: aarch64, image: manylinux_2_28 }
          - { os: macos-14,       arch: arm64,   image: "" }
          - { os: windows-latest, arch: AMD64,   image: "" }
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }     # setuptools-scm needs full history

      - name: Cache CMake build dir
        uses: actions/cache@v4
        with:
          path: build
          key: cmake-${{ matrix.os }}-${{ matrix.arch }}-${{ hashFiles('pixi.lock', 'pyproject.toml') }}

      - name: Build wheels
        uses: pypa/cibuildwheel@v2.21
        env:
          CIBW_ARCHS: ${{ matrix.arch }}
          CIBW_MANYLINUX_X86_64_IMAGE: ${{ matrix.image }}
          CIBW_MANYLINUX_AARCH64_IMAGE: ${{ matrix.image }}
          CIBW_BUILD: "cp39-* cp310-* cp311-* cp312-*"
          CIBW_ENVIRONMENT: "GDAL_CONFIG=/usr/bin/gdal-config PROJ_DIR=/usr/local"
          CIBW_BEFORE_ALL_LINUX: "bash scripts/install_gdal_proj.sh"
          CIBW_REPAIR_WHEEL_COMMAND_LINUX: "auditwheel repair -w {dest_dir} {wheel}"
          CIBW_TEST_COMMAND: "python -c 'import spatial_core; print(spatial_core.__file__)'"

      - uses: actions/upload-artifact@v4
        with: { name: wheels-${{ matrix.os }}-${{ matrix.arch }}, path: wheelhouse/*.whl }

Two matrix dimensions deserve attention for spatial builds. The aarch64 row needs either an emulated or native ARM runner and a matching base image — the cross-compilation path and base-image selection are covered in manylinux and manyarm Docker base images, and the toolchain setup itself in cross-compiler toolchain setup. Because a single abi3 wheel covers every interpreter, you can prune the CIBW_BUILD list to one Python per platform once the Stable-ABI build is proven, cutting matrix cost dramatically. The caching keys above hash pixi.lock and pyproject.toml so the CMake build directory is reused across runs until a real dependency changes; the deeper reuse strategy — ccache/sccache, package-index caches, and pre-warmed containers — is documented in async build execution and cache strategies and its companion how to set up build caching for C extensions.

Validation & Repair #

A freshly built wheel is not yet portable: it links against libraries by absolute host paths and carries RPATH entries pointing at the build machine. The repair step bundles the dependent shared objects into the wheel and rewrites the load paths to be relative, then validation confirms the result is self-contained.

On Linux, auditwheel does both jobs. Inspect first, then repair:

# 1. See which external libs the wheel still depends on and its platform tag
auditwheel show wheelhouse/spatial_core-1.2.0-cp39-abi3-linux_x86_64.whl

# 2. Bundle them and re-tag to a manylinux policy
auditwheel repair --plat manylinux_2_28_x86_64 \
  -w dist/ wheelhouse/spatial_core-1.2.0-cp39-abi3-linux_x86_64.whl

A passing auditwheel show reports the wheel as already conforming to a manylinux_2_28_x86_64 platform tag with no external references outside the allowed policy set; a failing one lists libraries like libgdal.so.34 as “not bundled” and refuses the tag. On macOS the equivalent is delocate:

delocate-listdeps --all dist/spatial_core-1.2.0-cp39-abi3-macosx_11_0_arm64.whl
delocate-wheel -w fixed/ -v dist/spatial_core-1.2.0-cp39-abi3-macosx_11_0_arm64.whl

After repair, the pass/fail check is that the bundled extension resolves every dependency from inside the wheel. Unpack and run ldd against the .so:

python -m wheel unpack dist/spatial_core-1.2.0-cp39-abi3-manylinux_2_28_x86_64.whl -d /tmp/wh
ldd /tmp/wh/spatial_core-1.2.0/spatial_core/_spatial_core.abi3.so

Every geospatial dependency should resolve to a path inside the unpacked wheel’s .libs directory (e.g. libgdal-…so => …/spatial_core.libs/libgdal-abc123.so). Any line reading not found, or any GDAL/PROJ entry resolving to a system path like /usr/lib, means the repair missed a library and the wheel will fail on a clean host. The final gate before publishing is a clean-environment import and a metadata check:

python -m venv /tmp/fresh && /tmp/fresh/bin/pip install dist/*.whl
/tmp/fresh/bin/python -c "import spatial_core; spatial_core.self_test()"
twine check dist/*.whl     # validates METADATA, RECORD, and the WHEEL tag

The structural side of validation — correct .dist-info contents, architecture-specific extension placement, and bundled licence files — is detailed in build artifact structuring and packaging.

Failure Modes & Diagnostics #

Most geospatial wheel failures reduce to a handful of recurring patterns. Each has a distinct signature and a concrete fix.

ImportError: libproj.so.25: cannot open shared object file: No such file or directory — the extension was built dynamically against a system PROJ that is absent or has a different SONAME on the target. Root cause: the repair step never bundled libproj, or the build linked a system lib the wheel does not carry. Fix: run auditwheel repair (or delocate-wheel) so the SONAME is vendored, then re-check with ldd that libproj.so.25 resolves inside .libs. Hosts that intentionally use system libraries instead must satisfy the SONAME via the rules in shared library path resolution.

ImportError: undefined symbol: GDALCreate (or a PROJ symbol) — an ABI mismatch between the GDAL the wheel was compiled against and the one resolved at import. The header set and the linked library disagree on a symbol version. Root cause: the build image and the runtime image pin different GDAL minor versions. Fix: pin GDAL/PROJ to one build in both the build image and the runtime environment (a single lock file is the durable cure), then rebuild. The interpreter-side variant — an abi3 wheel touching a symbol outside the limited set — is dissected in C-API vs CPython ABI compatibility and the recovery steps in how to fix ABI version mismatch in GDAL wheels.

auditwheel: cannot repair … requires libgdal which is not in the policy — the wheel depends on a library so large or so externally linked (e.g. a CUDA-enabled GDAL) that bundling it violates the chosen manylinux policy. Root cause: trying to vendor a dependency that itself pulls in disallowed system libraries. Fix: raise the policy (manylinux_2_28 over manylinux2014), trim GDAL’s driver set, or move to a hybrid conda/pixi distribution where the solver supplies the heavy library. Base-image and policy selection is covered in manylinux2014 vs musllinux for spatial libs.

RPATH pollution — wheel imports on the build machine but fails elsewhere — the extension carries an absolute RUNPATH such as /home/runner/work/build/lib. Root cause: CMake baked the build-tree RPATH into the artifact and the repair step did not rewrite it to $ORIGIN. Fix: set INSTALL_RPATH "$ORIGIN/.libs" (and BUILD_WITH_INSTALL_RPATH ON) in CMake, then let auditwheel finalise relative paths; confirm with readelf -d _spatial_core.abi3.so | grep -E 'RPATH|RUNPATH'.

CMake Error: Could NOT find PROJ (missing: PROJ_DIR) during the isolated build — find_package(PROJ CONFIG REQUIRED) cannot locate the PROJ CMake config because the build environment never provisioned it. Root cause: system libraries were assumed present but the isolated build sees only what the image installed. Fix: install PROJ into the image in CIBW_BEFORE_ALL, or point PROJ_DIR/CMAKE_PREFIX_PATH at the vendored prefix; the precise find_package repair is walked through in fixing CMake find_package for PROJ.

Chapters in This Reference #

This reference branches into six chapters, each owning one stage of the manifest-to-registry pipeline. Read them in roughly pipeline order:

• Mastering pyproject.toml for spatial wheels maps spatial metadata, dynamic versioning, and backend selection into a manifest that builds in isolation.
• Integrating CMake with scikit-build-core bridges the Python backend to CMake targets — including optimizing scikit-build-core for GDAL and fixing CMake find_package for PROJ.
• Environment isolation with Pixi and Conda provisions reproducible C/C++ toolchains and spatial libraries from a lock file, with a worked setup in configuring pixi environments for wheel building.
• Async build execution and cache strategies cuts pipeline latency through parallelism and cache hydration.
• Manylinux and manyarm Docker base images selects the right base image and cross-compilation path for glibc and musl targets.
• Build artifact structuring and packaging finalises .dist-info, licences, and the metadata that registries validate before promotion.

Adopting modern build tooling turns geospatial Python distribution from an environment-dependent accident into a deterministic, signable pipeline: PEP 517/518 isolation guarantees the manifest alone drives the build, vendoring-aware repair guarantees the wheel imports anywhere, and a cached matrix keeps the cost bounded across every interpreter and architecture. The most actionable next step for a maintainer starting from a legacy setup.py is to lock the manifest first — begin with mastering pyproject.toml for spatial wheels.

Related #

• Geospatial C-Extension Fundamentals & ABI Architecture — the companion reference defining the binary-interface contracts every wheel built here must satisfy.
• C-API vs CPython ABI compatibility — why a single abi3 wheel can serve every interpreter, and the macro guards that keep it valid.
• Vendoring PROJ and GDAL vs system libraries — the static-vs-dynamic decision behind the dependency-topology table above.
• Manylinux and manyarm Docker base images — base-image and policy choices that make auditwheel repair succeed.
• Async build execution and cache strategies — the caching and parallelism that keep the build matrix affordable.