How to Set Up Build Caching for C-Extensions

Compiling geospatial C-extensions such as the GDAL, PROJ, Rasterio, or PyGEOS bindings can take 15–40 minutes per CI run, and without a cache every matrix entry recompiles the same object files — this page shows exactly how to wire ccache/sccache, build-directory caching, and a deterministic cache key into a spatial wheel pipeline so unchanged C/C++ sources are never recompiled. It sits under Async Build Execution and Cache Strategies, part of the broader Modern Python Build Tooling & Wheel Configuration reference.

Each cache layer short-circuits the work the layer below it would otherwise repeat:

Three layered caches that short-circuit a geospatial C-extension wheel build A left-to-right pipeline. Source files plus compiler flags feed three caches in series, each keyed on the same hash of pyproject.toml, the C/C++/Cython sources and the platform tag. Layer 1 is the ccache or sccache object cache in ~/.ccache, which reuses compiled .o objects and skips the per-file compile. Layer 2 is the backend build directory such as _skbuild or build/, which reuses the CMake configuration and skips the reconfigure. Layer 3 is the manylinux base image, which keeps the GDAL and PROJ headers and shared objects so the system dependencies are never rebuilt. On a cache hit each layer short-circuits the slower work in the layer below it, producing a much faster wheel build; on a miss the work flows on to the next stage. Three caches in series — each hit short-circuits the work below it SHARED CACHE KEY hashFiles(pyproject.toml, **/*.c, *.cpp, *.pyx) + platform tag Source + compiler flags CC="ccache gcc" LAYER 1 Object cache ccache / sccache ~/.ccache hit → skip per-file compile LAYER 2 Build directory _skbuild · build/ meson-private hit → skip CMake reconfigure LAYER 3 System deps manylinux image GDAL/PROJ .so hit → skip PROJ/GDAL rebuild Faster wheel build 70–85% shorter on unchanged sources Restore caches before the wrapper runs · keep PEP 517 isolation intact · namespace the key per platform tag

Context & Root Cause

The recompilation tax comes from build isolation, not from a slow compiler. Modern backends — scikit-build-core, meson-python, and setuptools — run under PEP 517, so pip spins up a fresh temporary environment, installs build dependencies, compiles the extension, and then deletes the directory on success. Nothing about the compilation survives, so the next matrix entry (cp311, a different arch, the same commit on a rerun) starts from zero.

Geospatial packages make this worse: a single pyproj or rasterio build pulls in thousands of GDAL/PROJ/GEOS header lines and heavy C++17/20 templates, so the per-file compile cost is high. The fix is to persist three independent caches across runs — the compiler object cache (ccache/sccache), the backend build directory (_skbuild/, build/, meson-private/), and the system headers/.so files — while keeping the PEP 517 isolation boundary intact so the wheel stays reproducible.

Solution / Fix

Prerequisites: ccache ≥4.6 (or sccache ≥0.7), pip ≥23.1, build ≥1.0, and a pinned manylinux Docker base image so the glibc target never drifts under the cache.

1. Inject a compiler wrapper before any build command

ccache is the manylinux standard; sccache is preferable when you want a shared S3/Redis backend across parallel matrix jobs. Export the wrapper into CC/CXX before pip or build runs:

# Install and configure ccache (Ubuntu/Debian manylinux)
apt-get update && apt-get install -y ccache
export CC="ccache gcc"
export CXX="ccache g++"
export CCACHE_DIR="${HOME}/.ccache"
export CCACHE_MAXSIZE="2G"
export CCACHE_COMPRESS="1"
export CCACHE_COMPRESSLEVEL="9"
# Critical for reproducible spatial wheels: ignore __TIMESTAMP__ variations
export CCACHE_SLOPPINESS="time_macros,locale,include_file_mtime"

On Alpine / musllinux runners, swap gcc/g++ for musl-gcc or the clang equivalents. The CCACHE_SLOPPINESS directive is non-negotiable for spatial packages: without time_macros, the __TIMESTAMP__/__DATE__ macros embedded in some PROJ and GDAL headers change every run and force a 0% hit rate.

2. Restore the caches before the wrapper runs

Cache restoration must happen first, keyed on the source hashes that actually affect compilation — never on a timestamp. Use one actions/cache step for the compiler cache, the pip download cache, and the backend build directory:

- name: Restore ccache and build artifacts
  uses: actions/cache@v4
  with:
    path: |
      ~/.ccache
      ~/.cache/pip
      ${{ github.workspace }}/.build-cache
    key: ${{ runner.os }}-geo-ccache-${{ hashFiles('pyproject.toml', 'src/**/*.c', 'src/**/*.cpp', 'src/**/*.pyx') }}
    restore-keys: |
      ${{ runner.os }}-geo-ccache-

The restore-keys fallback gives you a partial hit when only a subset of Cython/C++ files changed, so the cache survives ordinary commits. Caching ~/.cache/pip alongside ~/.ccache also avoids re-downloading build dependencies such as Cython, numpy, or pybind11 on every run.

3. Route compilation through the cache without breaking isolation

A PEP 517 backend ignores host-level CC/CXX unless you pass the launcher explicitly or relax isolation. For scikit-build-core and meson-python, inject the CMake compiler launchers through --config-settings so isolation stays intact:

# pip (PEP 517 compliant): launchers reach CMake through config-settings
pip wheel . \
  --config-settings=cmake.define.CMAKE_C_COMPILER_LAUNCHER=ccache \
  --config-settings=cmake.define.CMAKE_CXX_COMPILER_LAUNCHER=ccache

You can pin the same two launchers permanently in pyproject.toml so every contributor and CI runner benefits without remembering the flags:

[tool.scikit-build.cmake.define]
CMAKE_C_COMPILER_LAUNCHER = "ccache"
CMAKE_CXX_COMPILER_LAUNCHER = "ccache"

Reserve python -m build --wheel --no-isolation for cases where you have already installed every [build-system] dependency in the runner; it lets the host CC="ccache gcc" reach the compiler directly but trades away the reproducibility guarantee, so prefer the launcher route above for release builds.

4. Tune the system-dependency layer

The slowest cold builds spend their time fetching and rebuilding PROJ/GDAL/GEOS. Bake those into the manylinux base image (or a layer on top of it) and pin the versions, so the .pc files and .so libraries are already present and ccache only has to compile your extension against them:

FROM quay.io/pypa/manylinux_2_28_x86_64:latest
ENV GDAL_VERSION=3.8.4 PROJ_VERSION=9.3.1
RUN yum install -y ccache proj-devel sqlite-devel curl-devel zlib-devel
ENV CCACHE_DIR=/root/.ccache CCACHE_MAXSIZE=2G \
    CCACHE_SLOPPINESS=time_macros,include_file_mtime

Verification

Confirm the cache is actually being hit before you trust the speedup. Run these after the second build of the same commit (ccache ≥4.6):

# 1. Inspect the cache statistics
ccache -s
# Expected after run 2: "cacheable calls" with direct + preprocessed
# hits well above 60%, "cache miss" near 0 for unchanged sources.

# 2. Measure the wall-clock delta across two identical builds
time python -m build --wheel
# Expected: 70-85% shorter than the cold build for unchanged C/C++ sources.

# 3. Prove the cache did not break determinism
SOURCE_DATE_EPOCH=1700000000 PYTHONHASHSEED=0 python -m build --wheel
sha256sum dist/*.whl
# Expected: identical hash on a rebuild of the same commit.

If ccache -s still shows a near-zero hit rate on the second run, the cache directory was not restored (check the actions/cache key) or CCACHE_SLOPPINESS is missing time_macros.

Where a Compiler Cache Fits

A compiler cache sits between the build system and the compiler, and it is the only cache in the pipeline whose key is computed from content rather than declared by you. That property is what makes it both safe and easy to enable.

Where ccache intercepts a compilation and how it decides The build system invokes the compiler driver, which is ccache. Ccache preprocesses the source, hashes the preprocessed text together with the compiler identity and the exact flags, and looks the hash up in its store. On a hit it returns the cached object file without running the compiler. On a miss it runs the real compiler and stores the result under that hash. Because the key is derived from content, a hit can never return an object built from different inputs. ninja / make invokes the driver ccache hash(preprocessed source, compiler id, exact flags) hit return the object miss run the real compiler store under that hash because the key is content-derived, a hit cannot return an object built from different sources or flags — unlike a hand-written key

That last sentence is the reason to prefer a compiler cache over a hand-rolled prefix cache wherever both would work. A prefix cache is only as correct as the key you wrote; a compiler cache is correct by construction, and the worst outcome of a bad configuration is a miss.

Making It Survive Across CI Runs

A compiler cache only helps if its store persists between jobs, and that is the part that needs configuration.

Three ways to persist a compiler cache between CI runs Three options. Restoring the cache directory through the CI provider's cache action is simplest and works everywhere, with a key that changes rarely and a restore-keys prefix. Using a shared object store such as S3 with sccache removes per-job upload and download and works across runners. Baking a warm cache into a custom container image is fastest at job start but goes stale between image rebuilds. Each notes the trade-off. the provider's cache action cache ~/.cache/ccache with a stable key and a restore-keys prefix — simplest, works on every provider cost: the store is uploaded and downloaded per job, which is slow once it grows past a few hundred megabytes a shared object store sccache pointed at S3 or a compatible bucket — no per-job transfer, shared across every runner and branch cost: credentials to manage, and a bucket policy that keeps release builds from reading branch-written objects a warm cache baked into the image build the native stack once when the image is built, so jobs start with the objects already present cost: goes stale between image rebuilds, and the image grows by the size of the cache

For most spatial projects the first option is enough, and the second becomes worthwhile once the store outgrows the provider’s per-entry limit — which a full GDAL build reaches sooner than people expect. Whichever you choose, print the cache statistics at the end of the job: a hit rate that quietly drops to zero is otherwise invisible until someone notices the build got slower.

Pitfalls & Alternatives

  • Caching across manylinux tags. Reusing one ~/.ccache between a manylinux_2_17 and a manylinux_2_28 job poisons it: objects compiled against the newer glibc surface as ABI mismatch: expected manylinux_2_17_x86_64, got manylinux_2_28_x86_64 after auditwheel repair. Namespace the cache key by platform tag, and weigh the glibc-vs-musl split in manylinux2014 vs musllinux for spatial libs.
  • Reaching for --no-build-isolation to “make CC stick.” It does route compilation through ccache, but a backend that expected an isolated environment now finds host packages and the build stops being reproducible. Pass CMAKE_C_COMPILER_LAUNCHER through --config-settings instead and keep isolation on.
  • Not invalidating the cache when GDAL/PROJ change. A bumped GDAL_VERSION with a stale object cache links against old headers and fails at runtime with undefined reference to 'proj_create'. Fold GDAL_VERSION/PROJ_VERSION into the cache key (or pin them in the image as above) so a system-dependency change forces a clean recompile.