C-API vs CPython ABI Compatibility in Geospatial Python Wheels

When building Python GIS packages such as pyproj, rasterio, fiona, or hand-rolled GDAL bindings, the distinction between the Python C-API and the CPython ABI decides whether a single wheel runs across every supported interpreter or whether you must rebuild once per minor Python release. This guide sits under the Geospatial C-Extension Fundamentals & ABI Architecture reference and isolates the binary compatibility contract from the toolchain, linking, and memory concerns covered elsewhere in that section. It targets CPython 3.8–3.12, cibuildwheel 2.16+, auditwheel 6.x, setuptools 69+, and scikit-build-core 0.8+, and it focuses narrowly on how API declarations translate into runtime symbol resolution, wheel tags, and a smaller CI matrix for geospatial extensions.

Prerequisites & Environment #

Pin every moving part before you touch a single compiler flag. ABI behaviour shifts subtly between Python patch releases and between auditwheel versions, so reproducibility starts with the environment.

• Python interpreters: 3.8.0 is the floor for an abi3 tag of 0x03080000. The Stable ABI only guarantees that a wheel runs on the declared minimum and newer, so you compile against the oldest interpreter you intend to support.
• Build backend: setuptools>=69 (for reliable py_limited_api wheel naming) or scikit-build-core>=0.8. The CMake-driven path is documented under the scikit-build-core backend that translates pyproject.toml into CMake invocations.
• Repair tooling: auditwheel>=6.0 on Linux, delocate>=0.11 on macOS. Older auditwheel releases do not understand the manylinux_2_28 policy.
• Base image: build inside one of the manylinux_2_28 Docker base images that anchor glibc compliance, not your host distro. The host’s libpython and glibc version will otherwise bleed into the artifact.
• Native libraries: GDAL ≥ 3.6, PROJ ≥ 9.2, GEOS ≥ 3.11. Whether these are bundled or system-resolved is decided in Vendoring PROJ and GDAL vs System Libraries; the choice affects whether C++ runtime coupling can defeat your abi3 guarantee.

Lock the toolchain itself, not just the Python deps. A reproducible build environment — managed through a pixi environment lock file — pins the C/C++ compiler, GDAL, and PROJ to exact builds so the ABI you test in CI is the ABI you ship:

# pixi.toml — pin the native toolchain, not just Python
[dependencies]
python = "3.8.*"
gdal = "3.8.*"
proj = "9.3.*"
geos = "3.12.*"
c-compiler = "*"
cxx-compiler = "*"

Export the environment variables that the limited API depends on so they are identical locally and in CI:

export CFLAGS="-DPy_LIMITED_API=0x03080000 -fvisibility=hidden"
export SOURCE_DATE_EPOCH=1700000000   # deterministic wheel metadata

The Binary Compatibility Contract #

The Python C-API is a set of C headers (Python.h, numpy/arrayobject.h) exposing functions, macros, and type definitions. It guarantees source compatibility across the 3.x series but says nothing about binary compatibility. The CPython ABI, by contrast, fixes the exact memory layout of PyObject, the calling conventions, struct sizes, and symbol visibility for one specific interpreter build.

A standard extension compiles against the full C-API and embeds version-specific symbols (for example _Py_NoneStruct, PyUnicode_FromString, or inline expansions of Py_INCREF). That hard-codes a dependency on one Python minor version and produces a wheel tagged like cp310-cp310-manylinux_2_28_x86_64.

The Stable ABI exposes a restricted, forward-compatible subset through the Py_LIMITED_API macro. When the extension is compiled with -DPy_LIMITED_API=0x03080000 and linked against the shared Python runtime (python3.dll on Windows, the interpreter’s exported table on POSIX), the resulting .cpython-38-abi3.so runs unchanged on every CPython 3.8 and later. The contract forbids direct access to internal structures: you must use opaque pointers and the public accessor functions (PyObject_GetAttrString instead of reaching into the struct, PyTuple_GetItem instead of indexing ob_item).

Two rules follow directly and govern everything below:

1. Any symbol your .so imports from CPython must belong to the limited API for the version you declared, or the wheel is silently mistagged and will crash on a different interpreter.
2. The hex version in Py_LIMITED_API is the floor. Declare 0x03080000 even when building on 3.12 if you want 3.8 compatibility — the interpreter you build on does not set the floor; the macro does.

Geospatial-Specific ABI Constraints #

For geospatial stacks, adopting abi3 is a deliberate trade-off rather than a free win. GDAL’s SWIG-generated wrappers, PROJ’s C-API bindings, and raster I/O layers frequently lean on full C-API features that the limited API excludes:

• Direct PyCapsule manipulation for passing C pointers (GDAL dataset handles, PROJ transformation contexts) between modules.
• NumPy C-API struct access (PyArrayObject, PyArray_Descr) for zero-copy coordinate and raster buffers.
• Inline type-check macros (Py_TYPE(), Py_SIZE()) that expanded to struct field reads before they became function calls.

Historically the NumPy C-API was incompatible with Py_LIMITED_API because its headers exposed struct layouts directly. NumPy 2.0+ ships limited-API-compatible headers, but a geospatial package must still audit its Cython and SWIG interfaces by hand. When a PyCapsule wraps a GDAL dataset or a PROJ context, register the destructor through the public PyCapsule_New API rather than relying on internal reference-count tricks — the capsule lifecycle is exactly where ownership crosses the interpreter boundary, a topic developed in Memory Management in Geospatial Extensions.

Vendored native libraries add a second hazard. A statically linked libgdal or libproj can drag in C++ RTTI and exception-handling machinery whose symbol versioning is tied to the toolchain, not to Python — but a careless #include <Python.h> inside a vendored C++ header (without the limited-API guard) re-introduces full C-API coupling and quietly breaks abi3. Guard every translation unit, not just the module entry point.

Core Configuration #

The primary control surface is pyproject.toml, the canonical layout for which is covered in mastering pyproject.toml for spatial wheels. Two things must agree: the build frontend (cibuildwheel) must inject the limited-API flag, and the backend must emit an abi3-tagged wheel.

# pyproject.toml
[tool.cibuildwheel]
# Build one interpreter per platform; abi3 covers the rest.
build = ["cp38-*"]
skip = ["pp*", "*-musllinux*"]
environment = { CFLAGS = "-DPy_LIMITED_API=0x03080000 -fvisibility=hidden" }

[tool.cibuildwheel.linux]
before-all = "yum install -y proj-devel gdal-devel || (apt-get update && apt-get install -y libproj-dev libgdal-dev)"

# Tell bdist_wheel to emit an abi3 wheel tagged cp38-abi3-*
[tool.distutils.bdist_wheel]
py-limited-api = "cp38"

For a setuptools project, the Extension object must also opt in, or the wheel keeps a full-ABI tag even though the code compiled against the limited API:

# setup.py — both halves are required to get abi3 naming
from setuptools import setup, Extension

setup(
    ext_modules=[
        Extension(
            "_geospatial_ext",
            sources=["src/_geospatial_ext.c"],
            define_macros=[("Py_LIMITED_API", "0x03080000")],
            py_limited_api=True,          # without this the tag stays cp38-cp38
            extra_compile_args=["-fvisibility=hidden"],
        )
    ]
)

A CMake-based project routes the same intent through scikit-build-core, which wires up the SABI components automatically when you set the wheel API floor:

# pyproject.toml — scikit-build-core variant
[tool.scikit-build]
wheel.py-api = "cp38"     # emits the abi3 tag; CMake links Python::SABIModule

Step-by-Step Implementation #

Each step below is runnable; do not skip the audit step, because the limited-API tag is a promise the compiler does not enforce at link time.

1. Inventory the non-limited surface. Find every place the code reaches past the Stable ABI before you commit to it:

   grep -rEn 'PyArrayObject|PyArray_Descr|->ob_type|->ob_size|Py_TYPE\(|Py_SIZE\(' src/
   

2. Guard NumPy and third-party includes. Ensure NumPy is built against the limited API and that no vendored header pulls in Python.h unguarded:

   #define Py_LIMITED_API 0x03080000
   #define NPY_TARGET_VERSION NPY_1_22_API_VERSION
   #include <Python.h>
   #include <numpy/arrayobject.h>
   

3. Compile against the floor interpreter. Build inside the pinned manylinux image so the linker resolves only the host interpreter’s Stable ABI table:

   CFLAGS="-DPy_LIMITED_API=0x03080000 -fvisibility=hidden" python -m build --wheel
   

4. Confirm the wheel filename carries abi3. The tag is the contract pip relies on:

   ls dist/
   # _geospatial_ext-1.0.0-cp38-abi3-linux_x86_64.whl   <-- abi3, not cp38-cp38
   

5. Repair to a portable tag. Run auditwheel so the wheel advertises a glibc baseline and bundles its native dependencies:

   auditwheel repair dist/*-linux_x86_64.whl --plat manylinux_2_28_x86_64 -w wheelhouse/
   

6. Promote one platform build to the full matrix. With abi3, cibuildwheel produces a single Linux wheel that installs on 3.8 through 3.12, collapsing what was 5 versions × 3 OS × 2 arch = 30 builds down to one per platform/architecture.

Verification #

Verification is the only place the abi3 promise is actually checked. Run all three; a green build with a leaked symbol passes CI but fails on a user’s interpreter.

List the CPython symbols the extension imports and confirm each one belongs to the limited API:

nm -D build/lib*/_geospatial_ext.abi3.so | grep ' U Py'
# Every line must be a documented Stable ABI symbol, e.g.:
#          U PyCapsule_New
#          U PyObject_GetAttrString

Ask auditwheel what the wheel actually needs and what tag it qualifies for:

auditwheel show wheelhouse/*.whl
# This constrains the wheel to: manylinux_2_28_x86_64
# The wheel references the following external versioned symbols: GLIBC_2.17 ...

Finally, install the same wheel into multiple interpreters and import it for real:

for v in 3.8 3.10 3.12; do
  python$v -m venv /tmp/v$v && /tmp/v$v/bin/pip install wheelhouse/*.whl
  /tmp/v$v/bin/python -c "import _geospatial_ext; print('$v', 'ok')"
done
# Expected: 3.8 ok / 3.10 ok / 3.12 ok  — one wheel, three interpreters

Optimization & Edge Cases #

• Matrix pruning is the headline win. Once the wheel is genuinely abi3, drop every redundant interpreter from tool.cibuildwheel.build. Keep one cp38-* Linux build, one macOS build per architecture, and one Windows build. Build-cache keys should hash pyproject.toml plus the pinned GDAL/PROJ versions so cache invalidation tracks the native ABI, not just the Python code.
• musl vs glibc still forks the build. The Stable ABI removes the Python version axis but not the libc axis. A manylinux wheel will not load on Alpine; if you support musl you still need a parallel musllinux_1_2 build, because the abi3 tag says nothing about the C library. The trade-offs are detailed in manylinux2014 vs musllinux for spatial libs.
• Cross-compilation gotcha. When building aarch64 wheels on an x86_64 host, the limited-API headers are architecture-independent but the linked libpython3.so is not. Use the target image’s interpreter, never the host’s, or auditwheel will reject the wheel for mixed-architecture symbols.
• Partial abi3 is worse than none. If even one translation unit forgets the Py_LIMITED_API define, the final .so imports a non-limited symbol while still being tagged abi3. The wheel installs everywhere and crashes on any interpreter newer than the build host. Make the define a project-wide compiler flag, never a per-file #define.
• Windows symbol set is the strictest. python3.dll exports only Stable ABI symbols, so Windows often surfaces leakage that Linux’s RTLD_GLOBAL resolution hides at import time. Treat a clean Windows build as the real abi3 gate.

Troubleshooting #

ImportError: /opt/venv/lib/.../_geospatial_ext.abi3.so: undefined symbol: PyUnicode_AsUTF8 The extension imports PyUnicode_AsUTF8, which only entered the limited API in 3.10. On a 3.8 or 3.9 interpreter the symbol is absent. Fix: switch to PyUnicode_AsUTF8AndSize (limited since 3.10 as well) or raise your declared floor to 0x030A0000, accepting that 3.8/3.9 users lose the wheel.

ValueError: Unable to find pure Python or extension build tag for ... cp38-abi3 — actually surfacing as the wheel staying cp38-cp38. Root cause: py_limited_api=True was set on the Extension but py-limited-api = "cp38" was missing from [tool.distutils.bdist_wheel] (or vice versa). Both halves are mandatory; set them together and rebuild.

auditwheel: error: cannot repair "..." to "manylinux_2_28_x86_64" ABI because of the presence of too-recent versioned symbols A vendored libgdal or libstdc++ pulled in a symbol newer than the policy allows. This is a native-library ABI problem, not a Python one — rebuild GDAL inside the manylinux image, or lower the target policy. The version-mismatch workflow is laid out in how to fix ABI version mismatch in GDAL wheels.

ImportError: dynamic module does not define module export function (PyInit__geospatial_ext) The PyInit_* entry point was compiled without limited-API visibility, or -fvisibility=hidden hid it without a matching PyMODINIT_FUNC export. Confirm the init function is declared PyMODINIT_FUNC and that it is the one symbol left visible. Where symbol leakage originates, the full step-by-step C-extension lifecycle for Python GIS traces each phase from PyInit_* through teardown.

ABI Strategy Decision Reference #

Use this to pick a strategy before wiring the matrix. The flow captures the one question that decides everything: do you touch internal structs?

Adopting the Stable ABI in geospatial packaging requires upfront interface auditing, but it yields compounding CI savings and simpler dependency resolution. When full C-API features are unavoidable, strict symbol isolation and automated auditwheel verification remain the only reliable path to production-grade wheels.

Related #

• Geospatial C-Extension Fundamentals & ABI Architecture — the parent reference covering compile, link, repair, and import for native geospatial extensions.
• how to fix ABI version mismatch in GDAL wheels — diagnostics for undefined symbol and libgdal.so.NN collisions downstream of a mistagged wheel.
• step-by-step C-extension lifecycle for Python GIS — the full path from PyInit_* to teardown where symbol leakage usually appears.
• Memory Management in Geospatial Extensions — how PyCapsule ownership and native heap teardown interact with the ABI boundary.
• Vendoring PROJ and GDAL vs System Libraries — why static C++ runtime coupling can defeat an otherwise clean abi3 build.

Further Reading #

• PEP 384 — Defining a Stable ABI (peps.python.org/pep-0384).
• CPython Stable Application Binary Interface (docs.python.org/3/c-api/stable.html).