Debugging Import Errors and Linker Failures in Spatial Wheels

When a geospatial wheel installs cleanly but blows up on import, the traceback is almost always a symptom of a link that was resolved wrong at build time — a missing SONAME, an undefined symbol, or a loader that found the host’s library instead of the bundled one. This guide sits under the Geospatial C-Extension Fundamentals & ABI Architecture reference and turns the opaque ImportError/OSError surface into a systematic diagnostic procedure: read the error class, inspect the binary with the right tool, and map the finding to a fix. It targets GDAL 3.6–3.9, PROJ 9.2+, GEOS 3.11+, auditwheel 6.x on Linux and delocate 0.11+ on macOS, and assumes wheels built to the ABI contract described in C-API vs CPython ABI compatibility.

Prerequisites & Environment #

Effective binary debugging needs the target’s own inspection tools, not the host’s guesses. Install and confirm the following before diagnosing:

• binutils (nm, readelf, objdump) and patchelf on Linux; otool and install_name_tool on macOS (ship with Xcode command-line tools).
• auditwheel 6.x and delocate 0.11+ — both a repair tool and the best inspector of what a wheel actually links.
• A clean runtime image, python:3.12-slim, with none of your build dependencies installed. Every import bug reproduces there and hides on the build box.
• The exact wheel under test, unzipped: unzip -o dist/*.whl -d /tmp/w. You debug the artifact, never the source tree.

# Confirm the toolchain sees the wheel's extension module
unzip -o dist/*.whl -d /tmp/w >/dev/null
find /tmp/w -name '*.so' -o -name '*.pyd' -o -name '*.dylib'

Core Configuration: three error classes, three tools #

Every geospatial import failure sorts into one of three classes, and each class has one authoritative diagnostic:

The distinction matters because the fixes diverge completely: a missing SONAME is a repair problem, an undefined symbol is a link-line problem, and a version mismatch is a loader-precedence problem. Guessing wastes the most time; reading the class first is the whole method. The deep dives are fixing “libgdal.so: cannot open shared object file” and diagnosing undefined-symbol errors in spatial extensions.

Step-by-Step Implementation #

1. Reproduce in a clean container. If it imports on the build box, you are debugging the wrong environment:

   docker run --rm -v "$PWD/dist:/d" python:3.12-slim \
     bash -c "pip install /d/*.whl && python -c 'from osgeo import gdal'"
   

2. Capture the exact error text. Copy the last line verbatim; the SONAME version (libgdal.so.34) and symbol name are the entire clue.

3. Classify with the table above, then run only that class’s tool:

   ldd /tmp/w/*.so | grep -i 'not found'      # class A
   nm -D /tmp/w/*.so | grep ' U '             # class B
   readelf -d /tmp/w/*.so | grep -E 'RPATH|RUNPATH|NEEDED'   # class C
   

4. Apply the class-specific fix (repair / relink / dtags), rebuild, and return to step 1. Never fix two classes at once — you lose the signal.

Verification #

A correctly linked geospatial wheel passes all three of these in a clean container:

# No unresolved dependencies
ldd /tmp/w/*.so | grep -c 'not found'          # expected: 0

# No undefined symbols outside the permitted base libc set
nm -D /tmp/w/*.so | grep ' U ' | grep -vE 'GLIBC|__gmon|_ITM'   # expected: empty

# Bundled libraries resolve via $ORIGIN, not the host
readelf -d /tmp/w/*.so | grep RUNPATH          # expected: [$ORIGIN/../name.libs]

The acceptance test is unchanged from the parent reference: python -c "from osgeo import gdal; print(gdal.__version__)" in a base image with no GDAL installed. If that prints a version, the link graph is sound end to end.

Where Each Class Was Introduced #

Each of the three error classes was created at a specific point in the build, and each is detectable there — earlier, more cheaply, and with a message that names the cause rather than the symptom.

Adding all four to CI is perhaps twenty lines of configuration, and between them they move essentially every failure in this page from a user’s traceback to a red build.

Optimization & Edge Cases #

• macOS uses different verbs. ldd becomes otool -L, RPATH inspection becomes otool -l | grep -A2 LC_RPATH, and repair is delocate-wheel. The three classes are identical; only the tools rename.
• auditwheel show is a fast pre-flight. Before hand-inspecting, auditwheel show wheel.whl lists external references and the platform tag it would assign — a lower tag than expected is an early warning that a link is wrong.
• Emulated tests can lie. An aarch64 wheel imported under QEMU may resolve a host library that will not exist on real hardware; validate on native arches, as cross-compiler toolchain setup explains.

Troubleshooting #

ldd: exited with unknown exit status or a hang. ldd executes the object to resolve symbols and can run init code; prefer objdump -p extension.so | grep NEEDED for a static, side-effect-free list of dependencies.

nm: no symbols. The extension was stripped. Inspect the dynamic table with nm -D, which survives stripping, or rebuild without -s for a debugging pass.

A symbol shows defined but import still fails. Two copies of the same library are loaded and one shadows the other — a symbol-visibility problem covered in symbol visibility and namespace isolation.

The Decision Tree, in Order #

The three-class table above is the classification; this is the procedure that uses it. Working the steps in order matters more than knowing any individual command, because each step eliminates a family of causes and narrows what the next step has to consider. Skipping ahead — the universal temptation — is how an afternoon disappears into LD_LIBRARY_PATH experiments for a problem that was a missing link-line entry.

The rightmost branch of that tree is worth dwelling on, because it is the one people most often mis-diagnose. version 'GDAL_3.8' not found does not mean the library is absent; it means a library was found and it is the wrong one. Almost always the host’s libgdal was picked up ahead of the bundled copy, which happens when the extension’s RUNPATH is missing, when LD_LIBRARY_PATH is set in the environment (it takes precedence over RUNPATH, though not over the legacy RPATH), or when the repair step wrote its rewrite into some objects but not all of them.

There is a second-order version of the same problem that is even harder to see: the bundled library loads correctly, but it depends on something that resolves to the host. A wheel that bundles libgdal but not libproj will import successfully on any machine with PROJ installed, and fail with a version error on machines whose PROJ is older. auditwheel show catches this by listing external references, which is why it is worth running as a pre-flight before hand-inspection begins.

Reading a Binary Without Guessing #

Four tools cover essentially every question you will need to ask of a spatial extension, and knowing which one answers which question removes most of the flailing.

objdump -p prints the dynamic section without executing anything, and it is the right first look. It lists the NEEDED entries — the sonames this object will ask the loader for — plus RPATH, RUNPATH and the soname of the object itself. Prefer it over ldd for anything untrusted, because ldd works by invoking the dynamic loader on the object, which can run initialiser code.

nm -D shows the dynamic symbol table: what the object defines (T, D, B) and what it expects someone else to define (U). For an undefined-symbol failure this is the whole diagnosis: find the symbol, check whether any bundled library defines it, and if none does, the link line was incomplete. The common spatial case is a symbol from a C++ library — a mangled _ZN4geos... name — which means GEOS was linked as C but used as C++, or two GEOS builds with different C++ standard library assumptions were mixed.

readelf -d answers precedence questions. The distinction between RPATH and RUNPATH is the one that matters: RPATH is consulted before LD_LIBRARY_PATH, RUNPATH after. Modern toolchains emit RUNPATH by default, which is usually what you want for a wheel — a user who sets LD_LIBRARY_PATH deliberately can override your bundled libraries — but it also means an unrelated LD_LIBRARY_PATH in a user’s shell profile can hijack your load. If you need the bundled copy to win unconditionally, --disable-new-dtags restores RPATH semantics.

auditwheel show is the summary view, and it is the only one that reasons about policy rather than mechanics: it names the platform tag the wheel qualifies for and the libraries it still expects from outside. Run it first, then drop to the lower-level tools for whatever it flags.

# The four questions, in the order they are usually worth asking
auditwheel show dist/*.whl                       # policy: tag + external refs
objdump -p /tmp/w/*.so | grep -E 'NEEDED|RPATH|RUNPATH|SONAME'
nm -D /tmp/w/*.so | awk '$2=="U" {print $3}' | sort -u | head -40
readelf -d /tmp/w/*.so | grep -E 'RPATH|RUNPATH'

One habit is worth building: record this output in CI for every release, not just when something breaks. The reports are small, they diff cleanly, and when a regression appears the diff between this release and the last frequently identifies the cause in one line — a NEEDED entry that appeared, a RUNPATH that vanished — without reproducing anything.

Failures That Are Not Import Failures #

A meaningful share of reports that arrive as “the wheel does not import” are not link problems at all, and recognising them early saves the whole procedure above. Four shapes account for most of them.

The first is a data problem wearing an import error’s clothes. pyproj raises DataDirError: Valid PROJ data directory not found at import time, not at first use, because the module initialises a context eagerly. The binary loaded perfectly; proj.db is missing or the search path points somewhere that does not exist in the installed layout. Nothing in ldd or nm will show anything wrong, and every minute spent on the linker is wasted. The fix belongs in packaging, covered by bundling proj.db and datum grids in a wheel.

The second is an interpreter mismatch. ImportError: dynamic module does not define module export function means the loader found the object and could not find PyInit_<name> inside it. In an abi3 build the usual cause is a module name that does not match the init symbol, or an extension compiled without Py_LIMITED_API being loaded by an interpreter it was not built for. nm -D confirms it in one command: the init symbol is either absent or spelled differently from the module.

The third is a partially-installed package. A wheel unpacked over a previous version can leave a stale .so from the old release beside the new one, and Python will import whichever the finder reaches first. The signature is an error mentioning a symbol or version that matches neither the installed version nor the one you built. Reinstalling into a fresh environment resolves it, and if that fixes it, the bug was never in the wheel.

The fourth is a second copy of the same library already in the process, which produces the strangest reports because the failure depends on import order. If import mypkg works alone but fails after import rasterio, or vice versa, the diagnosis is symbol collision rather than a broken link, and the remedy is the isolation work described in symbol visibility and namespace isolation rather than anything in this page.

A quick triage script separates these from genuine link failures in about ten seconds, and it is worth keeping in the repository so that a bug reporter can run it and attach the output:

python - <<'PY'
import importlib, sys, traceback
for name in ("osgeo.gdal", "pyproj", "shapely", "fiona", "rasterio"):
    try:
        m = importlib.import_module(name)
        print(f"ok   {name:14} {getattr(m, '__version__', '?')} -> {getattr(m, '__file__', '?')}")
    except Exception as exc:                      # noqa: BLE001 - triage script
        print(f"FAIL {name:14} {type(exc).__name__}: {exc}")
        traceback.print_exc(limit=1)
print(sys.version)
PY

Importing each package alone, then in different orders, distinguishes an unconditional failure (a genuine link problem) from an order-dependent one (a collision). Printing each module’s __file__ catches the stale-install case immediately, because the path will point somewhere unexpected. And printing the interpreter version catches the abi3 mismatch, since a wheel built for a floor of 3.9 loaded under 3.8 fails in exactly this way.

Frequently Asked Questions #

Why does the wheel import on my machine but not in CI? #

Because your machine has the libraries the wheel forgot to bundle. Development environments accumulate GDAL, PROJ and GEOS from system packages, Homebrew, conda and previous experiments, and any of them can satisfy a dependency the wheel should have carried. The clean container is not a formality — it is the only environment where the wheel’s own contents are what gets tested.

Is LD_LIBRARY_PATH ever the right fix? #

For a user working around a broken wheel today, yes. For the wheel itself, never: shipping instructions that require an environment variable means the artifact is not self-contained, and the variable will leak into unrelated processes and break other packages that bundle different versions of the same library. Treat it as a diagnostic that confirms a hypothesis, then fix the build.

What does an undefined C++ symbol tell me that a C symbol does not? #

That an ABI boundary was crossed with mismatched assumptions. Mangled names encode the parameter types, so an undefined _ZN4geos8geom11GeometryFactoryC1Ev means something expected a specific constructor signature from a specific GEOS build. The usual causes are mixing a libstdc++ build with a libc++ build, or compiling against headers from one GEOS version while linking a different one — both invisible until link or import time.

Can I diagnose a Windows import failure with the same method? #

The structure of the method holds — reproduce cleanly, classify, inspect, fix the build — but every tool changes. dumpbin /dependents replaces objdump -p, dumpbin /exports replaces nm -D, and there is no RPATH to inspect because Windows has none. The dominant failure class is correspondingly narrower: nearly every Windows import error is a DLL the loader never looked for in the right place, covered in fixing “DLL load failed” for GDAL on Windows.

Is there a single command that catches most of these before release? #

Close to it: install the wheel in a container with no geospatial libraries and import it. That one step exercises the repair, the load paths and the bundled data simultaneously, and it fails for every class described on this page. The static inspections then tell you which class, but the container is what tells you there is a problem at all.

Related #

• Fixing “libgdal.so: cannot open shared object file” — the missing-SONAME class, end to end.
• Diagnosing undefined-symbol errors in spatial extensions — the wrong-link-line class, with nm walkthroughs.
• Shared library path resolution — the loader-precedence rules behind the version-mismatch class.
• C-API vs CPython ABI compatibility — why a Stable-ABI violation surfaces as an undefined symbol at import.

Further Reading #

• auditwheel documentation (pypa/auditwheel) on the manylinux policy and repair internals.