Shared Library Path Resolution in Python Geospatial Wheels

When you compile a Python geospatial extension such as pyproj, rasterio, fiona, or shapely, the dynamic linker’s ability to locate native .so, .dylib, and .dll binaries at import time decides whether the wheel runs deterministically across CI runners, cloud data platforms, and end-user machines. This guide sits under the Geospatial C-Extension Fundamentals & ABI Architecture reference and isolates the path-resolution contract from the toolchain, ABI, and memory concerns covered elsewhere in that section. It targets CPython 3.8–3.12, cibuildwheel 2.16+, auditwheel 6.x on manylinux/musllinux, delocate 0.11+ on macOS, and delvewheel 1.4+ on Windows, and it focuses narrowly on how RPATH/RUNPATH, @loader_path, and the Windows DLL search model are embedded, repaired, and verified so a single archive resolves every dependency from inside itself.

The POSIX dynamic linker (ld.so), macOS dyld, and the Windows loader each follow a strict, platform-specific search order: embedded RPATH/RUNPATH directives first, then environment variables such as LD_LIBRARY_PATH or DYLD_LIBRARY_PATH, then system caches (/etc/ld.so.cache), and finally default directories like /usr/lib. Production geospatial wheels deliberately rely on only the first branch. Environment-variable lookups introduce non-determinism, break container isolation, and let the loader silently bind a host copy of libgdal whose symbol versions disagree with what the extension was compiled against — exactly the ABI hazard described in C-API vs CPython ABI Compatibility, where a mismatched libproj or libgeos corrupts coordinate transforms instead of failing loudly.

Prerequisites & Environment #

Pin every tool that touches the link or repair step before you start; path-rewriting behaviour changes between auditwheel, delocate, and delvewheel releases, and a relocatable wheel is only reproducible if the build environment is.

On Linux the build runs inside the manylinux_2_28 Docker base images that anchor the glibc baseline, so the only libraries allowed outside the wheel are those whitelisted by the platform tag. Lock the toolchain itself the same way you lock the interpreter matrix — a reproducible pixi environment (or a conda-lock file) keeps patchelf, cmake, and the compiler at fixed versions across every runner so the embedded paths never drift between a developer laptop and CI.

Decide early whether you are bundling native libraries or relying on host-provided ones, because that choice dictates what the run path must point at; the trade-offs are covered in Vendoring PROJ and GDAL vs System Libraries. The rest of this page assumes the production-default posture: vendor the C stack and resolve it from inside the wheel.

Core Configuration #

The single most important decision is which run-path token to embed and which dynamic-tags variant the linker writes. Use $ORIGIN on Linux and @loader_path on macOS — both resolve at load time relative to the directory holding the extension module, which is what makes the wheel relocatable no matter where pip unpacks it.

On Linux, always pair the -rpath flag with --enable-new-dtags so the linker emits DT_RUNPATH instead of the legacy DT_RPATH. The distinction is not cosmetic: DT_RPATH is searched before LD_LIBRARY_PATH and is not transitive to a library’s own dependencies, whereas DT_RUNPATH is searched after LD_LIBRARY_PATH and applies only to the object that declares it — the behaviour auditwheel expects when it patches the wheel.

# pyproject.toml — passed through cibuildwheel to the compiler and linker
[tool.cibuildwheel.environment]
LDFLAGS = "-Wl,-rpath,$ORIGIN/../lib -Wl,--enable-new-dtags -Wl,--no-undefined"
CFLAGS  = "-O2 -fPIC"

[tool.cibuildwheel.config-settings]
# When the project is driven by CMake via scikit-build-core, let CMake
# write the relative ORIGIN-based run path rather than an absolute one.
"cmake.define.CMAKE_BUILD_RPATH_USE_ORIGIN" = "TRUE"

If the build is driven by CMake through the scikit-build-core backend that translates pyproject.toml into CMake invocations, set the run path declaratively so you are not fighting CMake’s default of stripping the build-tree RPATH at install time:

# CMakeLists.txt — keep a relocatable run path on the installed extension
set_target_properties(_pyproj PROPERTIES
    BUILD_RPATH_USE_ORIGIN TRUE
    INSTALL_RPATH "$ORIGIN/../lib"
    INSTALL_RPATH_USE_LINK_PATH FALSE)

--no-undefined is included deliberately: it forces the link to fail at build time if a symbol is unresolved, rather than deferring the failure to a runtime ImportError on a user’s machine. The full pyproject wiring for these environment blocks lives in Mastering pyproject.toml for Spatial Wheels.

Step-by-Step Implementation #

Each step below is runnable in CI. The sequence assumes a cibuildwheel job that has already produced an unrepaired wheel in dist/.

1. Build the unrepaired wheel with the run path embedded. The LDFLAGS from the previous section ensure the .so carries a relative DT_RUNPATH before any repair runs.

   python -m cibuildwheel --output-dir dist
   

2. Inspect the embedded run path before repair. Confirm the linker honoured --enable-new-dtags; you should see a RUNPATH (not RPATH) entry.

   unzip -o -q dist/*.whl -d /tmp/pre
   patchelf --print-rpath /tmp/pre/*/*.cpython-*.so
   # expected: $ORIGIN/../lib
   

3. Repair on Linux with auditwheel repair. This copies every external dependency into the wheel’s .libs directory, rewrites RUNPATH to $ORIGIN/.libs, and stamps the correct manylinux/musllinux platform tag.

   auditwheel repair dist/*.whl --plat manylinux_2_28_x86_64 -w wheelhouse
   

4. Repair on macOS with delocate-wheel. delocate invokes install_name_tool under the hood to strip absolute Homebrew/MacPorts paths and rewrite each reference to @loader_path.

   delocate-listdeps dist/*.whl
   delocate-wheel -w wheelhouse -v dist/*.whl
   

5. Repair on Windows with delvewheel. Python 3.8+ disabled PATH-based DLL resolution, so dependencies must be bundled and loaded through an injected init patch.

   delvewheel repair dist/*.whl -w wheelhouse
   

6. Promote only repaired wheels. Publish from wheelhouse/, never from the raw dist/ output, so an unrepaired archive with dangling DT_NEEDED entries can never reach an index.

Verification #

A wheel is correct only when every native dependency resolves from inside the archive with zero absolute paths. Treat the following as CI gates, not manual spot checks.

On Linux, extract the repaired wheel and prove that ldd finds nothing missing and that the run path points at the bundled .libs:

unzip -o -q wheelhouse/*.whl -d /tmp/post
patchelf --print-rpath /tmp/post/*/*.cpython-*.so   # expect: $ORIGIN/.libs
ldd /tmp/post/*/*.cpython-*.so | grep "not found" && exit 1
auditwheel show wheelhouse/*.whl                     # confirms platform tag + bundled libs
readelf -d /tmp/post/*/*.cpython-*.so | grep -E "RUNPATH|RPATH"

A clean result shows a (RUNPATH) entry and no (RPATH) entry. On macOS, otool -L should list only @loader_path/@rpath references plus system frameworks:

otool -L /tmp/post/*/*.so | grep -vE "@loader_path|@rpath|/usr/lib|/System" && exit 1

On Windows, confirm the repaired wheel actually imports in a clean interpreter that has no GDAL on PATH:

python -c "import rasterio; print(rasterio.__version__)"

The pass criterion is uniform across platforms: a deterministic wheel contains zero absolute paths, zero DT_RPATH entries, and a fully satisfied dependency closure inside the archive. The way these checks slot into auditwheel show output and the wider artifact layout is detailed in Build Artifact Structuring and Packaging.

RPATH, RUNPATH, and Who Wins #

The two dynamic-section entries look interchangeable and are not, and the difference decides whether a user’s environment can override your bundled libraries. Getting this wrong produces a wheel that works everywhere except on the machines of the users most likely to file a bug — the ones with their own GDAL installed.

The non-transitivity noted in the diagram is the subtler of the two facts and the cause of a specific bug: an extension with a correct RUNPATH loads libgdal from the package, and then libgdal looks for libproj using its own dynamic section. If the repair step wrote an entry only into the extension, libproj resolves from the host — producing a wheel that mixes a bundled GDAL with a system PROJ, which is exactly the combination that yields version errors in unrelated places.

Making the Resolution Observable #

Because path resolution happens inside the loader, the most useful debugging techniques are the ones that make it print what it is doing rather than the ones that inspect files.

LD_DEBUG=libs deserves to be better known. It prints each search path as the loader tries it, so a question like “why did it pick the system PROJ?” is answered in one run rather than inferred from file inspection. Pairing it with LD_DEBUG_OUTPUT writes the trace to a file, which keeps it out of the program’s own output and makes it attachable to a bug report.

Optimization & Edge Cases #

manylinux vs musllinux. The two libc families resolve and cache shared objects differently — glibc consults /etc/ld.so.cache populated by ldconfig, while musl walks /etc/ld-musl-*.path and has no equivalent cache daemon. A run path embedded as $ORIGIN/.libs works identically on both, which is precisely why embedding paths (rather than seeding a cache) is the portable choice. Build the musllinux leg in its own matrix entry; deep configuration of the glibc image and its loader cache is covered in Managing shared library paths in manylinux.

Cross-architecture builds. Compiling on x86_64 and running on aarch64 without a matching --sysroot or CMAKE_SYSTEM_PROCESSOR produces a .so whose DT_NEEDED entries reference libraries that do not exist on the target. The run path is correct but the soname is wrong, which surfaces only on the target machine. Align the linker and sysroot through the cross-compiler toolchain setup before trusting any emulated build.

Caching the repair step. auditwheel repair is deterministic given identical inputs, so cache the vendored dependency tree (for example a prebuilt /opt/vendor of PROJ/GDAL) rather than the wheel itself; the repair then becomes a fast copy-and-patch instead of a recompile. Tie the cache key to the dependency lockfile so a PROJ bump invalidates it — the caching mechanics are expanded in Async Build Execution and Cache Strategies.

Multiple extensions in one wheel. When a package ships several .so modules that share a vendored libgdal, point every module’s run path at the same $ORIGIN/.libs directory so only one copy is loaded. Divergent run paths load duplicate copies into the process, doubling resident memory and risking two live heaps — an allocation hazard explored in Memory Management in Geospatial Extensions.

Troubleshooting #

ImportError: libgdal.so.32: cannot open shared object file: No such file or directory — the loader found the extension but not its dependency. Either the repair step did not run, or RUNPATH was stripped. Re-run patchelf --print-rpath on the installed .so; if it does not show $ORIGIN/.libs, the wheel was published from dist/ instead of wheelhouse/. Re-repair and republish.

Silent fallback to a system library. ldd reports /usr/lib/libgdal.so.32 rather than the bundled $ORIGIN/.libs/libgdal-*.so. This means DT_RPATH (searched after nothing) was emitted instead of DT_RUNPATH, or ldconfig shadowed the bundled copy. Confirm with readelf -d; if you see (RPATH), you forgot --enable-new-dtags. Rebuild with the flag and re-repair.

auditwheel: error: cannot repair "dist/foo.whl" to "manylinux_2_28_x86_64" ABI because of the presence of too-recent versioned symbols — a vendored library links against a glibc newer than the platform floor. The path resolution is fine; the libc baseline is not. Build inside the pinned manylinux_2_28 image so the symbol versions match the tag.

OSError: [WinError 126] The specified module could not be found on Windows — the DLL search no longer consults PATH under the Python 3.8+ secure loading model. Repair the wheel with delvewheel so dependencies are bundled and the generated _delvewheel_init_patch_*.py adds them via os.add_dll_directory(); importing the unrepaired wheel will keep failing regardless of how PATH is set.

Frequently Asked Questions #

Should a wheel use RPATH or RUNPATH? #

RUNPATH for most packages, which is what modern toolchains emit by default. It leaves a user able to substitute their own library deliberately by setting LD_LIBRARY_PATH, which is occasionally what they want. The trade-off is that an unrelated LD_LIBRARY_PATH in a user’s shell can hijack the load, so a package that must guarantee its own copy wins should use --disable-new-dtags and say so in its documentation.

Why does the repair step write an entry into every bundled library? #

Because the search is per object, not per process. The extension finds libgdal through its own entry, and then libgdal looks for libproj through its entry. A wheel whose extension is rewritten but whose libraries are not will import on machines that happen to have PROJ installed and fail on the rest, which is one of the harder failure modes to reproduce.

Can I set the search path from Python before importing? #

Not usefully on Linux. LD_LIBRARY_PATH is read by the loader when the process starts, so changing os.environ inside a running interpreter has no effect on subsequent loads. Windows is the exception — os.add_dll_directory genuinely changes the search at run time — which is why the Windows shim exists and has no Linux counterpart.

How do I find out which library actually got loaded? #

Ask the loader: LD_DEBUG=libs python -c "import mypkg" prints every directory tried and the path that won. For a running process, reading /proc/<pid>/maps shows the files currently mapped, which is the definitive answer when two copies of a library are suspected.

Does any of this apply inside a container image? #

All of it, and containers make one failure less likely and another more so. A minimal image is unlikely to have a system GDAL to shadow yours, which removes the precedence problem. But a minimal image also lacks libraries that a poorly repaired wheel expected to borrow, so the missing-soname failure becomes more likely rather than less — which is exactly why the clean container is the right test environment.

Does static linking remove all of these concerns? #

It removes the search entirely for whatever is statically linked, which is why a fully static extension has no path resolution to get wrong. In practice most spatial wheels are partly static and partly bundled — GEOS statically, GDAL and PROJ as shared objects — so the rules still apply to whatever remains dynamic.

Related #

• Geospatial C-Extension Fundamentals & ABI Architecture — the parent reference covering compile, link, repair, and import for native geospatial extensions.
• Managing shared library paths in manylinux — container-level ldconfig and loader-cache control for the glibc baseline.
• C-API vs CPython ABI Compatibility — why a resolved-but-wrong host library corrupts spatial operations instead of failing cleanly.
• Vendoring PROJ and GDAL vs System Libraries — the dependency-sourcing decision that determines what your run path must point at.
• manylinux and manyarm Docker base images — the image pins that define which libraries may legitimately live outside the wheel.

Further Reading #

• PEP 599 / PEP 600 — the manylinux platform tag specifications (peps.python.org/pep-0599, peps.python.org/pep-0600).
• os.add_dll_directory and the Windows DLL search model (docs.python.org/3/library/os.html#os.add_dll_directory).