Symbol Visibility and Namespace Isolation for Spatial Extensions

When two Python geospatial packages each vendor their own GEOS or PROJ and both are imported into one interpreter, their identically-named symbols collide in the process’s global symbol table, and the second dlopen can bind the first library’s function pointers against the second library’s data structures — a corruption that surfaces as a mid-run Segmentation fault. This guide sits under the Geospatial C-Extension Fundamentals & ABI Architecture reference and covers the compile-time and link-time controls — -fvisibility=hidden, linker version scripts, and RTLD_LOCAL loading — that keep each extension’s native dependencies in their own namespace. It targets GCC 13 / Clang 16, GEOS 3.11+, GDAL 3.8+, and wheels built to the abi3 contract from C-API vs CPython ABI compatibility.

Global symbol export versus hidden visibility for two vendored GEOS copies On the left, two extensions each export their vendored GEOS symbols globally, so the process symbol table holds two definitions of GEOSGeomCreate and the second load binds against the first, causing a crash. On the right, both extensions compile with hidden visibility and a version script exporting only their module init function, so each keeps a private GEOS and no collision occurs. global export → collision hidden → isolated ext A + GEOS exports GEOS* ext B + GEOS exports GEOS* one global symbol table B binds A's GEOSGeomCreate Segmentation fault ext A + GEOS exports PyInit only ext B + GEOS exports PyInit only private GEOS per module no shared symbols imports ✓

Prerequisites & Environment

  • GCC 13 or Clang 16 with support for -fvisibility=hidden and __attribute__((visibility("default"))).
  • A linker (ld/lld) that honours version scripts (--version-script) on Linux, or -exported_symbols_list on macOS.
  • The extension’s vendored GEOS/PROJ built as static archives or private shared objects, not linked against a shared system copy.
  • nm -D and readelf --dyn-syms to audit what the finished extension exports.
# See how many symbols your extension currently exports — should be ~1
nm -D --defined-only build/_geospatial_ext*.so | wc -l

A healthy abi3 geospatial extension exports exactly one symbol: its PyInit_ function. Anything more is a collision risk.

Core Configuration

Two flags do most of the work. -fvisibility=hidden makes every symbol hidden by default, so only symbols you explicitly mark as visible are exported; a version script then narrows the export list to the single module entry point.

# Compile every translation unit with hidden default visibility
CFLAGS += -fvisibility=hidden -fvisibility-inlines-hidden
# Export only the module init symbol via a version script
LDFLAGS += -Wl,--version-script=export.map

# export.map — the ONLY symbol the extension exposes to the process
{
  global: PyInit__geospatial_ext;
  local:  *;
};

The local: * line hides everything else, including the entire vendored GEOS and PROJ symbol set, so a second extension’s copy can never bind against it. This is the compile-time complement to the runtime isolation covered in securely compiling spatial C-extensions.

Step-by-Step Implementation

  1. Mark the module init function visible. With hidden default visibility, the init function must be explicitly exported or the interpreter cannot find it:

    #define PYEXPORT __attribute__((visibility("default")))
PYEXPORT PyMODINIT_FUNC PyInit__geospatial_ext(void) { /* ... */ }

  2. Add the version script to the link. Confirmed above; on macOS use -Wl,-exported_symbols_list,export.syms with a single _PyInit__geospatial_ext line.

  3. Static-link the geospatial dependencies so their symbols are absorbed into the extension and then hidden, rather than exported by a shared object the loader can see.

  4. Audit the export list before shipping (see Verification). One symbol out means one symbol that can collide.

Verification 

# 1. The extension must export exactly its init symbol
nm -D --defined-only build/_geospatial_ext*.so | grep -v ' PyInit_'
# expected: empty

# 2. GEOS/PROJ symbols must NOT be dynamically exported
readelf --dyn-syms build/_geospatial_ext*.so | grep -iE 'GEOS|proj_' | grep -i default
# expected: empty (no DEFAULT-visibility GEOS/PROJ symbols)

# 3. Two-extension co-import smoke test
python -c "import _geospatial_ext, shapely, pyproj; print('coexist ok')"
# expected: coexist ok  (no Segmentation fault)

A clean run shows only PyInit_ exported and both packages importing together. If GEOS symbols still appear with DEFAULT visibility, the version script did not apply — confirm it is on the final link, not an intermediate object.

Optimization & Edge Cases

  • RTLD_LOCAL is Python’s default, but C++ statics fight it. CPython loads extensions with RTLD_LOCAL, yet a shared C++ runtime can still deduplicate globals. Static-linking libstdc++ (or ensuring a single vendored copy) prevents cross-extension state bleed, as memory management in geospatial extensions discusses for allocator arenas.
  • Hidden visibility shrinks the binary. Fewer dynamic symbols means a smaller dynamic symbol table and faster load — a free win alongside the isolation.
  • Third-party wheels you do not control. If shapely or pyproj export their GEOS globally, your only defence is to keep your copy hidden; you cannot un-export theirs.

Troubleshooting

ImportError: dynamic module does not define module export function (PyInit__geospatial_ext). Hidden visibility hid the init symbol too. Add __attribute__((visibility("default"))) to the init function and confirm the version script lists it under global:.

Segmentation fault only when a specific second package is imported. Classic symbol collision. Audit both extensions’ exports; hide yours with the version script. See diagnosing undefined-symbol errors for the inverse case where hiding too much leaves a symbol undefined.

Version script silently ignored. Passing --version-script to the compiler instead of the linker drops it. Route it through -Wl, and verify with readelf --dyn-syms.

How a Collision Actually Happens

The word “collision” makes the failure sound like a name clash that a compiler could catch. It is not. It is a runtime binding decision made by the dynamic loader, at a moment when neither package knows the other exists, and it follows a rule that is entirely reasonable in isolation: the first definition of a symbol to enter the process’s global scope wins, and every later reference binds to it.

Walk through the sequence with two packages that each vendor GEOS. The interpreter imports package A, whose extension pulls in a private libgeos_c.so.1 built from GEOS 3.11. That library’s symbols — GEOSGeom_createPoint_r, GEOSContext_setNoticeHandler_r, several hundred more — are added to the process. The interpreter then imports package B, whose extension was compiled and linked against GEOS 3.12, where one of those functions gained a parameter and a struct grew a field. B’s extension asks the loader for GEOSGeom_createPoint_r; the loader already has one, from A; and B’s carefully-built GEOS 3.12 code now calls into GEOS 3.11 with a 3.12 struct layout.

Nothing fails at that moment. The call succeeds, reads a field at an offset that means something else, and returns a pointer that is plausible. The crash arrives later, usually in a function neither package’s author has ever looked at, with a stack that implicates the innocent caller. That delay is what makes these bugs expensive, and it is why the defence has to be structural rather than diagnostic.

Why the second extension binds to the first extension's GEOS A timeline in four steps. Package A is imported and its vendored GEOS 3.11 symbols enter the process global scope. Package B is imported next, having been compiled against GEOS 3.12. The loader resolves B's GEOS references to the already-present 3.11 definitions rather than B's own copy. B's 3.12 struct layouts are then passed into 3.11 code, and the process crashes later in unrelated work. A side panel shows the same sequence with hidden visibility, where B's references resolve inside its own module and no cross-binding occurs. default visibility — symbols escape into the process 1 · import A vendored GEOS 3.11 process global symbol scope GEOSGeom_createPoint_r → A's 3.11 definition 2 · import B built for GEOS 3.12 3 · B's calls bind to A's code 3.12 struct layout, 3.11 field offsets 4 · Segmentation fault, elsewhere, later stack implicates whichever call touched the bad field hidden visibility — nothing escapes A + its GEOS exports PyInit_ only B + its GEOS exports PyInit_ only process global symbol scope contains two PyInit_ names, no GEOS symbols at all each module resolves inside itself two GEOS versions coexist without ever meeting the cost is one linker flag and one map file; the benefit is that co-installation stops being a compatibility matrix you have to maintain CPython loads extensions with RTLD_LOCAL, which is necessary but not sufficient: transitive dependencies can still be global

That last line deserves emphasis, because it is the detail that surprises people who believe RTLD_LOCAL already solves this. CPython does load extension modules with RTLD_LOCAL, so the extension’s own symbols stay private. But the shared objects the extension depends on are loaded according to their own linkage, and a libgeos_c.so.1 that was built with default visibility exports its full symbol table into whatever scope it lands in. Worse, some packages deliberately re-open their extension with RTLD_GLOBAL to make plugins work, which promotes the whole dependency chain to global scope and re-creates the problem for everyone else in the process.

Three Ways to Keep Two Copies Apart

Hiding is one of three isolation techniques, and they differ in what they actually separate. Choosing between them is easier when the axes are laid out rather than argued about.

Static linking with hidden visibility, symbol prefixing, and separate processes compared Three techniques compared across four properties. Static linking with hidden visibility isolates symbols and library state, costs wheel size, and works for nearly all cases. Symbol prefixing also isolates symbols and state and allows two copies of the same version, but is fragile for C plus plus code. Separate processes isolate everything including global registries and crashes, at the cost of serialisation and inter-process communication. isolates symbols isolates library state survives a crash main cost use when static + hidden yes yes no wheel size — one copy of GEOS per package the default; covers nearly every case symbol prefixing yes yes no fragile for C++: mangled names and RTTI resist it two copies of the same version must coexist separate process yes yes yes serialisation and IPC on every call boundary a driver insists on global registration, or input is untrusted

The third column is the only one that isolates failure as well as naming, which is why it reappears in the security discussion under security boundaries and sandboxing — the same mechanism, chosen for a different reason.

Static Linking, Prefixing, and When Hiding Is Not Enough

Hidden visibility solves the export side of the problem. It does not solve the case where two copies of a library must genuinely coexist and each needs to keep its own process-wide state. GEOS, PROJ and GDAL all maintain global state: a driver registry, a context cache, error handlers, a lazily-initialised database connection. Two copies means two registries, and code that assumes there is one — for example, an error handler installed by package A that package B’s copy never sees — behaves in ways that look like intermittent bugs.

Three techniques address that layer, in increasing order of effort.

Static linking with hidden visibility is the default answer and the one most spatial extensions should use. The vendored library is compiled into the extension as an archive, its symbols are hidden by the version script, and the result is a single .so with one exported name. State is genuinely private, load time drops because there is one object to map instead of five, and there is no soname for anything else to bind to. The cost is size — every extension that vendors GEOS carries its own copy — and the loss of the ability to upgrade the library without rebuilding.

Symbol prefixing goes further by renaming the library’s symbols at build time, typically with a -D macro rename or objcopy --prefix-symbols. This is what several JavaScript and Rust ecosystems do routinely, and it is worth reaching for when a library’s headers make a static build awkward, or when you need two copies of the same version to be genuinely independent. It is fragile in C++ because mangled names and RTTI comparisons are involved, and GEOS in particular relies on C++ internals that do not survive naive renaming.

Separate processes is the technique nobody wants and everybody eventually needs for at least one case. When a driver or plugin insists on global registration, and its assumptions conflict irreconcilably with another package’s, the honest fix is to move it out of the interpreter — a subprocess, a worker, a service — and accept the serialisation cost. It is worth naming as an option because teams often spend weeks on linker archaeology before considering it.

# Prefix every symbol in a static archive before linking it in
objcopy --prefix-symbols=mypkg_ vendor/libgeos.a vendor/libgeos-prefixed.a
nm vendor/libgeos-prefixed.a | grep -c ' T mypkg_GEOS'   # sanity: non-zero

Whichever technique you choose, the audit does not change: the finished extension should export exactly one symbol. Make that assertion part of the build rather than a habit, because it is the single check that catches every regression in this area, including the ones introduced by a dependency’s own build system rather than your code.

check-exports: build
	@count=$$(nm -D --defined-only $(EXT) | grep -vc ' PyInit_'); \
	 if [ "$$count" != "0" ]; then \
	   echo "FAIL: $$count symbols leak from $(EXT)"; nm -D --defined-only $(EXT) | grep -v ' PyInit_'; \
	   exit 1; \
	 fi; echo "exports ok"

Making the Rule Survive a Build System

Version scripts have a reputation for being set once and then quietly stopping working, and the reason is almost never the script itself. It is that the flag reaches the wrong invocation. Three build systems dominate spatial packaging, and each drops linker flags in a different place.

With scikit-build-core driving CMake, the flag belongs on the target, not in CFLAGS. target_link_options applied to the extension target survives the generator, whereas a global CMAKE_SHARED_LINKER_FLAGS set after add_library may not be picked up at all. The visibility properties have first-class CMake support and should be used in preference to raw flags, because CMake then propagates them correctly to every source file including those from vendored subprojects.

set_target_properties(_geospatial_ext PROPERTIES
  C_VISIBILITY_PRESET hidden
  CXX_VISIBILITY_PRESET hidden
  VISIBILITY_INLINES_HIDDEN ON)
if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
  target_link_options(_geospatial_ext PRIVATE
    "-Wl,--version-script=${CMAKE_CURRENT_SOURCE_DIR}/export.map")
elseif(APPLE)
  target_link_options(_geospatial_ext PRIVATE
    "-Wl,-exported_symbols_list,${CMAKE_CURRENT_SOURCE_DIR}/export.syms")
endif()

With meson-python, the equivalent is link_args on the extension_module call, guarded by a host-machine check for the same reason. Meson’s gnu_symbol_visibility : 'hidden' keyword does the compile side, and it applies per target rather than per project, so a vendored subproject compiled as a static library needs the setting too — a subtlety that produces the exact symptom this section exists to prevent, where the extension is clean but the archive it absorbed is not.

With setuptools and a hand-written Extension, the flags go into extra_compile_args and extra_link_args. This is the most fragile of the three because there is no platform abstraction: a hard-coded --version-script breaks the macOS build and a hard-coded -exported_symbols_list breaks Linux, so the arguments have to be assembled conditionally in setup.py from sys.platform. Packages migrating away from this arrangement are covered in the modern-tooling reference under integrating CMake with scikit-build-core.

Whichever system you use, wire the export audit into the build as a test rather than trusting the configuration. A single check that the finished object exports one symbol catches every way this can silently regress: a CMake upgrade that changes property propagation, a vendored dependency that appends its own linker flags, a refactor that moves the extension to a different target. It costs one line in CI and it is the only assertion in this area that does not go stale.

Two smaller pitfalls round out the picture. First, link-time optimisation interacts with visibility: with LTO enabled, symbols the compiler believes are unreferenced may be dropped entirely, and a symbol you marked visible but never call from within the module can vanish unless it is also referenced by the version script — which is one more reason to keep the script authoritative. Second, on Linux the version script applies to the final link only; passing it while producing an intermediate relocatable object silently does nothing, and a build that produces an object file and links it in a later step will appear to honour the script while exporting everything.

Frequently Asked Questions

Does hidden visibility break debugging or profiling?

Not meaningfully. Hidden visibility affects the dynamic symbol table, which is what the loader consults; the ordinary symbol table used by debuggers and profilers is unaffected unless you also strip the binary. A stack trace through a hidden-visibility extension is as readable as before, provided debug information is retained or shipped separately.

Should the version script list anything besides PyInit_?

Rarely. The exceptions are extensions that deliberately expose a C API to other extensions — the way NumPy exposes its array API — and packages that must export an allocator or an error handler so a sibling extension can install one. Both are deliberate design decisions with documented consumers; anything exported by accident should be hidden.

What is the macOS equivalent, and does it behave the same way?

-exported_symbols_list with a file containing the single underscore-prefixed init symbol, and behaviour is close but not identical. macOS uses two-level namespaces, so a dependency records which library each symbol came from, which prevents much of the cross-binding described above by default. Hiding is still worth doing — it reduces the export table, avoids surprises with flat-namespace builds, and keeps the build honest across platforms.

How do I check whether another package is exporting its GEOS globally?

Run nm -D --defined-only over the extension modules inside the installed package and look for GEOS, proj_ or GDAL prefixes with DEFAULT visibility. If they are there, you cannot fix it from your side, but you can protect yourself: keep your own copy hidden and statically linked so nothing of yours ever binds to theirs, and note the finding in an issue upstream — most maintainers accept a version-script patch readily once the failure mode is demonstrated.

Is this worth doing for a package with no known conflicts?

Yes, because the conflict is created by someone else’s installation choices rather than by yours. A package that exports only its init symbol cannot participate in a collision no matter what a user installs alongside it, and the cost is one linker flag and a four-line map file that never needs revisiting.

Further Reading

  • GCC visibility documentation (gcc.gnu.org/wiki/Visibility).