Resolving GDAL conflicts between conda and PyPI

This page answers one question: your environment has a conda libgdal and a pip-installed wheel that vendored its own, so imports behave differently depending on order and coordinates come out wrong — how do you diagnose which copy is winning and get back to one coherent stack? It sits inside the Dependency Resolution and Lockfiles section of the Modern Python Build Tooling & Wheel Configuration reference, and gives you the diagnosis, the two clean resolutions and the rule that prevents a recurrence.

A mixed environment with two GDALs and no agreement about which is used The environment contains a conda-installed libgdal in the prefix's lib directory, used by conda-installed packages, and a pip-installed wheel that bundled its own libgdal inside site-packages. Both are loaded into one process. Which one serves a given call depends on import order and on the run paths recorded in each object, so behaviour varies between runs and between machines. one environment, two native stacks conda packages rasterio, fiona from conda-forge pip-installed wheel bundled its own libgdal $PREFIX/lib/libgdal.so.34 GDAL 3.8.4 from conda-forge site-packages/pkg.libs/libgdal-*.so.34 GDAL 3.9.1, vendored one process, both loaded which serves a call depends on order and on recorded run paths

Context & Root Cause

conda-forge and PyPI solve the native-dependency problem in opposite ways. conda-forge makes libgdal a package in its own right and constrains every binding to the build that matches it, so agreement is achieved by the solver. PyPI wheels vendor a private copy each, so agreement is achieved by isolation. Both models work; the failure is at their intersection.

Installing a vendored wheel into a conda environment loads two GDALs into one process. Nothing errors at install time, because neither installer knows what the other did. At run time the outcome depends on which library the loader binds each symbol to, which depends on import order and on the run paths recorded in each object — so the same code produces different results on two machines with the same package list, and a coordinate transformation can silently use a different PROJ than the one you believe you installed.

Solution / Fix

This targets conda-forge GDAL 3.8.x, pip 24+, and an environment where the mixing has already happened.

1. Find out what is actually installed 

conda list | grep -Ei 'gdal|proj|geos'
pip list --format=freeze | grep -Ei 'gdal|rasterio|fiona|pyproj'
python -c "import pyproj, rasterio; print(pyproj.__file__); print(rasterio.__file__)"

2. Find out which native library is loaded 

LD_DEBUG=libs python -c "import rasterio" 2>&1 | grep -m2 'libgdal.*init'
# a path under $CONDA_PREFIX/lib  → the conda copy
# a path under site-packages      → a vendored copy

# Or, after the fact, from inside the process
python -c "
from osgeo import gdal
print(gdal.__file__, gdal.VersionInfo('RELEASE_NAME'))"

3a. Resolve by going all-conda 

pip uninstall -y rasterio fiona pyproj      # remove the vendored copies
conda install -c conda-forge rasterio fiona pyproj

3b. Or resolve by going all-PyPI 

conda remove --force libgdal proj geos      # drop the shared native layer
pip install --force-reinstall rasterio fiona pyproj

4. Prevent the recurrence 

# In a conda environment, restrict pip to packages with no native stack
pip install --only-binary :all: --no-deps pure-python-thing

Verification 

# 1. Exactly one libgdal is loaded in the process
python - <<'PY'
import rasterio, pyproj, ctypes.util, subprocess, os
pid = os.getpid()
maps = open(f"/proc/{pid}/maps").read()
libs = {l.split()[-1] for l in maps.splitlines() if "libgdal" in l}
print(libs)
assert len(libs) == 1, f"{len(libs)} libgdal copies loaded"
PY

# 2. Every spatial package reports the same GDAL and PROJ versions
python -c "
from osgeo import gdal
import rasterio, pyproj
print('gdal', gdal.VersionInfo('RELEASE_NAME'))
print('rasterio sees', rasterio.__gdal_version__)
print('proj', pyproj.proj_version_str)"
# expected: consistent versions, not two different GDALs

# 3. A known transform gives the expected answer
python -c "
import pyproj
t = pyproj.Transformer.from_crs(4277, 4258, always_xy=True)
print(t.transform(-1.5, 53.8))"
# expected: matches the value from a clean single-stack environment

The first check reads the process’s own memory map, which is the only unambiguous answer to “how many GDALs are loaded”. Two entries is a mixed environment regardless of what the package lists say.

Choosing Which Way to Resolve

Both resolutions are correct; which one fits depends on what else the environment has to contain.

All-conda versus all-PyPI, compared across five properties An all-conda environment gives one shared native stack, a solver that enforces agreement, easy access to compilers and system libraries, and security patches from the channel; it requires conda to be available everywhere the environment is reproduced. An all-PyPI environment gives isolated vendored copies, works with plain pip anywhere, and makes each package responsible for its own patches; it duplicates native libraries and cannot mix in packages that need a shared prefix. all conda-forge all PyPI wheels native stack one shared copy one private copy per package who enforces agreement the solver nobody — isolation instead works with plain pip no — conda required yes, anywhere security patches from the channel from each package maintainer disk footprint one stack duplicated per package choose by deployment: conda where the environment is built by conda, wheels where it is built by pip

The deciding question is usually how the environment will be reproduced elsewhere. A team whose production images are built with pip install -r requirements.txt should be all-wheels, because a conda-based development environment then diverges from what actually ships. A team whose pipeline is a conda environment specification should be all-conda, because vendored wheels inside it are exactly the mixing this page is about.

Mixing is defensible in one narrow case: pure-Python packages installed with pip into a conda environment. They add no native code, so there is nothing to collide, and this is the ordinary way to get a package that conda-forge does not carry.

Making the Rule Enforceable

A rule that lives in a README is a rule that gets broken during an incident. Two mechanical checks make it stick.

Two checks that catch a mixed environment before it produces wrong results The first check counts distinct libgdal and libproj files mapped into the process and fails if more than one of each appears; it runs as a test and catches mixing at the moment it happens. The second check compares the versions reported by every spatial package in the environment and fails on disagreement; it catches the subtler case where two copies exist but only one is currently loaded. count loaded copies read /proc/self/maps, count distinct libgdal paths fails the moment two stacks are present, whichever one is currently serving calls cheap enough to run as an ordinary test in every environment you build compare reported versions gdal, rasterio, fiona and pyproj must agree catches the case where two copies exist and only one has been loaded yet and doubles as the diagnostic to paste into a bug report run both in the environment your users will actually have — a conda test environment proves nothing about a pip one and run them in the container image, where the mixing usually originates

The second check has a secondary use worth mentioning: its output is exactly the information a maintainer needs when someone reports odd coordinate results. Shipping it as a small function in your package — the same provenance() idea described in reproducible builds and supply-chain attestation — turns a long diagnostic conversation into a pasted dictionary.

Pitfalls & Alternatives

Using conda remove --force casually. It removes a package without touching its dependents, which is exactly what you want when dismantling the native layer deliberately and a good way to break an environment otherwise. Do it in a throwaway environment first.

Assuming pip check catches this. It validates declared Python metadata and knows nothing about native libraries, so a mixed environment passes cleanly. The process map is the check that sees it.

Reinstalling on top rather than removing first. Installing the conda package while the vendored wheel is still present leaves both, which is the state you are trying to leave. Remove, then install.

Believing the problem is confined to GDAL. PROJ is the more insidious case, because two PROJ copies with different databases produce different coordinates rather than crashes. The version-comparison check covers it; a crash-based test does not.

Frequently Asked Questions

Is it ever safe to pip-install a spatial wheel into a conda environment?

When nothing else in the environment provides the same native library — a fresh conda environment with only Python, into which every spatial package comes from PyPI. That is really an all-wheels environment that happens to have been created by conda, and it is fine.

Why does it work on my machine and not in the container?

Because the loading order or the run paths differ, and both are environment-specific. That variability is the symptom rather than an accident: an environment with two copies has no defined behaviour, so two machines can legitimately disagree.

Does using a lock file prevent this?

It prevents it within the model the lock covers. A conda lock pins the conda side and says nothing about a later pip install; a pip lock pins the wheels and says nothing about a conda prefix. The prevention is the discipline of one installer owning the native layer, with the lock recording the result.

How do I tell which copy produced a wrong coordinate?

Compare pyproj.proj_version_str and pyproj.datadir.get_data_dir() against what you intended to install. A data directory outside the package that provided the library is the strongest single indicator that the two halves of the stack are not the pair you think.

Can I keep conda for the build and wheels for runtime?

Yes, and it is a common and sound arrangement: conda-forge supplies compilers and native headers for the build, the repair step bundles what the wheel needs, and the runtime environment has no conda at all. The rule is about a single environment, not about the whole toolchain.

What about micromamba or pixi environments?

The same rule applies unchanged — they are conda-model environments with a different front end, so a vendored wheel inside one creates the same duplication. The isolation guidance in environment isolation with pixi and conda covers how to keep build, test and development environments separate under those tools.

Does the same problem occur with two conda channels?

It can, in a milder form: two channels publishing the same library with different build strings can produce an environment whose packages disagree about which build they were compiled against. Strict channel priority with conda-forge first is the standard remedy, and unlike the conda-plus-pip case the solver can at least see the conflict.

How do I audit an environment I did not build?

The process map is the fastest route — it reports what is actually loaded rather than what was declared. Following it with the version comparison across every spatial package gives a complete picture in under a minute, and both are worth keeping as a small script.

Can a container image contain a mixed environment?

Easily, and it is one of the most common places to find one: a base image with conda-installed geospatial packages, then a pip install -r requirements.txt layer that pulls vendored wheels. Running the two checks inside the image build is what catches it before it ships.