Bundling proj.db and datum grids in a wheel

This page answers one question: your wheel bundles libproj correctly, yet at runtime PyProj raises pyproj.exceptions.DataDirError: Valid PROJ data directory not found, so how do you package proj.db and the datum-shift grids as wheel data and point PROJ at them at import — since auditwheel bundles shared objects but never data files? It sits inside the Build Artifact Structuring and Packaging section of the Modern Python Build Tooling & Wheel Configuration reference, and gives you the package_data layout, the runtime path shim, and the size-versus-completeness trade-off.

Why auditwheel bundles libproj but not proj.db, and how the data shim closes the gap The repair tool copies the libproj shared object into the wheel but ignores proj.db and the datum grid files because they are data, not linked libraries. The package therefore ships proj.db as package data and, at import, sets the PROJ data directory to that bundled path so PROJ finds its datum database. auditwheel repair bundles .so only libproj.so → .libs/ ✓ linked library — bundled proj.db → skipped ✗ data file — ignored package_data ship proj/ + set data dir PROJ resolves datum DB found ✓

Context & Root Cause

PROJ splits into two things: libproj, a shared object of transformation code, and a data payload — proj.db, an SQLite database of coordinate reference systems and datum definitions, plus optional .tif grid files for high-accuracy datum shifts. The linked library is what auditwheel and delocate understand; they trace the NEEDED entries of your extension and copy the .so/.dylib into the wheel. They have no notion of data dependencies, so proj.db is silently left behind. At runtime PROJ opens the database by searching a compiled-in path and the PROJ_DATA/PROJ_LIB environment variable — neither of which points inside your wheel — and raises DataDirError.

The fix is orthogonal to library bundling: ship the data as package data and set PROJ’s search directory to the bundled location from Python before any transform runs. This is a packaging concern, which is why it lives under Build Artifact Structuring and Packaging rather than the vendoring guide — the library decision is already made; this is about the files repair tools ignore.

Solution / Fix

This targets PROJ 9.3.x and PyProj 3.6+. The layout ships proj.db inside the package and registers it at import.

1. Stage the data into the package tree 

# Copy PROJ's data payload into the importable package before building
mkdir -p src/mypkg/proj_data
cp "$(pkg-config --variable=pkgdatadir proj)"/proj.db src/mypkg/proj_data/
cp "$(pkg-config --variable=pkgdatadir proj)"/proj.ini src/mypkg/proj_data/ 2>/dev/null || true

2. Declare it as package data 

# pyproject.toml
[tool.setuptools.package-data]
mypkg = ["proj_data/*.db", "proj_data/*.ini", "proj_data/*.tif"]

[tool.setuptools.packages.find]
where = ["src"]

3. Point PROJ at the bundled data at import 

# src/mypkg/__init__.py — runs before any pyproj transform
import os
from pathlib import Path
import pyproj

_data = Path(__file__).with_name("proj_data")
if (_data / "proj.db").exists():
    # pyproj's own resolver — preferred over exporting PROJ_DATA globally
    pyproj.datadir.set_data_dir(str(_data))
    os.environ.setdefault("PROJ_DATA", str(_data))   # covers raw C-API callers

pyproj.datadir.set_data_dir is preferred over exporting PROJ_DATA for the whole process, because a global variable would also redirect any other PROJ-using library in the interpreter.

Verification 

# 1. The data file is actually inside the wheel
unzip -l dist/*.whl | grep proj.db
# expected: mypkg/proj_data/proj.db

# 2. A transform that needs the datum DB succeeds in a clean container
docker run --rm -v "$PWD/dist:/d" python:3.12-slim bash -c "
  pip install /d/*.whl &&
  python -c \"import mypkg, pyproj; t=pyproj.Transformer.from_crs(4326,3857); print(t.transform(52.0,5.0))\""
# expected: a coordinate pair, not DataDirError

# 3. Confirm pyproj resolved the bundled directory, not a system one
python -c "import mypkg, pyproj; print(pyproj.datadir.get_data_dir())"
# expected: .../site-packages/mypkg/proj_data

A proj.db inside the wheel, a successful transform in a bare image, and a data-dir path pointing inside the package confirm the bundling. DataDirError in step 2 means the import shim did not run before the transform.

How Much Data to Ship

proj.db is about nine megabytes and is not optional — without it PROJ cannot resolve an EPSG code at all. The grids are a different question, and it is the question that decides whether your wheel is 30 MB or 300 MB.

Datum-shift grids are correction surfaces used when transforming between reference frames that are not related by a simple parameter set: NAD27 to NAD83 in North America, OSGB36 to ETRS89 in Britain, vertical datum shifts almost everywhere. Without the relevant grid, PROJ does not fail — it falls back to a lower-accuracy transformation path, typically a seven-parameter Helmert, and returns an answer that is wrong by a metre or several. For a web map that is invisible. For a cadastral survey it is a defect.

Four packaging options for PROJ data, by wheel size and transformation accuracy Four options are compared. Shipping proj.db alone adds about nine megabytes and gives full CRS lookup with fallback-accuracy datum shifts. Adding a regional grid set adds about twenty-five megabytes and gives full accuracy in the chosen region. Shipping the complete proj-data package adds more than five hundred megabytes and exceeds the PyPI file size limit. Enabling PROJ network access adds nothing to the wheel but requires internet access at transform time and caches grids on the user's machine. option wheel size impact accuracy catch proj.db only the recommended default +9 MB every CRS resolves; datum shifts fall back sub-metre error, silently proj.db + regional grids pick the region you serve +25 MB full accuracy inside the chosen region wrong outside it complete proj-data everything upstream ships +500 MB full accuracy everywhere exceeds the PyPI per-file size limit PROJ network on grids fetched on demand no change full accuracy when the network is available fails closed offline

For most packages the second row is the right answer: proj.db plus the grids for the regions your users actually work in, chosen deliberately and documented. The fourth row — PROJ_NETWORK=ON, where PROJ fetches grids from a CDN and caches them — is attractive and has a sharp edge: a transform that silently changes behaviour depending on whether the machine has internet access is very hard to debug, and in a CI environment it makes results non-reproducible. If you enable it, make it opt-in rather than the default, and never enable it in tests.

Whatever you ship, expose it. A one-line helper that reports the data directory and whether a given grid is present converts a class of “my coordinates are slightly wrong” reports into a single command:

def proj_data_report():
    import pyproj
    from pathlib import Path
    d = Path(pyproj.datadir.get_data_dir())
    grids = sorted(p.name for p in d.glob("*.tif"))
    return {"data_dir": str(d), "proj_db": (d / "proj.db").exists(),
            "grid_count": len(grids), "grids": grids[:10],
            "proj_version": pyproj.proj_version_str}

Where PROJ Looks for Its Data

PROJ resolves its data directory through an ordered search, and knowing the order explains both why the error happens and why setting a global environment variable is the wrong remedy.

The order in which PROJ searches for proj.db PROJ consults, in order, any directory set through its own API, then the PROJ_DATA environment variable, then the path compiled into the library at build time, then a small set of well-known system directories. In an unpatched wheel the first three all miss, because the compiled-in path refers to the build machine, and the search fails with DataDirError. Setting the directory through pyproj's API makes the first step succeed and leaves other libraries in the process unaffected. 1 · set_data_dir() / proj_context_set_search_paths 2 · PROJ_DATA environment variable 3 · path compiled in at build time 4 · /usr/share/proj and friends process-scoped and precise — the wheel's own directory, set before the first transform process-wide: redirects every PROJ-using library in the interpreter, not just yours points at the build machine's prefix — the directory that does not exist for users present only if the user installed PROJ separately — and then it may be a different version step 3 is why the wheel fails on every machine but yours; step 1 is why the fix belongs in the package rather than the environment

Note the difference between steps one and two carefully: setting PROJ_DATA fixes your package and simultaneously redirects any other PROJ-linked library in the same interpreter — including one that bundled a different PROJ version and needs its own database.

Keeping the Database and the Library in Step

proj.db is versioned against the PROJ library that reads it, and a mismatch is one of the more confusing failures in this area because it can be partial. PROJ checks the database schema version at open time and will refuse a database that is too new; a database that is too old may open and then lack coordinate operations the library expects, producing CRSError for particular EPSG codes while everything else works.

The rule that avoids it is to take proj.db from the same build as the libproj you bundle, in the same step, rather than from a package installed separately. In practice that means copying from $(pkg-config --variable=pkgdatadir proj) during the build — as the earlier step does — and never from a system PROJ or a downloaded release archive that happens to be to hand.

Two operational habits reinforce it. Record the PROJ version and the database’s own schema version alongside the wheel, so a mismatch shows up in a diff rather than in a user’s traceback. And add a functional assertion to the validation gate that exercises a grid-based transform, not just a Helmert one — the failure mode of a stale database is that ordinary transforms keep working while specific datum pipelines stop resolving.

# Fails if the bundled database cannot serve a grid-based transformation
python - <<'PY'
import pyproj
t = pyproj.Transformer.from_crs("EPSG:4277", "EPSG:4258", always_xy=True)  # OSGB36 → ETRS89
x, y = t.transform(-1.5, 53.8)
assert abs(x + 1.4988) < 0.01 and abs(y - 53.8004) < 0.01, (x, y)
print("grid-based transform ok:", pyproj.proj_version_str)
PY

Frequently Asked Questions

Why does the error appear at import rather than at first use?

Because the binding initialises a PROJ context eagerly, and building that context opens the database. That is convenient for diagnosis — the failure arrives immediately rather than in the middle of a pipeline — but it means the traceback points at the import line and looks like a linking problem when it is a data problem.

Can the data live outside the package directory?

It can, and it makes the package fragile. Anything outside the installed package is not guaranteed to be present after installation, is not covered by the wheel’s record of files, and will not be removed on uninstall. Keeping the data inside the package is what makes the path computable from the module’s own location.

Pitfalls & Alternatives

Relying on PROJ_LIB from the environment. Setting PROJ_LIB in CI makes local tests pass while the shipped wheel still fails for users who never set it. Bundle the data and register it in code.

Bundling the full grid set unconditionally. The optional datum-shift .tif grids run to gigabytes. Ship proj.db (a few MB) always, but treat the high-accuracy grids as opt-in — most users are served by the database alone, and shipping them all bloats the wheel the way a vendored PROJ does in why vendoring PROJ causes wheel bloat. For grids beyond the database, enabling PROJ’s network fetch (PROJ_NETWORK=ON) is the leaner alternative.

Forgetting GDAL’s own data. GDAL has a parallel GDAL_DATA directory (gcs.csv, projection tables). A GDAL-plus-PROJ wheel must bundle and register both, following the identical pattern.