Why Vendoring PROJ Causes Wheel Bloat: From 5 MB to 200 MB Per Platform Tag

Vendoring PROJ turns a ~5 MB Python extension into an 80–200 MB wheel because the build embeds the entire proj.db geodetic database plus statically copied libproj, libgdal, libsqlite3, libtiff, and libcurl into every platform tag — this page explains exactly where those megabytes come from and how to claw them back without breaking coordinate transforms. It sits under the vendoring PROJ and GDAL vs system libraries decision guide, part of the broader Geospatial C-Extension Fundamentals & ABI Architecture reference.

A typical vendored spatial wheel breaks down roughly as follows — the CRS database and native libraries dominate, while the actual Python extension is a rounding error:

Approximate size breakdown of a vendored PROJ/GDAL wheel A horizontal bar chart of the four contributors to a vendored spatial wheel, sized in megabytes per platform tag. The GDAL and PROJ shared libraries are largest at about 60 MB, the proj.db CRS database is about 50 MB, the transitive dependencies sqlite, tiff and curl together are about 25 MB, and the actual Python extension module is only about 12 MB. The native database and shared libraries dominate; the Python extension is a rounding error. Where the megabytes go in one vendored wheel approximate size per platform tag, in MB 0 20 40 60 GDAL + PROJ shared libs ~60 proj.db CRS database ~50 sqlite + tiff + curl ~25 Python extension module ~12

Context & Root Cause

PROJ is not a small math library; it is a geodetic database runtime. When pyproj, rasterio, or fiona vendor it, the wheel must be self-contained so imports never hit the host with ImportError: libproj.so.25: cannot open shared object file. To guarantee that, auditwheel repair (Linux) or delocate (macOS) copies every dynamically linked .so/.dylib into the wheel’s internal .libs/ directory and rewrites each RPATH to $ORIGIN/.libs — the loader contract detailed in shared library path resolution. Three things then inflate the archive: the proj.db SQLite file (~45–55 MB of EPSG and NGA CRS definitions), the statically bundled native libraries, and PROJ’s default-on CMake features (ENABLE_TIFF, ENABLE_CURL, PROJ_NETWORK) that drag in libtiff, libcurl, and libssl. Because the native tree is frozen at compile time but the CPython ABI is not, you also pay this size once per cp3X tag and per architecture, multiplying the storage bill across the build matrix.

How vendored native libraries flow into one fat wheel per build-matrix cell A left-to-right data flow. Four inputs — PROJ source with its static libproj, the ~50 MB proj.db CRS database, the GDAL shared libraries, and transitive shared objects for tiff, curl and sqlite — all feed into auditwheel repair. That stage copies every shared object into the wheel's internal .libs directory and rewrites each RPATH to $ORIGIN/.libs, on macOS this is delocate. The result is a single fat .whl of 80 to 200 MB. Because the native tree is frozen at compile time while the CPython ABI is not, that whole payload is paid once per cell of the build matrix: a grid of CPython tags cp39 through cp312 crossed with architectures x86_64 and aarch64, so the bloat is multiplied across every cell. One repaired wheel — paid once per build-matrix cell vendored inputs are frozen into the wheel, then the CPython tag × arch matrix multiplies the size PROJ source static libproj proj.db ~50 MB CRS data GDAL libs libgdal · libproj transitive .so tiff · curl · sqlite auditwheel repair copy each .so → wheel/.libs rewrite RPATH → $ORIGIN/.libs delocate does the same on macOS one .whl 80–200 MB × N cp3X × arch x86_64 aarch64 cp39 cp310 cp311 cp312 full size paid in every cell

Solution / Fix

The goal is not to break the self-contained guarantee but to strip everything the runtime does not need. Steps assume PROJ 9.4+, GDAL 3.8+, auditwheel 6.x, and cibuildwheel 2.16+ inside a manylinux_2_28 Docker base image.

1. Disable optional PROJ features at configure time

Most of the transitive weight comes from features few pipelines use. Turn them off in the CMake step so libcurl/libtiff never enter the link graph:

cmake -DCMAKE_BUILD_TYPE=Release \
      -DBUILD_TESTING=OFF \
      -DENABLE_CURL=OFF \
      -DENABLE_TIFF=OFF \
      -DBUILD_PROJSYNC=OFF \
      -DCMAKE_INSTALL_PREFIX=/opt/vendor ..

If you drive the native build through the scikit-build-core backend, pass the same flags via [tool.scikit-build.cmake.define] so the toggles are reproducible rather than baked into a shell script.

Optimize for size and let the linker garbage-collect unreferenced sections. Add the flags to your cibuildwheel environment so they apply to PROJ, GDAL, and the extension uniformly:

[tool.cibuildwheel]
environment = { PROJ_NETWORK="OFF", CFLAGS="-Os -ffunction-sections -fdata-sections", LDFLAGS="-Wl,--gc-sections -s" }
before-build = "rm -rf /opt/_internal/cpython-*/lib/python*/test"

-Os plus --gc-sections removes dead code paths; -s strips the symbol table at link time, which the later strip pass would otherwise have to do.

3. Strip debug tables from every shared object

Vendored libraries ship with DWARF debug info that is useless at runtime and can double a .so. Strip after the build but before auditwheel repair:

find /opt/vendor -name "*.so*" -exec strip --strip-unneeded {} +
objcopy --remove-section=.comment --remove-section=.note /opt/vendor/lib/libproj.so.25

4. Exclude libraries you let the host provide

When you deliberately keep a dependency dynamic, exclude it so auditwheel does not copy it in and abort with a policy error:

auditwheel repair --plat manylinux_2_28_x86_64 \
  --exclude libcurl.so.4 \
  --exclude libtiff.so.6 \
  --exclude libsqlite3.so.0 \
  dist/*.whl

5. Prune legacy transformation grids — never gut proj.db

The database itself is licensed CRS data; deleting rows breaks transforms and can violate redistribution terms. Instead remove rarely used legacy grid files (conus, ntv2_0.gsb) from the bundled share/proj directory while leaving proj.db intact, and keep PROJ_NETWORK=OFF so the runtime never tries to fetch them back.

For a deeper split, publish a thin core wheel (the proj.db and grids omitted, mounted separately as a read-only volume) alongside a full wheel, so most installs pull only the ~12 MB extension. Cache the heavy native build between runs using the strategy in build caching for C extensions so size reduction does not cost rebuild time.

Verification

Confirm the fix on the repaired wheel before publishing. ELF tools read .so files, not .whl zips, so extract first.

# 1. Confirm no external deps leaked — expect "0" non-system libraries
auditwheel show dist/*.whl

# 2. Break down what actually fills the wheel
unzip -l dist/*.whl | sort -k1 -rn | head
# Expect: proj.db <= 55 MB, each .so <= 15 MB

# 3. Confirm the vendored DB still resolves CRS with the host wiped
PROJ_DATA="" PROJ_LIB="" python -c "
from pyproj import CRS
crs = CRS.from_epsg(4326)
assert crs.to_proj4() == '+proj=longlat +datum=WGS84 +no_defs +type=crs'
print('Vendored PROJ runtime validated.')
"

Clearing PROJ_DATA and PROJ_LIB forces the extension to use only the bundled database; if step 3 prints the assertion instead of a CRSError, the strip and prune passes did not damage the geodetic data. Gate the size in CI so a regression fails the pipeline:

test "$(stat -c%s dist/*.whl)" -lt 85000000 || { echo "wheel exceeds 85 MB budget"; exit 1; }

Pitfalls & Alternatives

  • Deleting rows from proj.db to shrink it. Trimming the CRS table with sqlite3 proj.db "DELETE FROM ..." produces CRSError: Invalid projection the moment a user requests a pruned authority code, and it breaks the BSD redistribution terms attached to the EPSG data. Prune optional grid files and disable network sync instead of editing the database.
  • Stripping the .so files so aggressively that imports break. Running strip without --strip-unneeded (or stripping dynamic symbols) yields ImportError: undefined symbol: proj_create because the dynamic symbol table the extension links against is gone. Use --strip-unneeded/--strip-debug only, and verify with the smoke test above.
  • Bumping to the newest manylinux tag to “compress better.” The platform tag has nothing to do with size; targeting manylinux_2_35 only raises the glibc floor and breaks older hosts with GLIBC_2.NN not found. If size on Alpine is the real concern, weigh the static-bundling trade-off in manylinux2014 vs musllinux for spatial libs — musl needs patchelf --set-rpath rather than auditwheel, and mixing it with glibc runners causes silent ABI mismatches.

Further reading: the platform-tag rules governing which libraries may be excluded are defined by the PyPA manylinux specification (PEP 600).