Handling wheel size limits on PyPI for GDAL

This page answers one question: your self-contained GDAL wheel is rejected with HTTPError: 400 File too large because a vendored GDAL, PROJ, GEOS, and their data exceed PyPI’s per-file limit, so how do you shrink the wheel below the cap — or get the cap raised — without breaking portability? It sits inside the Publishing and Distributing Spatial Wheels section of the Modern Python Build Tooling & Wheel Configuration reference, and gives you the strip-and-exclude techniques, the data-slimming options, and the limit-increase request.

What fills a vendored GDAL wheel and where the size can be cut A vendored GDAL wheel's size breaks down into debug symbols in the shared libraries, the PROJ datum grid files, unused GDAL drivers, and the core code. Stripping debug symbols and excluding grid files and unused drivers cuts the largest slices, bringing the wheel under the per-file limit; if it still exceeds, a limit increase is requested. where a vendored wheel's megabytes go debug symbols strip → big cut datum grids (.tif) exclude → GBs saved unused drivers exclude → medium cut core code keep — required strip + exclude first; request a limit increase only if still over

Context & Root Cause

PyPI enforces a per-file size limit — 100 MB by default — and a self-contained geospatial wheel is one of the few Python artifacts that routinely hits it. The reason is vendoring: to be portable the wheel must bundle GDAL, and GDAL drags in PROJ, GEOS, libtiff, libcurl, libsqlite3, and often libjpeg/libpng/libwebp, plus data. Two things inflate the total far beyond the code itself. First, unstripped shared libraries carry debug symbol tables that can double a library’s size. Second, PROJ’s optional datum-shift grids and GDAL’s full driver set add hundreds of megabytes of files most users never touch. The size explosion is the direct cost of the vendoring decision analysed in why vendoring PROJ causes wheel bloat.

The fix order is: cut what is dead weight (debug symbols), make heavy data opt-in (grids, drivers), and only then ask PyPI to raise the limit. Requesting an increase first is the common mistake — a wheel that ships gigabytes of grids should be slimmed, not enlarged by fiat.

Solution / Fix

This targets auditwheel 6.x, strip from binutils, GDAL 3.8.x, and PROJ 9.3.x.

1. Strip debug symbols from bundled libraries

auditwheel repair can strip while it bundles; otherwise strip the .libs/ payload after repair:

auditwheel repair --strip -w dist/ wheelhouse/*.whl     # strip during repair
# or, post-hoc:
unzip -o dist/*.whl -d /tmp/w >/dev/null
strip --strip-unneeded /tmp/w/*.libs/*.so*

Stripping is safe for release wheels and is usually the single largest cut.

2. Exclude datum grids; rely on the database plus network

Ship proj.db (a few MB) but not the multi-gigabyte .tif grid set, letting PROJ fetch grids on demand:

# at import, enable on-demand grid fetching instead of bundling them
import os
os.environ.setdefault("PROJ_NETWORK", "ON")

Bundle grids only for offline/air-gapped users, as a separate extra. The packaging split is covered in bundling proj.db and datum grids in a wheel.

3. Trim GDAL’s driver set

Build GDAL with only the drivers you support rather than the full set:

# configure GDAL with a reduced driver footprint
cmake -DGDAL_ENABLE_DRIVER_ALL=OFF \
      -DGDAL_ENABLE_DRIVER_GTIFF=ON -DGDAL_ENABLE_DRIVER_GPKG=ON \
      -DOGR_ENABLE_DRIVER_GEOJSON=ON ...

4. If still over the cap, request an increase

Only after 1–3, open a PyPI limit-increase request for the project, citing the stripped, driver-trimmed size. Increases are granted for legitimately large native packages.

Verification 

# 1. The wheel is under the per-file cap
ls -l dist/*.whl | awk '{print $5, $9}'
find dist -name '*.whl' -size +100M && echo "STILL OVER" || echo "under cap"
# expected: under cap

# 2. Stripping did not break the wheel — clean import still works
docker run --rm -v "$PWD/dist:/d" python:3.12-slim \
  bash -c "pip install /d/*.whl && python -c 'from osgeo import gdal; print(gdal.__version__)'"
# expected: 3.8.x

# 3. A transform still resolves via database + network (no bundled grids)
docker run --rm -v "$PWD/dist:/d" python:3.12-slim bash -c \
  "pip install /d/*.whl && PROJ_NETWORK=ON python -c 'import pyproj; print(pyproj.Transformer.from_crs(4326,3857).transform(52,5))'"
# expected: a coordinate pair

Under the cap, a clean import, and a working transform confirm the wheel shrank without losing function.

Where the Megabytes Come From

Shrinking a wheel productively means knowing which reductions are large and which are noise. In a typical vendored GDAL build the savings available fall into four bands, and only the first two are worth engineering effort.

Size reductions available in a vendored GDAL wheel, largest first Four reduction techniques compared. Stripping debug symbols from the bundled libraries removes about twenty-six megabytes. Disabling unused format drivers removes about eighteen megabytes and also removes their transitive dependencies. Moving the PROJ database and grids to a companion package removes about eleven megabytes per platform wheel. Switching the wheel's compression level removes about two megabytes. The first two together typically halve the wheel. reduction from a 78 MB baseline strip debug symbols −26 MB one command, no functional change drop unused drivers −18 MB also removes their dependencies data to a companion wheel −11 MB per platform adds a dependency edge higher zip compression −2 MB, slower install stripping alone usually takes a wheel under the limit; the rest is worth doing for download size, not for the cap

Stripping is the first move because it is free in every sense: the symbols removed are used by debuggers, not by the loader, and no code path changes. If you want a debuggable build, produce it separately and publish it as a distinct artifact rather than carrying its symbols in every user’s download.

Driver pruning is the second because it compounds. Removing a format reader removes the library it wrapped, and often that library’s own dependencies — dropping a rarely-used raster format can take a codec and a compression library with it. The judgement call is which formats your users need, and the honest answer usually requires asking rather than guessing.

The Cost of Being Large

Wheel size is not only a limit to stay under; it is a recurring cost paid by everyone who installs the package, and the places it shows up are worth naming because they are where the complaints originate.

Where a large wheel imposes a cost after publication Four downstream costs. A container image layer grows by the wheel size on every rebuild that does not hit a cache. A CI pipeline that installs the package on every job pays the download repeatedly. A serverless deployment package may exceed the platform's own size limit. A mirror or proxy stores every platform of every release. Each is annotated with the multiplier that turns one wheel into a recurring cost. container image layers every rebuild that misses the cache re-downloads and re-stores the wheel — multiplied by images and tags CI pipelines a matrix of twenty jobs installing the package pays the download twenty times, per run, per project serverless deployment packages several platforms cap the unzipped bundle well below what a vendored GDAL needs — a hard blocker, not a cost mirrors and proxies an internal index stores every platform of every release you have ever published the per-file limit is the visible constraint; these four are the reasons to go well below it rather than just under it publishing the wheel's size in the release notes lets downstream teams plan rather than discover

The serverless case is worth singling out because it is the one where size is a blocker rather than an annoyance. Teams hitting it typically end up using a container-based deployment instead, or splitting the geospatial work into a separate service — both reasonable outcomes, but both cheaper to plan for than to discover during a deadline.

Frequently Asked Questions

Can I ask PyPI to raise the limit for my project?

Yes, and for a legitimately large scientific package the request is usually granted. Make it before you need it, include the reason — vendored native libraries with their versions and sizes — and describe what you have already done to reduce the artifact. A request that arrives with a stripped, driver-pruned wheel and a clear explanation is a very different conversation from one that arrives with an unexamined 120 MB build.

Does stripping break stack traces from native crashes?

It removes the symbol names a debugger would use, so a native backtrace shows addresses rather than function names. For a released wheel that is the right trade — users cannot act on either — provided you can reproduce the crash with an unstripped build. Keeping the unstripped objects as a build artifact, or publishing a separate debug package, preserves that ability without shipping symbols to everyone.

Is a companion data package worth the extra dependency?

It is when the data is large relative to the code and identical across platforms, which is exactly the case for proj.db and the datum grids. One architecture-independent download serves every platform wheel, and updating the data no longer requires rebuilding native code five times. The cost is a dependency edge users cannot skip and a runtime lookup that must find the data package — small, but real.

Why is my wheel bigger than the sum of the libraries it bundles?

Usually because the same code is present twice: a static archive linked into the extension and the shared object copied in by the repair step. Auditing the wheel’s contents with unzip -l and comparing against what the extension actually links catches it, and the fix is to choose one linkage model rather than accidentally shipping both.

Does compression choice matter much for a GDAL wheel?

Barely. Wheels are zip archives and the native libraries are already poorly compressible, so a higher compression level buys a small fraction at the cost of slower installs. Effort is far better spent on stripping and on driver selection, both of which remove content rather than re-encoding it.

Should the wheel size be published anywhere?

Yes — in the release notes, alongside the vendored library versions. Downstream teams building container images or serverless bundles need the number to plan, and providing it costs a line. It also creates gentle pressure to keep the number from drifting upward release by release.

Will stripping affect the platform tag?

No. The tag is computed from the glibc symbol versions the objects reference, and stripping removes debugging information rather than dynamic symbols. A stripped wheel qualifies for exactly the same tag as the unstripped one, which is worth confirming once with the report and then not worrying about again.

Pitfalls & Alternatives

Stripping the extension too aggressively. strip without --strip-unneeded can remove the dynamic symbols the loader needs and break import. Use --strip-unneeded (or let auditwheel --strip handle it), and re-run the smoke test in smoke-testing GDAL wheels in a clean container.

Dropping grids without a fallback. Excluding the datum grids and disabling PROJ_NETWORK leaves high-accuracy transforms failing silently for some datums. Enable network fetching or offer an offline extra.

Requesting a limit increase first. A wheel that is 400 MB because it ships every grid and driver should be slimmed, not blessed. Exhaust strip-and-exclude before asking for more headroom.