Shrinking GDAL wheels with LTO and strip

This page answers one question: your vendored GDAL wheel is eighty megabytes and you want it under forty without dropping a driver anybody uses — so which of stripping, section garbage collection and link-time optimisation actually delivers, and in what order? It sits inside the Binary Size and Startup Performance Tuning section of the Modern Python Build Tooling & Wheel Configuration reference, and gives you the flags, the measurements and the one that usually disappoints.

Size removed by each technique on a typical vendored GDAL wheel Starting from seventy-eight megabytes, stripping debug symbols removes about twenty-six. Section garbage collection with function and data sections removes about six more. Hidden visibility removes about two. Link-time optimisation removes about three, and occasionally adds size instead. The residual after all four is about forty-one megabytes, which is code and data that has to be there. megabytes removed, from a 78 MB baseline strip debug symbols −26 MB --gc-sections −6 MB hidden visibility −2 MB link-time optimisation −3 MB, sometimes negative what remains 41 MB of code and data that has to be there the first bar is most of the answer; the last is the one people try first and the one that disappoints

Context & Root Cause

A release build of GDAL still carries symbol tables and, unless told otherwise, debug information. Neither is used by the loader; both are used by debuggers. For a library the size of GDAL that metadata can be as large as the code, which is why stripping is the single largest reduction available and why it is the first thing to do.

Beyond that the returns fall off quickly. Section garbage collection removes code that nothing references, which in a library designed for selective use is a meaningful but bounded amount. Hidden visibility shrinks the dynamic symbol table, which is a small file-size win and a real load-time one. Link-time optimisation is the technique people reach for first and the one most likely to disappoint: it can shrink code through cross-module analysis and can equally inflate it through aggressive inlining, and on a codebase the size of GDAL it lengthens the build considerably for a result you have to measure to know.

Solution / Fix

This targets GCC 13 or Clang 16, binutils 2.38+, auditwheel 6.x and GDAL 3.8.x built from source.

1. Strip, during the repair step 

auditwheel repair --strip -w dist/repaired wheelhouse/*.whl

If your pipeline repairs without stripping, do it after — never before, because the repair tool reads symbol information to decide what to bundle:

unzip -o dist/*.whl -d /tmp/w >/dev/null
strip --strip-unneeded /tmp/w/*.libs/*.so* /tmp/w/**/*.so

2. Compile so the linker can drop what is unused 

export CFLAGS="-O2 -g0 -ffunction-sections -fdata-sections -fvisibility=hidden"
export CXXFLAGS="$CFLAGS -fvisibility-inlines-hidden"
export LDFLAGS="-Wl,--gc-sections -Wl,--as-needed"

-ffunction-sections and -fdata-sections put each function and object in its own section so --gc-sections can discard the unreferenced ones. Without the compile flags the linker flag has nothing to work with.

3. Try LTO, and measure rather than assume 

export CFLAGS="$CFLAGS -flto=auto -ffat-lto-objects"
export LDFLAGS="$LDFLAGS -flto=auto"
# then compare against the previous build before keeping it

4. Record the numbers so the next change can be compared 

printf '%s\t%s\n' "$(git rev-parse --short HEAD)" "$(stat -c%s dist/repaired/*.whl)" \
  >> size-history.tsv

Verification 

# 1. The size moved, and by how much
du -h dist/repaired/*.whl
unzip -l dist/repaired/*.whl | sort -k1 -n | tail -6
# expected: the largest entries are the libraries you intended to bundle

# 2. Stripping did not break anything
docker run --rm -v "$PWD/dist/repaired:/d" python:3.12-slim bash -c \
  "pip install -q /d/*.whl && python -c '
from osgeo import gdal; import pyproj
print(gdal.__version__, pyproj.proj_version_str)
print(pyproj.Transformer.from_crs(4326,3857,always_xy=True).transform(5.0,52.0))'"

# 3. The driver set is intact
python - <<'PY'
from osgeo import gdal
gdal.UseExceptions()
need = {"GTiff", "GPKG", "GeoJSON", "ESRI Shapefile"}
have = {gdal.GetDriver(i).ShortName for i in range(gdal.GetDriverCount())}
assert need <= have, need - have
print(f"{len(have)} drivers present")
PY

The third check exists because --gc-sections can, in principle, discard code reached only through a mechanism the linker cannot see. GDAL’s drivers are registered through explicit calls rather than pure dlopen, so they survive — but asserting it takes one second and removes the doubt.

Why LTO Disappoints Here

Link-time optimisation is a good technique that fits this workload badly, and it is worth understanding why before spending a day on it.

What link-time optimisation gives and costs on a GDAL-sized build On the benefit side, cross-module inlining can remove some duplicated code and unreferenced functions the linker could not otherwise see. On the cost side, build time increases substantially, memory use during the link can exceed a runner's limit, debugging becomes harder, and aggressive inlining can increase code size rather than reduce it. The conclusion is to measure on your own stack and to treat a neutral result as a reason not to keep it. what it can give cross-module inlining removal of functions the linker could not otherwise prove unused occasionally faster code on this stack, single-digit megabytes and sometimes zero what it costs a substantially longer build high memory during the link, which can exceed a runner's limit harder debugging and profiling inlining that can grow the binary measure; a neutral result means drop it

The memory point is the practical blocker on hosted runners. Linking GDAL with LTO can require several gigabytes for the link step alone, and a runner that runs out produces a failure that looks like a compiler crash. Setting -flto=auto rather than a fixed parallelism helps, and it does not change the underlying appetite.

There is one situation where LTO earns its cost here: a build that statically links GEOS or another mid-sized library into the extension. Cross-module analysis across that boundary can remove a genuinely useful amount, because the linker can see that most of the library is unreachable from your handful of entry points. Even then, measure.

Stripping Without Losing the Ability to Debug

Stripping is unambiguously correct for a released wheel and it does remove your ability to interpret a crash report from a user. Keeping both is straightforward.

Separating debug information from the shipped wheel The build produces objects with full debug information. A separate step extracts that information into companion debug files, adds a link from the stripped object to them, and strips the objects. The wheel ships the stripped objects; the debug files are kept as a build artifact. A crash report from a user can then be symbolised against the matching debug files without every user having downloaded them. built objects with debug info extract and link objcopy --only-keep-debug stripped .so ships in the wheel .debug files kept as build artifacts symbolise a user's crash

The extraction is three commands, and keeping the debug files as a build artifact costs storage nobody downloads. What it buys is the ability to answer “the extension crashed, here is the stack” from a user who has only the stripped wheel — which is otherwise impossible once you have stripped.

objcopy --only-keep-debug lib.so lib.so.debug
objcopy --strip-unneeded lib.so
objcopy --add-gnu-debuglink=lib.so.debug lib.so

Pitfalls & Alternatives

Stripping before the repair step. The repair tool inspects symbols to decide what to bundle and how to rewrite paths; a stripped input can leave it without what it needs. Repair, then strip — or let the repair tool do both.

Using --strip-all on a shared library. It removes the dynamic symbol table the loader needs. --strip-unneeded is the correct option for a .so, and is what the repair tool uses.

Enabling LTO without measuring. It is the most-recommended and least-reliable of these techniques for this workload. Measure the wheel size and the build time before and after, and keep it only if the first improved enough to justify the second.

Assuming --gc-sections is free. It is close to free for size and requires the compile-side flags to do anything. Without -ffunction-sections the linker has whole object files rather than individual functions to reason about, and removes almost nothing.

Frequently Asked Questions

Does stripping affect the platform tag?

No. The tag is computed from glibc symbol versions referenced by the objects, and stripping removes debugging information rather than dynamic symbols. Confirming it once with the report is worthwhile; after that it can be ignored.

Can I strip the bundled libraries but not my extension?

Yes, and it is a reasonable middle ground while you are still debugging your own code — your extension is a small fraction of the size, so keeping its symbols costs little. The large libraries are where the megabytes are.

Is -Os worth trying instead of -O2?

For GDAL, rarely. It trades measurable run-time performance for a modest size reduction, and raster processing is exactly the workload where that trade is unattractive. It is more defensible for a small extension where the code is not hot.

What about compressing the wheel harder?

A few per cent at best. Native libraries compress poorly, and the higher level costs install time. It is the last thing to try and the first thing people ask about.

Do these flags affect reproducibility?

Positively. -g0 and stripping remove the largest source of path-dependent bytes, which is why the reproducibility work and the size work overlap so much — see making spatial wheels byte-reproducible.

How do I know when to stop?

When the remaining bytes are libraries you have decided to bundle at versions you have chosen. At that point further reduction means removing capability — a driver, a platform, a data set — which is a product decision rather than a build setting.

Does stripping affect the ability to profile the shipped wheel?

Profilers that sample addresses still work; what they lose is the ability to name functions without the debug files. Keeping the separated debug information as a build artifact restores that whenever it is needed, which is why the separation is worth the three extra commands.

Should the extension and the bundled libraries get the same treatment?

The same flags, yes, and the same stripping. The libraries are where the size is, so applying the flags only to your own sources produces a small fraction of the available reduction — which is the most common reason a size-tuning attempt disappoints.

Is there a size cost to hidden visibility?

It is a reduction rather than a cost: fewer dynamic symbols means a smaller symbol table and fewer relocations. The reason it appears low on the list is that GDAL’s own symbol table is already modest relative to its code, not that the technique is ineffective.