Binary Size and Startup Performance Tuning for Spatial Wheels

A self-contained GDAL wheel is one of the largest artifacts on any Python index, and its size is paid twice: once per download and once per import, because every megabyte of shared object has to be mapped and relocated before the first coordinate is transformed. This guide sits under the Modern Python Build Tooling & Wheel Configuration reference and covers the compile, link and packaging settings that shrink a spatial wheel and cut its import time, with the measurements to tell which changes actually helped. It targets GDAL 3.8.x, PROJ 9.3.x, GCC 13 / Clang 16, auditwheel 6.x, and the vendoring model described in vendoring PROJ and GDAL vs system libraries.

Where wheel size and import time each come from Two related but distinct budgets. Wheel size is dominated by debug symbols, the driver set, the datum data and the code itself. Import time is dominated by mapping and relocating the shared objects, running static initialisers, registering drivers and opening the PROJ database. Several levers affect both, but stripping mostly affects size while lazy driver registration mostly affects time. wheel size — paid on download debug symbols in bundled libraries the compiled driver set proj.db and any datum grids the extension and its own code levers: strip, prune drivers, split data measured with: ls -l and unzip -l felt by: container builds, CI, mirrors import time — paid on every run mapping and relocating the objects static initialisers in C++ libraries driver registration at import opening proj.db on first use levers: fewer symbols, lazy registration measured with: python -X importtime felt by: CLI tools, serverless, tests

Prerequisites & Environment

  • A repaired wheel to measure — the unrepaired one has no bundled libraries and tells you nothing about either budget.
  • binutils (strip, size, readelf) and, for import profiling, a CPython 3.7+ interpreter supporting -X importtime.
  • Control over how GDAL is configured, since the driver set is decided at its compile time rather than in your package.
  • A baseline: record the wheel size and the import time before changing anything, or you will not be able to tell which change helped.
# Baseline both budgets before touching the build
ls -l dist/*.whl
python -X importtime -c "from osgeo import gdal" 2>&1 | tail -3

Core Configuration

The compile and link settings that matter for a spatial wheel are few, and their effects differ sharply between the two budgets.

# Size- and load-oriented build flags for the native stack
export CFLAGS="-O2 -g0 -fvisibility=hidden -ffunction-sections -fdata-sections"
export CXXFLAGS="$CFLAGS -fvisibility-inlines-hidden"
export LDFLAGS="-Wl,--gc-sections -Wl,-O1 -Wl,--as-needed -Wl,-z,relro,-z,now"
Setting Effect on size Effect on import time
-g0 / strip --strip-unneeded large — often halves a library none directly
-fvisibility=hidden moderate — smaller dynamic symbol table small improvement: fewer relocations
--gc-sections with -ffunction-sections moderate — drops unreferenced code small improvement
--as-needed small moderate: fewer libraries opened
Link-time optimisation moderate, sometimes negative small
Fewer GDAL drivers large — code and dependencies large: less to register
Data in a companion package large per platform wheel none

Two entries deserve emphasis. --as-needed drops DT_NEEDED entries for libraries that were on the link line but whose symbols are never referenced — a common outcome when a configure script over-links — and each dropped entry is one fewer file the loader opens at import. And hidden visibility, which the symbol visibility and namespace isolation chapter recommends for correctness, is also a performance change: a dynamic symbol table with one entry instead of six hundred is faster to process.

Step-by-Step Implementation

  1. Measure first, and record the numbers where a future change can be compared against them:

    printf 'wheel %s\n' "$(du -h dist/*.whl | cut -f1)" > perf-baseline.txt
python -X importtime -c "from osgeo import gdal" 2>&1 | tail -1 >> perf-baseline.txt

  2. Strip the bundled libraries, either during repair or immediately after:

    auditwheel repair --strip -w dist/repaired wheelhouse/*.whl

  3. Narrow the driver set at GDAL’s configure step, enabling only the formats you support:

    cmake -DGDAL_ENABLE_DRIVER_ALL=OFF \
      -DGDAL_ENABLE_DRIVER_GTIFF=ON -DGDAL_ENABLE_DRIVER_GPKG=ON \
      -DOGR_ENABLE_DRIVER_GEOJSON=ON -DOGR_ENABLE_DRIVER_SHAPE=ON ..

  4. Defer the expensive work in your own package’s import, moving anything that touches the database or the driver registry behind a function call rather than running it at module scope.

  5. Re-measure and keep only what helped — several of these levers interact, and a change that helps in isolation can be neutral once another has already removed the same bytes.

Reading an Import-Time Profile

python -X importtime prints a cumulative and self time per module, and for a spatial package the shape of the output is highly diagnostic. Nearly all the time appears against the first module that triggers the native load, and the interesting question is what that load is doing.

Where the milliseconds go in a cold import of a vendored GDAL wheel A breakdown of about three hundred and forty milliseconds of cold import time. Mapping and relocating the bundled shared objects takes about one hundred and forty milliseconds. C++ static initialisers take about seventy. Driver registration takes about eighty. Opening the PROJ database takes about thirty. The Python-level module code takes about twenty. Warm imports, where the files are in the page cache, are roughly a third of the total. cold import, milliseconds map + relocate .so files 140 ms driver registration 80 ms C++ static initialisers 70 ms open proj.db 30 ms Python module code 20 ms stripping and pruning shrink the top bar; lazy registration removes the second; neither touches the last

The top bar is the one that responds to size work: fewer and smaller objects mean less to map and fewer relocations to process. The second responds to configuration — GDAL registers every compiled driver at registration time, so a build with forty drivers instead of two hundred spends proportionally less time there. The bottom bar is already small and is not worth optimising.

The practical target for most packages is to keep the package’s own import cheap and let the expensive native work happen on first use. That is a design decision in your __init__.py: importing the extension module is unavoidable, but registering drivers, opening the database and building a default transformer are not.

# mypkg/__init__.py — cheap import, expensive work deferred
from . import _geospatial_ext          # required: loads the native libraries

_registered = False

def _ensure_registered():
    global _registered
    if not _registered:
        from osgeo import gdal
        gdal.AllRegister()
        _registered = True

def open_raster(path):
    _ensure_registered()               # first call pays; later calls do not
    from osgeo import gdal
    return gdal.Open(path)

Choosing What to Cut

Not every reduction is worth its cost, and the ones that remove capability need a decision rather than a flag. Ordering them by benefit against risk keeps the work honest.

Size reductions ordered by benefit against the capability they cost Five reductions plotted by how much they save against how much capability they remove. Stripping debug symbols saves a great deal and removes nothing users can see. Garbage-collecting unused sections saves moderately and removes nothing. Splitting the datum data into a companion package saves a great deal per platform but adds a dependency. Pruning drivers saves a great deal but removes formats some users need. Dropping a platform saves the most and removes a whole audience. bytes saved capability removed → strip free — do it first gc-sections also free split data adds a dependency prune drivers ask users first drop a platform last resort, announce it work left to right — the two leftmost cost nothing, and together they usually take a vendored wheel under the index limit

Driver pruning deserves a note about method. The list of drivers your users need is discoverable rather than guessable: read the issue tracker, look at what your own test suite opens, and — if the package is established — ask. Removing a format that three users depend on to save four megabytes for everyone else is a poor trade, and removing forty formats nobody has ever opened is an excellent one.

Verification 

# 1. Size moved in the direction you intended
du -h dist/repaired/*.whl
unzip -l dist/repaired/*.whl | sort -k1 -n | tail -8
# expected: the largest entries are the libraries you chose to bundle

# 2. Import time improved, measured cold
sync; sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches' 2>/dev/null || true
python -X importtime -c "import mypkg" 2>&1 | tail -3
# expected: cumulative time below your baseline

# 3. Nothing was lost — the drivers you kept still work
python - <<'PY'
from osgeo import gdal
gdal.UseExceptions()
need = {"GTiff", "GPKG", "GeoJSON", "ESRI Shapefile"}
have = {gdal.GetDriver(i).ShortName for i in range(gdal.GetDriverCount())}
missing = need - have
assert not missing, f"pruned too far: {missing}"
print(f"{len(have)} drivers, all required formats present")
PY

Optimization & Edge Cases

  • Strip after repair, not before. The repair tool reads symbol information to decide what to bundle and how to rewrite paths; stripping first can leave it without the information it needs.
  • Link-time optimisation is not a reliable win here. It can shrink code and can also inflate it through aggressive inlining, and it lengthens builds considerably. Measure on your own stack rather than assuming.
  • Compression choices are noise. The native libraries dominate and compress poorly; a higher zip level buys a few per cent at the cost of slower installs.
  • Import time on network filesystems is a different problem. In a container with an overlay or a network-mounted site-packages, mapping dominates far more than locally, which raises the value of a smaller artifact independently of CPU.
  • Serverless platforms cap the unpacked bundle, not the wheel. A 60 MB wheel can unpack past a 250 MB limit once its dependencies are included, so measure the installed size as well as the download.

Troubleshooting

Stripping produced no size change. The libraries were already stripped upstream, or strip ran against copies that the repair step then replaced. Check the file sizes inside the wheel rather than in the build tree.

Import time got worse after enabling hidden visibility. Unlikely to be causal — re-measure with a warm cache and several runs. Import timing is noisy, and a single cold measurement on a busy machine can vary by tens of milliseconds.

A driver is missing after pruning that the test suite did not catch. The test suite exercises what it knows about; users exercise more. Publish the enabled driver list in the release notes so a missing format is an obvious, checkable fact rather than a mystery.

--gc-sections removed a symbol that was needed at runtime. It removes unreferenced sections, and a symbol only reached through dlopen looks unreferenced. Mark such entry points as used, or exclude the object from section garbage collection.

Frequently Asked Questions

What is a reasonable size for a vendored GDAL wheel?

After stripping and a sensible driver set, roughly 30 to 50 MB per platform is achievable while keeping the common formats. Above 80 MB, something is usually unstripped or bundling data that could ship separately. Below 20 MB, check that the drivers you think you have are actually present.

Does import time matter for a data pipeline?

Less than for a command-line tool, but it is not free: a pipeline that spawns a worker per task pays the import on every task, and a test suite that imports in a fresh interpreter pays it per test file. For a long-running service the cost is paid once and can be ignored.

Should the package import the extension at module scope?

Yes — deferring the extension import itself mostly moves the cost rather than removing it, and it makes failures appear at an arbitrary later point instead of at import, which is much harder to diagnose. Defer the expensive optional work, not the load.

Is a smaller wheel worth losing a driver?

Only if the driver is genuinely unused. Size is a cost paid by everyone; a missing driver is a wall hit by a few. The right sequence is to take the free reductions first and treat capability removal as a last step with evidence behind it.

Does a smaller wheel install faster?

Usually, and for two reasons rather than one: fewer bytes to download and fewer files to write. The second matters more than people expect — installation writes and records every file individually, so a package with thousands of small data files can install more slowly than a larger package with a handful of big libraries. If install time is the complaint, count the files as well as the megabytes.

How much of this applies to a pure-Python spatial package?

The import-time half applies in full: a package that builds transformers, opens databases or registers plugins at module scope pays that cost on every interpreter start regardless of whether it compiled anything. The size half largely does not, since without vendored libraries there are no megabytes to remove.

Measuring Honestly

Both budgets are easy to measure badly, and a bad measurement leads to weeks spent on a change that did nothing. Three practices separate a number you can act on from one you cannot.

Measure the artifact, not the working tree. Wheel size means the size of the file that will be uploaded, after repair and after stripping — not the size of a build directory, and not the installed size unless that is the number you care about. Those three differ by large factors: a build tree containing unstripped intermediates can be several times the wheel, and an installed package is larger than the wheel because the archive is compressed. Decide which number matters for your users — download size for most, unpacked size for serverless and container work — and track that one.

Separate cold from warm for import time. The first import of a package on a machine reads the shared objects from disk; every subsequent import in a fresh process reads them from the page cache. The difference is frequently three-fold, and mixing the two produces a series of measurements that look like random noise. Report both, and be explicit about which one a change is supposed to improve: stripping and pruning reduce the cold number by reducing bytes read, while lazy registration reduces both.

Take more than one sample. Import timing on a shared CI runner varies by tens of milliseconds run to run. Five runs and a median is a two-line change to a measurement script and turns “did that help?” from a guess into an answer.

# A measurement harness worth keeping in the repository
set -eu
whl=$(ls dist/*.whl | head -1)
printf 'wheel_bytes\t%s\n' "$(stat -c%s "$whl")"

python - <<'PY'
import subprocess, statistics, sys
def once():
    out = subprocess.run([sys.executable, "-X", "importtime", "-c", "import mypkg"],
                         capture_output=True, text=True).stderr.strip().splitlines()[-1]
    return int(out.split("|")[1].strip())          # cumulative microseconds
runs = [once() for _ in range(5)]
print(f"import_us_median\t{statistics.median(runs)}")
print(f"import_us_spread\t{max(runs) - min(runs)}")
PY

Record the output as a build artifact on every release. Two numbers per release, kept over a year, tell you far more than any single profiling session: a wheel that grew by 40% over four releases is visible in the series and invisible in any individual build, and the release where it happened is exactly the release to look at.

One more caution about interpreting results. The size levers interact, so applying them one at a time and summing the savings overstates the total — stripping and --gc-sections both remove some of the same bytes, and pruning drivers removes code that stripping would otherwise have shrunk. Measure the combination you intend to ship rather than assembling an estimate from individual measurements.

What Users Actually Experience

It is worth being concrete about who feels each budget, because the answer decides how much effort either deserves.

Container image builds feel wheel size most acutely, and they feel it repeatedly. A Dockerfile that installs the package produces a layer containing the unpacked files, and that layer is rebuilt whenever an earlier line changes. Teams building images in CI several times a day pay the download and the layer storage each time, which is why a 90 MB wheel generates complaints out of proportion to its one-time cost.

Serverless deployments feel it as a hard wall. The limits are on the unpacked bundle rather than the download, and a vendored GDAL plus NumPy plus the application code passes several platforms’ caps. There is no incremental fix at that point; the team either moves to a container-based deployment or splits the geospatial work into a separate service. Knowing the unpacked size in advance turns that from a late discovery into a design decision.

Test suites and CLI tools feel import time. A test suite that spawns a fresh interpreter per module pays the cold import repeatedly, and a command-line tool that a user runs in a loop over files pays it every invocation. For these, a 300 ms import is the difference between a tool that feels instant and one that feels sluggish, and it is entirely within the package’s control.

Long-running services feel neither much. They download once, import once and run for weeks. For a package whose users are overwhelmingly in this category, size work beyond the free reductions is not a good use of maintainer time — and knowing that is as valuable as knowing how to do the work.

The practical consequence is that the free reductions — stripping, section garbage collection, --as-needed, hidden visibility — should simply always be applied, because they cost nothing and help two of the four audiences substantially. Anything beyond them deserves evidence about which audience you actually have.

Further Reading

  • The GDAL build documentation on selective driver configuration.