Publishing and Distributing Spatial Wheels

Getting a validated GDAL wheel onto PyPI is its own discipline: the release must be authenticated without long-lived API tokens, the multi-platform artifact set must upload atomically, and a vendored PROJ can push a wheel past PyPI’s per-file size limit. This guide sits under the Modern Python Build Tooling & Wheel Configuration reference and covers the publishing half of the pipeline — trusted publishing with OpenID Connect, staged releases through TestPyPI, and the size constraints that hit geospatial packages hardest. It targets twine 5.x, PyPI trusted publishing (OIDC), GitHub Actions / GitLab CI release jobs, and the validated wheels produced by Testing and Validating Spatial Wheels.

Prerequisites & Environment #

• A validated dist/ from the build matrix — every wheel already past the gates in Testing and Validating Spatial Wheels.
• A PyPI (and TestPyPI) project with a configured trusted publisher bound to your CI workflow — no API token stored anywhere.
• twine 5.x for the size/metadata pre-flight; pypa/gh-action-pypi-publish on GitHub or twine upload under an OIDC token on GitLab.

# Pre-flight the whole set before any upload
python -m twine check dist/*

Core Configuration #

Three concerns define a safe geospatial release:

Trusted publishing replaces a stored PYPI_API_TOKEN with a short-lived OIDC token the CI provider mints per run, detailed in trusted publishing spatial wheels to PyPI. The size constraint is acute here because a self-contained GDAL wheel bundles PROJ, GEOS, libtiff, and their data — the mitigation lives in handling wheel size limits on PyPI for GDAL.

Step-by-Step Implementation #

1. Pre-flight metadata and size:

   python -m twine check dist/*.whl
   # Fail early if any wheel exceeds the per-file limit (100 MB default)
   find dist -name '*.whl' -size +100M -exec echo "OVERSIZE: {}" \;
   

2. Stage to TestPyPI and install from it in a clean container:

   twine upload --repository testpypi dist/*
   docker run --rm python:3.12-slim bash -c \
     "pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ mypkg && python -c 'from osgeo import gdal'"
   

3. Publish to PyPI from a tag-triggered release job authenticated via OIDC (no token):

   # GitHub: the release job
   permissions: { id-token: write }        # enables OIDC
   steps:
     - uses: actions/download-artifact@v4
       with: { name: dist, path: dist }
     - uses: pypa/gh-action-pypi-publish@release/v1   # uses the trusted publisher
   

4. Verify the public install once propagation completes (see Verification).

Verification #

# 1. No wheel exceeds the size cap before upload
find dist -name '*.whl' -size +100M | grep . && echo "BLOCK: oversize wheel" || echo "size ok"
# expected: size ok

# 2. TestPyPI install of the exact release works end to end
docker run --rm python:3.12-slim bash -c \
  "pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ mypkg==1.0 && python -c 'import pyproj; pyproj.Transformer.from_crs(4326,3857)'"
# expected: clean import and transform

# 3. After PyPI release, a plain install resolves every platform
pip download --only-binary=:all: --no-deps mypkg==1.0 -d /tmp/dl && ls /tmp/dl
# expected: wheels for each platform you published

Optimization & Edge Cases #

• Publish on tags only. Gate the release job on if: startsWith(github.ref, 'refs/tags/') (or a GitLab rules: tag condition) so every commit does not attempt an upload.
• Uploads are not atomic across files. If the job dies mid-set, some wheels land and some do not; re-running skips existing files with --skip-existing, so design the job to be safely retryable.
• sdist matters for source builds. Ship an sdist alongside the wheels so users on unsupported platforms can still build from source.

Troubleshooting #

HTTPError: 403 Forbidden on upload. The trusted publisher is not configured or the workflow identity does not match. Confirm the publisher’s owner/repo/workflow fields exactly match the running job, per the child guide.

400 File already exists. PyPI never allows overwriting a released file. Bump the version; use --skip-existing only for retrying a partially-failed same-version upload.

400 File too large. A vendored wheel exceeded the limit. Request a limit increase or shrink the wheel per handling wheel size limits on PyPI for GDAL.

The Trust Boundary a Release Job Crosses #

A release job is the one place in the pipeline where code you built acquires the authority to speak for your project name on a public index. Everything before it is reversible; everything after it is not, because PyPI never allows a released file to be replaced. That asymmetry is why the authentication design matters more here than anywhere else in the build, and why long-lived API tokens are the wrong tool even though they are the easy one.

A stored token is a bearer credential with no expiry, no audience restriction, and no binding to the job that uses it. Any workflow in the repository can read it, which means a pull request that adds a step to a workflow file can exfiltrate it; any maintainer with repository settings access can read its scope, and rotating it requires touching every project that shares it. Trusted publishing replaces it with an OpenID Connect assertion the CI provider mints for one job, valid for minutes, carrying claims about the repository, the workflow file, and the environment that requested it. PyPI verifies those claims against a publisher you registered in advance, so a token minted for a different workflow — or for a fork — is refused before any bytes are uploaded.

The practical consequence for a geospatial project is that the release job should be the only job with id-token: write permission, should be gated on a tag, and should download artifacts rather than build them. Building inside the release job re-opens every question the validation stage closed and hands compilation of vendored GDAL — thousands of lines of C++ pulled from a source tarball — the same privileges as the upload itself. Separating them means the privileged job runs about six lines of code, all of which a reviewer can read at a glance.

Use a protected deployment environment for the same reason. Binding the trusted publisher to an environment (rather than to the whole repository) lets you require an approval, restrict the branches and tags that may reach it, and get an audit entry per release. For a package whose users install compiled binaries that will run on their machines, that audit trail is not paperwork: a compromised release of a spatial wheel executes attacker-controlled native code inside every downstream data pipeline that upgrades.

Sizing, Mirrors and What Users Actually Download #

Geospatial wheels are large in a way most Python packages are not, and the distribution consequences are worth planning rather than discovering. A self-contained GDAL wheel with PROJ, GEOS, libtiff, libgeotiff, libwebp, SQLite and the datum database routinely lands between 40 MB and 90 MB per platform. Multiply that by the platform grid and a single release can add half a gigabyte to the index; multiply again by the number of releases a year and a busy project becomes a meaningful share of a mirror’s storage.

The numbers below show where the bytes actually go in a typical vendored build, which is the information you need before deciding what to cut:

Two levers do most of the work. Stripping debug information from the bundled objects (strip --strip-unneeded) frequently halves libgdal.so on its own, because a default release build still carries symbol tables that nobody consuming a wheel will ever use. Narrowing the driver set at configure time — disabling the format readers your users demonstrably do not need — removes both code and the transitive dependencies those drivers dragged in. Both are covered in detail by handling wheel size limits on PyPI for GDAL.

The third lever is structural rather than mechanical: ship the data separately. proj.db and the optional datum grids are architecture-independent, so a companion pure-Python data package installed as a dependency is downloaded once per environment rather than once per platform wheel, and updating the datum data no longer requires rebuilding native code on five platforms. The cost is a runtime lookup that must find the data package’s directory, and a hard dependency edge that users cannot skip.

Whatever you choose, publish the decision. Users who install a 90 MB wheel into a container image want to know whether a smaller variant exists, and maintainers of downstream distributions want to know whether your wheel duplicates a system GDAL they already ship. A short paragraph in the README covering wheel size, vendored versions, and the reason for each is far cheaper than the issue thread that replaces it.

Versioning and Compatibility Promises #

The version number on a spatial wheel carries more information than it does for a pure-Python package, because it implicitly promises something about native code the user never sees. Three separate versions are in play at once: your package’s version, the vendored GDAL/PROJ version, and the CPython ABI floor the wheel was built to. Users reason about all three, usually without saying so, and a release that changes one of them silently is the release that generates support load.

Treat the vendored library version as part of your public contract. A minor bump of PROJ can change the transformation pipeline selected for a given pair of coordinate reference systems, which moves results by centimetres to metres depending on the datum involved. That is not a bug in PROJ and it is not a bug in your package, but for a downstream user validating survey data against a fixed tolerance it is a breaking change. Record the vendored versions in the release notes, expose them at runtime so a user can query what they actually have, and consider a minor version bump of your own package whenever the vendored major or minor version moves.

# Make the vendored versions discoverable without importing the whole stack
from osgeo import gdal
import pyproj

print(gdal.__version__)            # e.g. 3.8.4
print(pyproj.proj_version_str)     # e.g. 9.3.1
print(pyproj.datadir.get_data_dir())

The ABI floor is the second promise. A wheel tagged cp39-abi3 claims it will import on every CPython from 3.9 upward; raising that floor in a patch release removes platforms from users who had no reason to expect it. Raise it deliberately, in a minor or major release, and keep publishing the previous line long enough for pinned environments to migrate. The mechanics of that contract are covered by C-API vs CPython ABI compatibility.

The third promise is the platform set itself. Dropping a platform — 32-bit Windows, an old macOS deployment target, a musl variant — is invisible to anyone reading a changelog, because nothing about the version number says which wheels exist. What users experience instead is a resolver that quietly falls back to the sdist and starts compiling GDAL on their laptop, or an installation error deep inside a Docker build that used to work. Announce platform removals in the release notes, keep the last release that supported the platform installable, and where possible publish a final release on the old platform set so a pin has somewhere to land.

Finally, decide early whether pre-releases are part of your process. For packages with large native surfaces they usually should be: an rc published to PyPI can be installed by downstream projects with --pre in their own CI, and that is the only realistic way to discover that your new GDAL vendoring breaks a consumer’s driver before it breaks it in a stable release. Pre-releases cost one extra tag and are ignored by ordinary resolution, so the risk of shipping one is close to zero.

Frequently Asked Questions #

Should I publish an sdist for a package with vendored GDAL? #

Yes, but set expectations in it. The sdist is what lets users on unsupported platforms — a niche architecture, an old glibc, a hardened build environment — compile the package themselves, and it is what several Linux distributions package from. Make the build fail early and clearly when GDAL development headers are missing, rather than halfway through compilation, and document the minimum GDAL/PROJ versions the source build requires.

What happens if the release job dies halfway through the upload? #

Some files land and some do not, because uploads are per-file rather than transactional. Re-run the job with --skip-existing so already-uploaded wheels are ignored and the remainder complete. Do not bump the version to “clean up” a partial upload: the version with missing platforms will still be resolvable by pip and will produce confusing source-build fallbacks for the platforms that never uploaded.

Should release notes list the vendored library versions? #

Always, and near the top. For a package whose behaviour is largely determined by GDAL and PROJ, “vendored GDAL 3.8.5, PROJ 9.3.1, GEOS 3.12.1” is the single most useful line in the notes — it tells a reader whether a driver they need is present, whether a transformation result may have moved, and whether a CVE in an upstream library affects them.

Can I delete a release that shipped a broken wheel? #

You can yank it, which is the correct action, and you cannot re-use the filename. Yanking leaves the file installable for anyone who pins the exact version — so existing lockfiles keep working — while removing it from ordinary resolution. Publish a patch release with the fix immediately afterwards, because a yanked latest version pushes new installs onto whatever came before it.

How do I publish to a private index as well as PyPI? #

Run the same release job twice with different repository targets, not a single job that uploads to both. Private indexes differ in their authentication model — most still use tokens or basic auth rather than OpenID Connect — and mixing them into one step means the privileged public upload inherits whatever credentials the private one needs. Two jobs, each with the narrowest possible permission set, also give you the option of publishing internally first and promoting to PyPI only after downstream pipelines have consumed the internal build.

Does the order of uploads within a release matter? #

It does for large platform grids. Upload the sdist last: as soon as a version exists on the index, resolvers can see it, and a version whose only visible artifact is a source distribution will send every installer into a from-source GDAL compilation until the wheels finish uploading. Publishing the wheels first and the sdist at the end keeps that window closed.

Do I need TestPyPI if every wheel already passed the validation gate? #

It is optional, and its value drops as the gate gets stronger. What it still catches is the class of error that only appears once an index is involved: a project name collision, a metadata field the index rejects, a dependency that resolves differently through an index than through a local --find-links directory. For a package with a large platform grid it is cheap insurance; for a small pure-Python release it is usually redundant.

Related #

• Trusted publishing spatial wheels to PyPI — configuring OIDC so no API token is stored.
• Handling wheel size limits on PyPI for GDAL — shrinking a vendored wheel and requesting a limit increase.
• Testing and Validating Spatial Wheels — the gate every wheel must pass before it reaches this stage.

Further Reading #

• PyPI trusted publishing documentation (docs.pypi.org/trusted-publishers/).