Migrating Legacy setup.py Spatial Builds

Most established geospatial Python packages were built before the current packaging standards existed, and their setup.py carries a decade of accumulated compiler probing, numpy.distutils calls and platform conditionals that newer toolchains no longer support. This guide sits under the Modern Python Build Tooling & Wheel Configuration reference and lays out an incremental migration path from an imperative setup.py to a declarative manifest with a modern backend, without a flag day and without losing the platform knowledge the old script encoded. It targets setuptools 69+, scikit-build-core 0.9+, meson-python 0.16+, Cython 3.0+, NumPy 2.x, GDAL 3.8.x and PROJ 9.3.x.

Prerequisites & Environment #

• A build that currently works, and a way to prove it: a wheel produced from the existing setup.py, validated in a clean container per testing and validating spatial wheels. That artifact is the reference the migration must reproduce.
• setuptools 69+, build 1.2+, and whichever modern backend you are moving to.
• Cython 3.0+ if the package uses Cython; the 0.29 series has different directive defaults and will produce different C.
• A record of what the old script probed for — every gdal-config call and every sys.platform branch is knowledge that has to end up somewhere.

# Capture the reference artifact and its contents before changing anything
python -m build --wheel -o reference/
unzip -l reference/*.whl > reference/contents.txt

Core Configuration #

The migration has a natural order, and following it keeps the build working at every step. Metadata moves first because it is the least risky; the native build moves last because it is the part that can break.

Stages one to four are mechanical and can be done in a single change without touching the compiler. Stage five is the real migration, and it is the one worth doing on a branch with the reference wheel to compare against.

# pyproject.toml after stages 1–4, still using setuptools to compile
[build-system]
requires = ["setuptools>=69", "wheel", "Cython>=3.0", "numpy>=1.23"]
build-backend = "setuptools.build_meta"

[project]
name = "geo-core"
version = "2.4.1"
requires-python = ">=3.9"
dependencies = ["numpy>=1.23"]

[tool.setuptools.packages.find]
where = ["src"]

Step-by-Step Implementation #

1. Move metadata and dependencies into pyproject.toml, leaving setup.py containing only the extension declarations. Build and diff the wheel contents against the reference.

2. Replace numpy.distutils — removed in NumPy 1.26 and unavailable on Python 3.12 — with an explicit include directory, which is all most spatial packages used it for:

   # setup.py, transitional
   import numpy
   from setuptools import Extension, setup
   
   setup(ext_modules=[Extension(
       "geo_core._ext",
       sources=["src/geo_core/_ext.pyx"],
       include_dirs=[numpy.get_include()],
       libraries=["gdal", "proj"],
       define_macros=[("NPY_NO_DEPRECATED_API", "NPY_1_23_API_VERSION")],
   )])
   

3. Write the CMake (or Meson) build alongside the working setup.py, so both exist during the transition and the new one can be compared before it becomes authoritative:

   cmake_minimum_required(VERSION 3.28)
   project(geo_core LANGUAGES C CXX)
   find_package(Python REQUIRED COMPONENTS Interpreter Development.Module NumPy)
   find_package(GDAL 3.8 CONFIG REQUIRED)
   find_package(PROJ 9.3 CONFIG REQUIRED)
   python_add_library(_ext MODULE src/geo_core/_ext.c WITH_SOABI)
   target_link_libraries(_ext PRIVATE GDAL::GDAL PROJ::proj Python::NumPy)
   install(TARGETS _ext DESTINATION geo_core)
   

4. Switch the backend and delete the extension declarations from setup.py:

   [build-system]
   requires = ["scikit-build-core>=0.9", "Cython>=3.0", "numpy>=1.23"]
   build-backend = "scikit_build_core.build"
   

5. Compare the new wheel against the reference — contents, linkage and platform tag — before deleting anything.

Translating What the Old Script Knew #

The riskiest part of the migration is not the mechanics; it is the platform knowledge encoded in conditionals that nobody has read in years. A typical spatial setup.py contains a gdal-config invocation, a fallback for Windows where that binary does not exist, a macOS branch adding -stdlib=libc++, and a handful of defines toggled by environment variables. Each of these is a fact about the world that must survive.

The third row is the one that matters most for a project that will ever cross-compile. A sys.platform check reports the machine running the build, which under cross-compilation is the wrong machine; CMake’s APPLE, WIN32 and CMAKE_SYSTEM_NAME describe the target, which is what the flags need to match. Migrating those conditionals is not a translation exercise — it fixes a latent bug, as cross-compiler toolchain setup explains in detail.

Proving the Migration Did Not Change the Artifact #

The migration is successful when the new wheel is equivalent to the reference, and “equivalent” has three checkable parts: the same files, the same linkage, and the same tag.

Run all three on the same machine, with the same native libraries available, so the only variable is the build system. A difference in the NEEDED list is the most informative single signal: it means the link line changed, and a link line that gained a library is as suspicious as one that lost it.

Verification #

# 1. The migrated wheel contains the same files as the reference
diff <(unzip -l reference/*.whl | awk '{print $4}' | sort) \
     <(unzip -l dist/*.whl      | awk '{print $4}' | sort)
# expected: no differences outside dist-info metadata

# 2. The extension links the same libraries
for w in reference dist; do
  unzip -o $w/*.whl -d /tmp/$w >/dev/null
  readelf -d /tmp/$w/*/_ext*.so | grep NEEDED | sort > /tmp/$w.needed
done
diff /tmp/reference.needed /tmp/dist.needed && echo "linkage unchanged"

# 3. Both wheels behave identically in a clean container
for w in reference/*.whl dist/*.whl; do
  docker run --rm -v "$PWD:/s" python:3.12-slim bash -c \
    "pip install -q /s/$w && python -c 'import geo_core; print(geo_core.transform_one(5.0, 52.0))'"
done
# expected: identical output from both

Optimization & Edge Cases #

• Keep setup.py as a shim only if something needs it. Editable installs and a few older tools benefit from a two-line setup.py that calls setup(); anything longer than that has not finished migrating.
• Cython 3 changes defaults. Language level, binding and several directives differ from 0.29. Set them explicitly in the build file so the generated C does not change silently between contributor machines.
• Do not migrate and upgrade at once. Changing the build system and bumping GDAL in the same change makes any difference impossible to attribute. Migrate first, prove equivalence, then upgrade.
• meson-python is a legitimate destination too. For packages whose native build is a handful of extensions and no external project, Meson’s declarative syntax is shorter than CMake’s; for packages that also build vendored libraries, CMake’s ecosystem is usually easier.
• The sdist needs the new build files. A migrated project whose source distribution omits CMakeLists.txt cannot be built from source at all, and the failure appears only for users on unsupported platforms.

Troubleshooting #

ModuleNotFoundError: No module named 'numpy.distutils'. The package is being built on Python 3.12 or with NumPy 1.26+, where it no longer exists. Replace it with numpy.get_include() — that is what nearly all spatial packages used it for.

The migrated wheel is missing package data. setup.py’s package_data did not carry over. Data files are declared in the backend’s own configuration now; compare the file listings and add what is absent.

The extension links a library it did not before. The new build found a dependency the old link line did not include, usually because the config package pulls in transitive requirements. That is normally an improvement, but confirm it is bundled by the repair step rather than borrowed from the host.

Editable installs stopped working. Modern backends implement them differently, and for a compiled package an editable install must rebuild on change. Check the backend’s editable-mode documentation before concluding it is broken; the behaviour is usually configurable.

Frequently Asked Questions #

Is migrating worth it if the current build works? #

It becomes worth it the moment a supported Python release breaks it, which for numpy.distutils-based builds has already happened, and again whenever cross-compilation, reproducibility or a new platform enters the picture. A working legacy build with no such pressures is a reasonable thing to leave alone — but know which of those pressures is coming.

Can the migration be done incrementally on a shipping package? #

Yes, and it should be. Stages one to four change no compiled output at all and can ship in ordinary releases. Only stage five needs a branch and a careful comparison, and even that can ship behind a release candidate so downstream projects test it before it becomes the default.

Which backend should a GDAL-linked package choose? #

scikit-build-core if the project already knows CMake or vendors native libraries, because CMake’s find_package ecosystem is what GDAL and PROJ ship configuration for. meson-python if the native build is small and self-contained and you value a shorter, more declarative build file. Both produce equivalent wheels; the choice is about which build language your maintainers will be comfortable in.

What happens to a package that also supports conda-forge? #

Very little — the conda recipe drives the same backend and benefits from the same clarity. If anything the migration simplifies the recipe, because the native dependencies are discovered through config packages rather than through a script that has to be told where the prefix is.

Can I migrate while continuing to support an old Python version? #

Yes, and it is one of the better reasons to migrate: the modern backends support the same interpreter range that setuptools does, while the legacy pieces most likely to break — numpy.distutils in particular — are the ones already unavailable on newer Pythons. A migration usually widens the supported range rather than narrowing it.

What if the package has no compiled extensions at all? #

Then the migration is stages one to four and nothing else, and it takes an afternoon. Move the metadata and dependency declarations into pyproject.toml, declare package discovery, and delete setup.py entirely. There is no native build to reproduce, so the comparison step reduces to checking that the wheel contains the same files.

Should the version number change during a migration? #

Not because of the migration itself — it produces an equivalent artifact, which is the whole point of the comparison step. Ship it as a patch release with a note saying the build system changed, so that anyone who does hit a difference knows immediately where to look.

Sequencing a Migration Nobody Notices #

A migration that lands as one large change is a migration that gets reverted the first time a downstream project reports a broken build. Sequencing it so that each step is separately shippable and separately revertible is what makes it survivable on a package with real users.

Ship stages one to four as ordinary releases. Metadata, dependencies, package discovery and build requirements produce byte-identical compiled output; the only thing that changes is where the declarations live. Each can go out in a normal patch release, and if something is wrong the blast radius is metadata rather than binaries. By the end of this phase the setup.py contains nothing but extension declarations, which makes the remaining work legible.

Do stage five behind a release candidate. The native build change is the one that can alter the artifact, so it deserves a pre-release that downstream projects can install with --pre in their own CI. Two weeks of that is worth more than any amount of local testing, because downstream projects exercise platform and driver combinations you do not have.

Keep both build paths alive briefly. During stage five it is entirely reasonable for the repository to contain both a working setup.py and a new CMakeLists.txt, with the backend switch as a one-line change in pyproject.toml. That makes A/B comparison trivial and rollback instant, and it costs only the discipline to delete the old path once the new one has shipped.

Migrate the CI last. The temptation is to modernise the workflow at the same time — new runners, new cibuildwheel version, new platforms. Resist it. A build failure during the migration should have exactly one candidate cause, and a simultaneous CI change doubles the search space at the worst possible moment.

A rough calendar for a mid-sized spatial package looks like a week for stages one to four including review, a week to write and compare the native build, two weeks of release candidate, and an afternoon to delete the old path. The step that consistently takes longer than expected is not the CMake authoring; it is recovering the knowledge encoded in the old conditionals, which is why capturing that inventory early — before touching anything — is the highest-leverage half hour in the whole project.

What the Old Script Was Protecting You From #

There is a failure mode in these migrations that deserves naming: deleting a conditional because it looks obsolete, and rediscovering six months later why it existed. Legacy setup.py files in this domain accumulate defensive code for reasons that were real, and a few of those reasons are still real.

The Windows fallback exists because gdal-config does not. The script that probes it on Linux and macOS and hard-codes paths on Windows is not sloppy; it is reflecting the fact that the tool is a Unix convention. In the migrated build the equivalent is a config-mode find_package that works on all three, which is genuinely better — but only if the Windows prefix is actually being told where to look, which the old hard-coded path was doing.

The macOS standard-library flag exists because of a real ABI split. Code that adds -stdlib=libc++ on Darwin is defending against linking a C++ library against the wrong standard library, which produces link errors that mention mangled names and nothing else. Modern toolchains default correctly, so the flag is usually redundant now — but confirm that the vendored C++ libraries were built the same way before removing it.

The environment-variable toggles exist because someone needed a variant build. A GEO_STATIC or USE_SYSTEM_GDAL switch usually traces back to a downstream packager who builds against system libraries rather than vendored ones. Deleting it breaks their build silently, and they find out at their next release. Migrating those toggles to declared build options keeps the capability and makes it discoverable, which is strictly better than either keeping the environment variable or dropping the feature.

The minimum-version checks exist because an older version genuinely failed. A guard rejecting GDAL below some version was written after a bug report. Carrying it into the new build as a find_package version requirement preserves the knowledge in a form the build system enforces, rather than as a comment nobody reads.

The habit that makes all of this manageable is to write the inventory down before starting: one line per conditional, saying what it does and what you believe it protects against. Reviewing that list with whoever has been maintaining the package usually resolves half of the entries immediately, and the remaining half are exactly the ones worth being careful with. It is a much cheaper artifact than the archaeology required to reconstruct the same information after the fact.

Related #

• Integrating CMake with scikit-build-core — the destination for most spatial migrations, in detail.
• Mastering pyproject.toml for spatial wheels — which table each migrated setting belongs in.
• Cross-compiler toolchain setup — why platform conditionals have to describe the target rather than the host.
• Testing and validating spatial wheels — the gate that proves the migrated artifact still works.

Further Reading #

• The scikit-build-core and meson-python migration guides for projects moving off setuptools.