Sysroot setup for manylinux cross builds

This page answers one question: how do you assemble and populate an aarch64 sysroot inside a manylinux container so that a cross-compiled GDAL/PROJ extension finds target headers and target libraries — and nothing from the host — at every stage of the build? It sits inside the Cross-Compiler Toolchain Setup section of the Geospatial C-Extension Fundamentals & ABI Architecture reference, and gives you the directory layout, the population steps, and the assertions that prove nothing host-shaped leaked in.

What a populated cross sysroot has to contain The sysroot mirrors a target root filesystem. It holds the target C library and its headers, the target C plus plus runtime, the geospatial libraries GDAL, PROJ, GEOS and SQLite with their headers, and the pkg-config metadata files describing them. Alongside it, on the host side, sit the cross compiler binaries and the build tools, which are host executables and must never be searched for inside the sysroot. the sysroot — target files only /opt/sysroots/aarch64/ usr/include/ gdal.h, proj.h, sqlite3.h usr/lib/ libgdal.so, libproj.so usr/lib/pkgconfig/ gdal.pc, proj.pc lib/ libc.so.6, libstdc++.so.6 usr/share/proj/ proj.db every file here is aarch64 or architecture-independent found through MODE_LIBRARY and MODE_INCLUDE set to ONLY the host side — never in the sysroot aarch64-linux-gnu-gcc aarch64-linux-gnu-ld cmake · ninja · pkg-config python · the build backend these are x86_64 executables that emit aarch64 code found through MODE_PROGRAM set to NEVER

Context & Root Cause

A cross-compiler needs somewhere to look for the target’s headers and libraries, and that somewhere is the sysroot: a directory tree shaped like the root filesystem of the machine the output will run on. The compiler is told about it once, with --sysroot, and every include and library search is then rooted there instead of at /.

The reason spatial cross-builds fail is that the sysroot is usually incomplete. A manylinux image ships the cross-GCC and a minimal target C library, which is enough to compile a pure C extension and nowhere near enough to link one against GDAL. The build then does one of two things: it fails with a missing header, which is annoying but honest, or it finds the host’s gdal.h through a path the toolchain file did not lock down, compiles against x86_64 struct layouts, and produces an object that links or fails much later with an error that names something unrelated. Populating the sysroot completely, and asserting that it is complete, is what turns the second outcome into the first.

Solution / Fix

This targets the manylinux_2_28_x86_64 image, GCC 13 cross-toolchains, GDAL 3.8.x / PROJ 9.3.x, and the toolchain file described in CMake toolchain file for cross-compiling PROJ.

1. Create the tree and record where it is 

export SYSROOT=/opt/sysroots/aarch64
mkdir -p "$SYSROOT"/usr/{include,lib,share}
mkdir -p "$SYSROOT"/usr/lib/pkgconfig

2. Populate the base system from target packages

The most reliable source of a target C library and C++ runtime is the target distribution’s own packages, unpacked rather than installed:

# Unpack target RPMs into the sysroot without running any scriptlets
cd "$SYSROOT"
for pkg in glibc glibc-devel libstdc++ libstdc++-devel zlib zlib-devel; do
  dnf download --arch aarch64 --destdir /tmp/rpms "$pkg"
done
for rpm in /tmp/rpms/*.rpm; do rpm2cpio "$rpm" | cpio -idmu; done

3. Build the geospatial stack into the same prefix

SQLite first (PROJ needs it), then PROJ, then GDAL — each cross-compiled and installed into the sysroot rather than into the host prefix:

cmake -S proj-9.3.1 -B build-proj -G Ninja \
  -DCMAKE_TOOLCHAIN_FILE="$PWD/aarch64.toolchain.cmake" \
  -DCMAKE_INSTALL_PREFIX="$SYSROOT/usr" \
  -DBUILD_TESTING=OFF
cmake --build build-proj && cmake --install build-proj

4. Point every discovery mechanism at it 

export PKG_CONFIG_LIBDIR="$SYSROOT/usr/lib/pkgconfig:$SYSROOT/usr/share/pkgconfig"
export PKG_CONFIG_SYSROOT_DIR="$SYSROOT"
export CMAKE_TOOLCHAIN_FILE="$PWD/aarch64.toolchain.cmake"

PKG_CONFIG_LIBDIR replaces the search path; PKG_CONFIG_PATH would append to it and leave the host’s .pc files reachable, which is the single most common way an architecture mix enters the build.

Verification 

# 1. Everything in the sysroot's library directories is aarch64
find "$SYSROOT" -name '*.so*' -type f -exec sh -c \
  'readelf -h "$1" 2>/dev/null | grep -q AArch64 || echo "HOST FILE: $1"' _ {} \;
# expected: no output

# 2. pkg-config resolves from the sysroot and rewrites the prefix
pkg-config --cflags proj
# expected: -I/opt/sysroots/aarch64/usr/include   (never /usr/include)

# 3. A configure resolves the target GDAL, not the host's
cmake -B /tmp/probe -DCMAKE_TOOLCHAIN_FILE="$PWD/aarch64.toolchain.cmake" .
grep -E 'GDAL_INCLUDE_DIR|PROJ_DIR' /tmp/probe/CMakeCache.txt
# expected: both paths inside $SYSROOT

The first check is the one to run after every change to the sysroot: it walks the whole tree and names any file that is not target architecture, which is exactly the contamination that produces confusing link errors hours later.

Populating in the Right Order

The dependency order matters because each library records where it found the previous one. Building GDAL before PROJ is installed produces a GDAL that either fails to configure or silently disables the coordinate-transformation support you needed.

The order in which the sysroot must be populated Five stages in sequence. The base system — the C library, the C plus plus runtime and zlib — comes first. Then SQLite and libtiff, which PROJ requires. Then PROJ itself, including its database. Then GEOS. Then GDAL, which links all of the preceding. Each stage installs into the same sysroot prefix so the next stage discovers it there, and skipping a stage produces a later library with silently disabled features rather than an error. base system glibc · libstdc++ sqlite · tiff PROJ needs both PROJ library + proj.db GEOS geometry ops GDAL links all above each stage installs into the same prefix, so the next stage finds it through the sysroot building out of order does not fail — it produces a library with features silently disabled check the configure summary of each stage before moving on

Reading the configure summary at each stage is the cheap insurance. PROJ prints whether it found SQLite and TIFF; GDAL prints a long list of enabled drivers and optional dependencies. A GDAL built against a sysroot missing PROJ still builds — it just cannot reproject anything, and the failure surfaces as a runtime error in a user’s pipeline rather than as a build failure in yours.

Caching the Sysroot Instead of Rebuilding It

Populating a sysroot takes as long as compiling GDAL, because that is most of what it is. Rebuilding it on every wheel build is the single largest avoidable cost in a cross-compiling pipeline.

Three ways to reuse a populated sysroot between builds Three approaches. Baking the sysroot into a custom container image gives the fastest start and requires rebuilding the image when a library version changes. Caching the sysroot directory through the CI cache is simpler to set up but pays an upload and download per job. Publishing the sysroot as a versioned tarball in an artifact store works across CI systems and needs a fetch step. All three key on the same thing: the set of library versions the sysroot contains. bake into a custom image fastest start; the sysroot is a layer, so a wheel build begins with everything present cost: an image rebuild and a digest bump whenever a library version changes cache the directory simplest to add; key on the native version lock so a version change invalidates it cost: upload and download per job, which for a full sysroot is not small publish a versioned tarball works across CI systems and for local reproduction; fetch and unpack in one step cost: somewhere to store it, and a checksum to verify what was fetched whichever you pick, the key is the library version set — never the branch or the commit

The image approach is usually best for a project that already builds a custom base image for other reasons, and it composes well with the hybrid pattern in cibuildwheel vs manual Docker matrix for GDAL wheels: the image carries the sysroot, the wheel build carries nothing, and a dependency bump is a digest change reviewed like any other.

Pitfalls & Alternatives

Copying host libraries into the sysroot “to make it build”. This always works and always produces a broken artifact. The architecture check above exists precisely to catch it, and it should run as a build step rather than as a habit.

Using PKG_CONFIG_PATH instead of PKG_CONFIG_LIBDIR. Appending leaves the host’s metadata reachable, so a library present in both places resolves to the host’s. Replace the path; never extend it in a cross build.

Forgetting PKG_CONFIG_SYSROOT_DIR. Without it, .pc files return absolute paths as written at their own build time, which point outside the sysroot. The flags look plausible and reference the wrong tree.

Assuming architecture-independent files do not matter. proj.db is architecture-independent and still has to be in the sysroot for a GDAL build to find PROJ’s data at configure time — and, separately, has to be bundled into the wheel as bundling proj.db and datum grids in a wheel describes.

Frequently Asked Questions

Can the sysroot be shared between architectures?

No — it describes one target’s root filesystem, so aarch64 and x86_64 need separate trees. What can be shared is the script that populates them, parameterised by the triple, which keeps the two from drifting apart in content even though they are separate directories.

How large does a populated sysroot get?

A base system plus the geospatial stack typically lands between 400 MB and 1 GB, dominated by GDAL and the debug information in the target libraries. Stripping the sysroot’s libraries is safe and roughly halves it, which matters if you are caching or shipping the tree between jobs.

Does the sysroot need the target’s Python?

Only if you are compiling against the Python headers, which for an abi3 extension means the limited-API headers rather than a full interpreter. Most cross builds get these from the build backend rather than from the sysroot; check which, because a mismatch produces confusing errors about Py_LIMITED_API.

What if the target distribution differs from the manylinux baseline?

The sysroot must match the policy baseline, not a particular distribution: the point is a glibc old enough for the tag you intend to claim. Populating it from a newer distribution’s packages produces a wheel whose computed tag is higher than expected, which the platform-tag check will catch.

Can I unpack Debian packages instead of RPMs?

Yes — the mechanism is the same, using dpkg-deb -x in place of rpm2cpio. What matters is that the packages come from a distribution whose glibc matches the policy baseline you intend to claim, not which packaging format they arrive in.

How do I keep the sysroot from drifting between the image and a developer machine?

Build it exclusively from a script that reads the pinned version file, and publish the result rather than the recipe. A developer who fetches the same tarball the CI uses is working with identical bytes; one who runs the script on a different host may not be, because the packages it downloads can move.

Should the sysroot include a compiler?

No. The compiler is a host executable that emits target code, so it belongs on the host side and is found through the program search mode set to never look inside the sysroot. A sysroot containing a target compiler is a sign that something has been unpacked from the wrong architecture, which the architecture audit will report. Running the architecture audit as the last step of the population script, rather than as something a person remembers to do, is what keeps a host file from surviving into a build weeks later.