Testing coordinate accuracy across PROJ versions

This page answers one question: a PROJ patch bump can change which transformation pipeline is chosen for a given datum pair, moving results by centimetres to metres — so how do you build a fixture set that catches that in CI rather than in a user’s survey data? It sits inside the Testing and Validating Spatial Wheels section of the Modern Python Build Tooling & Wheel Configuration reference, and gives you the fixtures, the tolerances and the way to report a change rather than merely fail on it.

Why the same coordinate pair can move between two PROJ patch releases A transformation request names a source and target coordinate reference system. PROJ searches its database for candidate coordinate operations and ranks them by accuracy and availability. A patch release can add an operation, change an accuracy figure or change grid availability, so a different candidate wins. The transform still succeeds; the answer differs by the difference between the two pipelines. a CRS pair EPSG:4277 → 4258 candidate operations ranked by accuracy grid-based, 0.1 m chosen when the grid is present Helmert, 3 m chosen when it is not a result that differs a patch release can change the ranking, the availability or the accuracy figures — and the call still succeeds

Context & Root Cause

PROJ does not apply a fixed formula per coordinate reference system pair. It searches its database for coordinate operations connecting them, ranks the candidates by declared accuracy and by whether the resources they need are available, and applies the winner. That design is what makes high-accuracy transformations possible, and it means the answer depends on the database, on which grids are present, and on whether network access is permitted.

Any of those can change in a patch release. A new operation is added; an accuracy figure is corrected; a grid becomes available. The transformation still succeeds and returns a plausible number, and the difference between the old pipeline and the new one can be metres for datum pairs that rely on grids. For a web map that is invisible; for cadastral, survey or engineering work it is a defect that propagates silently into downstream data — which is exactly why it belongs in the validation gate rather than in a user’s hands.

Solution / Fix

This targets PROJ 9.3.x, pyproj 3.6+, and the wheel validation gate described in testing and validating spatial wheels.

1. Pick fixtures that exercise different pipeline kinds 

# tests/fixtures/coords.py — each entry exercises a different mechanism
CASES = [
    # (name,        src,   dst,   x,       y,      expect_x,        expect_y,      tol_m)
    ("webmerc",     4326,  3857,  5.0,     52.0,   556597.453966,   6800125.454397, 1e-3),
    ("osgb_grid",   4277,  4258, -1.5,     53.8,  -1.498799,        53.800435,      0.05),
    ("nad27_conus", 4267,  4269, -95.0,    40.0,  -95.000389,       40.000021,      0.05),
    ("vertical",    5714,  5773,  8.0,     47.0,   0.0,             0.0,            0.10),
]

2. Assert against a tolerance in metres, not in degrees 

import math
import pytest
import pyproj

def _metres(dlon, dlat, lat):
    return math.hypot(dlon * 111_320 * math.cos(math.radians(lat)), dlat * 110_540)

@pytest.mark.parametrize("case", CASES, ids=lambda c: c[0])
def test_transform_accuracy(case):
    name, src, dst, x, y, ex, ey, tol = case
    t = pyproj.Transformer.from_crs(src, dst, always_xy=True)
    gx, gy = t.transform(x, y)
    err = _metres(gx - ex, gy - ey, y) if dst in (4258, 4269) else math.hypot(gx - ex, gy - ey)
    assert err <= tol, f"{name}: moved {err:.3f} m (PROJ {pyproj.proj_version_str})"

3. Pin the conditions the result depends on 

@pytest.fixture(autouse=True)
def deterministic_proj(monkeypatch):
    monkeypatch.setenv("PROJ_NETWORK", "OFF")     # no grid downloads
    monkeypatch.delenv("PROJ_DATA", raising=False)  # use the bundled database

4. Report the pipeline, not just the number 

def test_pipeline_is_stable():
    t = pyproj.Transformer.from_crs(4277, 4258, always_xy=True)
    print(t.description)          # the operation PROJ selected
    assert "OSGB" in t.description

Verification 

# 1. The fixtures pass against the wheel you are about to publish
docker run --rm -v "$PWD:/s" python:3.12-slim bash -c \
  "pip install -q /s/dist/*.whl pytest && PROJ_NETWORK=OFF pytest /s/tests/test_accuracy.py -q"

# 2. The selected pipelines are recorded for comparison
python - <<'PY' | tee dist/pipelines.txt
import pyproj
for src, dst in [(4326, 3857), (4277, 4258), (4267, 4269)]:
    t = pyproj.Transformer.from_crs(src, dst, always_xy=True)
    print(f"{src}->{dst}\t{pyproj.proj_version_str}\t{t.description}")
PY

# 3. Nothing changed since the last release
diff dist/pipelines.txt release-artifacts/2.4.1/pipelines.txt \
  && echo "pipelines unchanged" || echo "REVIEW: a pipeline selection changed"

The third check is the one that turns this from a pass/fail test into information. A changed pipeline is not necessarily wrong — it is frequently an improvement — but it is always something a downstream user of survey-grade data needs to be told about, and the diff is what makes telling them possible.

Choosing Fixtures That Actually Discriminate

A fixture set of ten web-Mercator points proves almost nothing, because that transformation is a closed-form projection with no database lookup at all. Useful fixtures are chosen to exercise the mechanisms that can change.

Four kinds of transformation and what each fixture would detect A pure projection such as geographic to web Mercator is closed-form and detects only a gross library failure. A datum shift by Helmert parameters detects a changed parameter set in the database. A grid-based datum shift detects a missing grid or a changed pipeline selection, which is the most common silent regression. A vertical transformation detects geoid model changes, which are the least frequently tested and most often wrong. pure projection · 4326 → 3857 closed-form; no database lookup detects only a gross failure — keep one, do not rely on it Helmert datum shift parameters come from the database detects a corrected or added parameter set grid-based datum shift depends on which grids are present detects the most common silent regression — always include one vertical transformation geoid models, rarely exercised detects geoid changes — the least tested and most often wrong

The third row is the one that catches the packaging failures this site is largely about. If your wheel ships proj.db but not the grids, a grid-based fixture falls back to the Helmert pipeline and misses by metres — which is precisely the state a user with offline requirements would be in. Asserting a grid-based case with a tight tolerance turns “we bundled the data” from a belief into a test.

Pick the regions your users work in. A package used mainly in Britain should have an OSGB fixture; one used in North America should have a NAD27 case. Fixtures for regions nobody uses cost the same to run and detect changes nobody will notice.

Reporting a Change Rather Than Just Failing

When a fixture moves, the useful outcome is a decision, not a red build. Structuring the test to say what changed makes the decision possible.

What a useful accuracy-regression report contains A useful report names the coordinate reference system pair, the old and new results, the distance between them in metres, the PROJ version before and after, and the pipeline description PROJ selected in each case. With those five facts a maintainer can tell within a minute whether the change is an upstream improvement to be announced or a packaging defect to be fixed. accuracy regression: osgb_grid 4277 → 4258 at (-1.5, 53.8) was -1.498799, 53.800435 (PROJ 9.3.1) now -1.498812, 53.800418 (PROJ 9.3.2) moved 2.14 m pipeline: OSGB36 to ETRS89 (2) → OSGB36 to ETRS89 (5) the last line is what decides the response: a different operation was selected, which is an upstream change had the pipeline stayed the same and the number moved, the cause would be data — a missing or changed grid

That distinction — same pipeline with a different number versus a different pipeline — is the whole diagnostic. A different pipeline means PROJ’s database changed its mind, which is an upstream improvement to be recorded in the release notes. The same pipeline producing a different number means the data underneath it changed, which for a vendored wheel usually means your proj.db and your libproj are no longer the pair they should be.

Pitfalls & Alternatives

Leaving network transforms enabled. With PROJ_NETWORK=ON a missing grid is fetched silently, so the test passes on a networked runner and the offline user gets a different answer. Turn it off in the fixture and turn it on only in a test that specifically covers the networked path.

Asserting in degrees. A tolerance of 1e-6 degrees means a different distance at different latitudes and tells a reader nothing. Convert to metres; the number then means something and the tolerance can be justified.

Testing only round-trips. A transformation and its inverse can both be wrong in compensating ways, so a round-trip returning the original point proves consistency rather than correctness. Assert against known published values.

Updating the expected value whenever it fails. That converts the test into a record of whatever the current version does. Investigate first, then update deliberately with a note saying which upstream change caused it.

Frequently Asked Questions

Where do the expected values come from?

From an authoritative source rather than from a previous run of your own code: national mapping agency examples, EPSG’s own test points, or published transformation examples. A value captured from your library records what it did, not what is correct.

What tolerance is reasonable?

Tight enough to catch a pipeline change and loose enough to survive floating-point noise — a few centimetres for grid-based shifts, a millimetre for closed-form projections. If a tolerance has to be metres to pass, the fixture is telling you the pipeline is not the one you think.

Should these run against the wheel or the source tree?

Against the installed wheel in a clean container, because the question is what the artifact does. Running them against a development environment tests whatever PROJ that environment has, which is exactly the confusion the validation gate exists to avoid.

How many fixtures are enough?

Four to six, chosen to cover the mechanism types and the regions your users work in. Beyond that the marginal detection rate falls off quickly, and a large fixture set that nobody maintains drifts into being ignored.

Do I need to test every PROJ version?

No — test the version you vendor, on every release. The fixtures exist to detect a change when you move the pin, which is the moment the risk appears. Testing a matrix of PROJ versions is a job for PROJ’s own test suite.

What if a change is genuinely an improvement?

Update the expected value, record the change in the release notes with the old and new pipelines named, and bump your own minor version rather than shipping it in a patch. Users comparing results across your releases need to be able to see where the change happened.

Should the fixtures run against every platform wheel?

Yes, because the data is bundled per wheel and a packaging mistake can affect one platform only. Running them in the same clean-container loop that already exercises the import costs a second per platform and covers the case where one wheel shipped without its grids.

What tolerance suits a vertical transformation?

Looser than a horizontal one — geoid models differ by more than datum grids do — and still tight enough to detect a missing model rather than a refinement.