Fixing memory leaks in GDAL Python bindings

This page answers one question: why does a long-running process using GDAL’s Python bindings grow to gigabytes of resident memory, and how do you enforce the dataset-close discipline that stops it — with a CI leak gate that fails the build before the leak ships? It sits inside the Memory Management in Geospatial Extensions cluster of the Geospatial C-Extension Fundamentals & ABI Architecture reference, and gives you the ownership rules, the correct teardown order, and a valgrind/tracemalloc gate that catches regressions.

Why a dropped GDAL dataset reference leaks the native raster block cache Two columns compare teardown. On the left, a Python variable holding a GDAL Dataset goes out of scope but a lingering band reference keeps the C++ GDALDataset alive, so its block cache of raster tiles is never freed and native memory grows. On the right, explicit ds.Close after dropping the band reference destroys the GDALDataset, frees the block cache, and returns memory to the allocator. leak: lingering reference fixed: ordered teardown band = ds.GetRasterBand(1) del ds # band still holds it GDALDataset stays alive block cache of tiles never freed RSS climbs each iteration band = None # drop first ds.Close() # explicit GDALDataset destroyed block cache freed to allocator RSS flat across iterations

Context & Root Cause

GDAL’s Python bindings (the SWIG-generated osgeo.gdal module) wrap C++ objects whose lifetime is not governed by Python’s garbage collector in the way maintainers assume. A Dataset owns a block cache — decoded raster tiles held in native memory — that can reach hundreds of megabytes per open file. When you write the common pattern of opening a dataset in a loop and relying on del or scope exit to clean up, two things go wrong. First, any surviving reference to a child object (a Band, which internally holds its parent Dataset) keeps the whole C++ object graph alive, so the native cache is never released. Second, GDAL flushes writes only on explicit close, so a dropped write-mode dataset can both leak and corrupt output.

The result is a process whose Python heap looks healthy under tracemalloc while its resident set climbs relentlessly, because the leak lives in the native allocator arena, not the interpreter’s. This is the arena-boundary problem that Memory Management in Geospatial Extensions frames in general; here the concrete rule is that GDAL objects require ordered, explicit teardown, not reference-count roulette.

Solution / Fix

This targets GDAL 3.6–3.9 with the osgeo Python bindings. The rule set is small but strict.

1. Close datasets explicitly, in child-before-parent order 

from osgeo import gdal
gdal.UseExceptions()

def process(path):
    ds = gdal.Open(path)
    band = ds.GetRasterBand(1)
    stats = band.ComputeStatistics(False)
    band = None          # drop the child reference FIRST
    ds.Close()           # GDAL 3.7+: deterministic close + flush
    return stats

ds.Close() (added in GDAL 3.7) is the deterministic teardown entry point; on older versions assign ds = None only after every child (band, layer, feature) is already None. The ordering is non-negotiable because the child holds the parent.

2. Cap the block cache so a leak is bounded, not unbounded 

# Bound GDAL's global raster block cache (bytes). A hard cap turns a slow
# leak into a visible plateau you can alarm on.
gdal.SetCacheMax(256 * 1024 * 1024)   # 256 MB

3. Use context managers to make ordering automatic 

from contextlib import contextmanager

@contextmanager
def open_raster(path):
    ds = gdal.Open(path)
    try:
        yield ds
    finally:
        ds.Close()        # runs even on exception — no leaked handle

with open_raster("scene.tif") as ds:
    arr = ds.ReadAsArray()   # arr is a NumPy copy; safe after close

GDAL 3.8+ makes Dataset itself a context manager, so with gdal.Open(path) as ds: works directly. Either form guarantees the close runs on the exception path, which manual del does not.

Verification 

# 1. RSS must be flat across many iterations, not climbing
python - <<'PY'
import os, resource
from mymod import process
for i in range(500):
    process("scene.tif")
    if i % 100 == 0:
        rss = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss // 1024
        print(f"iter {i}: {rss} MB")
PY
# expected: RSS plateaus after warm-up; a rising staircase means a leak remains

# 2. Valgrind gate for native leaks (run on a native arch, not emulation)
PYTHONMALLOC=malloc valgrind --leak-check=full --error-exitcode=1 \
  python -c "from mymod import process; process('scene.tif')"
# expected: "definitely lost: 0 bytes" and exit code 0

PYTHONMALLOC=malloc disables pymalloc’s pooled allocator so Valgrind sees real malloc/free pairs — without it, false negatives hide the leak. Wire this command into CI as a nightly gate, as the checklist in Memory Management in Geospatial Extensions recommends.

Pitfalls & Alternatives

Relying on del ds alone. del only decrements the reference count; if any band, layer, or feature still references the dataset, the native object survives and the cache leaks. Drop children first, then close.

Holding a Band past its Dataset. A Band is a view into its parent’s memory. Keeping the band and closing the dataset is both a leak (the dataset stays alive) and a latent use-after-free once you do close it. Copy the data you need into a NumPy array and drop the band.

Assuming write flushes happen automatically. A write-mode dataset that is garbage-collected instead of closed may lose buffered blocks, producing a truncated GeoTIFF. Always Close() writers explicitly. For where these buffers sit relative to the interpreter heap and the PROJ transform buffers that share the arena, see the parent guide.