Decomposing Earth Embeddings with Sparse Autoencoders¶

Vitus Benson^1,2,3 Fanny Yang³ Markus Reichstein^1,2

¹Max Planck Institute for Biogeochemistry, Jena, Germany ²ELLIS Unit Jena, Jena, Germany ³ETH Zürich, Zürich, Switzerland

NeurIPS 2026

Paper Code Data & checkpoints Reproduce

Overview: an SAE decomposes a pixel's embedding into sparse features, an evaluation protocol scores them, and two features map to regions — (a) A sparse autoencoder (SAE) decomposes the embedding of each pixel into a sparse code whose features are meant to each stand for one concept, whereas a dense dimension mixes many. (b) The evaluation protocol scores reconstruction at matched sparsity, alignment with reference concepts and spatial scale, each against baselines, including an SAE with a frozen random decoder. (c) Where two SAE features are active: the share of pixels per 2°×2° cell.

TL;DR¶

Sparse autoencoders split Earth embeddings (AlphaEarth, Tessera) into sparse features that reconstruct the embedding better than PCA, ICA or NMF at matched sparsity. On AlphaEarth, 88% of the features align with at most one of 20 concept groups, where a raw dimension aligns with a median of twelve. An SAE with a frozen random decoder is just as sparse in its concepts and as fine in its spatial scale, so those properties come from sparsity; only a trained dictionary also reconstructs the embedding accurately.

Abstract¶

Earth observation foundation models produce global metre-resolution embeddings that underpin work across agriculture, ecology, hydrology, and beyond. Yet these embeddings are opaque: each dimension entangles unrelated physical concepts, which makes them difficult to interpret and their robustness hard to assess. Here, we propose sparse autoencoders (SAEs) as a principled way to decompose Earth embeddings into sparse, approximately monosemantic features over an overcomplete dictionary. Because the standard practice of using LLMs to judge interpretations does not transfer to satellite pixels, and spatial autocorrelation matters, we introduce a tailored evaluation protocol. It scores any SAE on reconstruction at matched sparsity, alignment with reference concepts, and spatial scale, each against baselines that include an SAE whose decoder is a frozen random dictionary. On this protocol, SAEs reconstruct AlphaEarth and Tessera embeddings better than every baseline at matched sparsity. Their features align with far fewer reference concepts than raw embedding dimensions (on AlphaEarth, 88% align with at most one of 20 concept groups, most with none, against a median of 12 per raw dimension). They also carry more of their variance at fine spatial scales (23 to 32% within a 640 m patch for BatchTopK and Matryoshka, against 9%). An SAE with a frozen random decoder shares both properties, which therefore follow from sparsity, whereas only a learned dictionary reconstructs the embedding accurately: on AlphaEarth, trained SAEs leave eight times less variance unexplained. We test the sparse code on two applications, discovering land-cover sub-concepts and detecting out-of-distribution pixels by area-of-applicability estimation. Both indicate how sparse features could be used, but neither is yet reliable enough for practical use. Together, these results are a step toward Earth observation models whose features can be inspected, opening pathways for scientific discovery and more trustworthy geospatial applications.

Key findings¶

SAEs reconstruct better than every baseline¶

Fraction of variance explained against active features per pixel, for SAEs and baselines — Fraction of variance explained against the number of active features per pixel, on AlphaEarth (a) and Tessera (b). Every trained SAE lies above PCA, ICA, NMF and the raw dimensions at every sparsity, and well above the SAE with a frozen random decoder (grey).

Most features align with at most one concept¶

Number of concept groups per feature against its best alignment, and alignment heatmaps — (a) How many of 20 concept groups (water, built-up, forest, …) each feature aligns with. Most SAE features align with one or none, raw dimensions with many. (b, c) The best feature per concept group: the SAE's align with their own group, the raw dimensions' with several.

Features vary on finer spatial scales¶

Share of features by correlation length for SAEs, the frozen decoder and raw dimensions — Share of features by the distance over which they stay correlated. SAE features vary over short distances more often than raw dimensions. The frozen-decoder SAE does too, so the fine spatial scale comes with sparsity.

Features find sub-concepts, but not reliably yet¶

Water and built-up features on example patches, and three woody-vegetation features with their maps — (a) The best water and built-up features fire on the class and stay silent elsewhere, unlike the best raw dimensions. (b–f) Three features of woody vegetation split it into tropical wet forest, dry savanna and boreal forest, matching climate, biome and vegetation maps. The frozen-decoder SAE has such features too, though they cover less of each region. Neither this nor the second application, area-of-applicability estimation, is yet reliable enough for practical use.

Reproducing the paper¶

Every figure and table can be drawn in two ways:

From the paper data: run the figure script with --paper-data <dir> on the figure-data files of the Zenodo record. This draws the camera-ready figure from the exact numbers the paper was made from, in seconds, on a laptop.
From scratch: build the data store, train the SAEs, run the evaluation stages, then run the same figure script. This takes about 80 GPU-hours and a store of about 465 GB.

Reproducing: data, checkpoints and training sets up the environment, the paper's data store, the Zenodo checkpoints and the training sweep. Reproducing the figures and tables gives, for each figure and table, the cells to train, the evaluation stages and both plotting commands.

The configs live in configs/neurips2026/ (README) and the scripts in scripts/neurips2026/, one per figure or table. Every script takes --out <dir> and writes a PDF and a PNG there, plus a JSON with the numbers the paper quotes; the table scripts also write a LaTeX tabular. python scripts/neurips2026/<script>.py -h lists the arguments of each.

Citation¶

Benson, V., Yang, F., & Reichstein, M. (2026). Decomposing Earth Embeddings with Sparse Autoencoders. Advances in Neural Information Processing Systems (NeurIPS 2026).

@inproceedings{benson2026decomposing,
  title     = {Decomposing Earth Embeddings with Sparse Autoencoders},
  author    = {Benson, Vitus and Yang, Fanny and Reichstein, Markus},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS 2026)},
  year      = {2026},
  url       = {https://openreview.net/forum?id=kB29xkjNQK}
}

Acknowledgements¶

This work has received funding from the European Union under the Horizon Europe project WeatherGenerator (grant agreement 101187947). It rests on openly shared data: the AlphaEarth Foundations Satellite Embedding dataset (Google, Google DeepMind), Tessera, Copernicus-FM, OlmoEarth, MMEarth, the MajorTOM grid, and the reference products listed in the paper.