The Neural Geometry Series
The Neural Geometry Series
A series about mapping the inner geometry of neural networks: the multidimensional structures in models' activations, the computations that those structures support, and new methods that let us recover, understand, and control them.
The World Inside Neural Networks How neural geometry will unlock understanding and control of AI Neural networks develop rich geometric structure in their activations, mirroring the structure of the world they are trained on: days of the week form circles, colors form an HSL manifold, and the tree of life appears in genomic representations. This opening post makes the case that this "neural geometry" is a crucial frontier for understanding, improving, and controlling AI models. Geiger et al. · May 7, 2026](/content/research/the-world-inside-neural-networks/index.html)
Steering Along Manifolds to Control Neural Networks Steering along curved manifolds in representation space produces cleaner, more targeted behavior changes than conventional linear steering vectors. Wurgaft et al. · May 7, 2026](/content/research/manifold-steering/index.html)
A Geometric Calculator Inside a Neural Network We found a neural mechanism that operates over manifolds: a general-purpose addition module inside Llama 3.1 8B which manipulates circular representations of numbers. Feucht et al. · May 14, 2026](/content/research/a-geometric-calculator/index.html)
Can SAEs Capture Neural Geometry? Can we use sparse autoencoder features – i.e., straight lines – to reconstruct curved geometry? We study how, and implement an unsupervised pipeline for discovering manifolds using SAE features. Bhalla et al. · May 21, 2026](/content/research/can-saes-capture-neural-geometry/index.html)
Meandering on Manifolds: The Neural Geometry of Stories Over Time To fully understand LLM representations, we must understand how they change dynamically over the course of a prompt or conversation. We investigate these temporal dynamics with a simple case study: how do LLMs represent emotions while reading short stories, both geometrically (in activation space) and temporally (changing from sentence to sentence)? Bigelow et al. · June 23, 2026](/content/research/stories-in-space/index.html)
Uncovering Neural Geometry in Vision Models With Block-Sparse Featurizers We introduce Block-Sparse Featurizers (BSF), a family of methods to decompose a model's activations into multidimensional subspaces rather than single directions. Applied to vision models, we find that BSFs find interpretable, multidimensional features which offer a more parsimonious explanation of model internals; that those features enable fine-grained steering; and that most concepts in the models are multidimensional. Fel et al. · July 7, 2026](/content/research/bsf-vision/index.html)
Research
Uncovering Neural Geometry in Vision Models With Block-Sparse Featurizers
July 7, 2026 Thomas Fel, Matthew Kowal, Mozes Jacobs, Dron Hazra, Usha Bhalla, Lee Sharkey, Lucius Bushnaq, Satchel Grant, Tal Haklay, Thomas Icard, Can Rager, Michael Pearce, Daniel Wurgaft, Aiden Swann, Fenil Doshi, Siddharth Boppana, Curt Tigges, Nick Cammarata, Thomas Serre, Vasudev Shyam, Owen Lewis, Thomas McGrath, Jack Merullo, Ekdeep Singh Lubana, Atticus Geiger
Meandering on Manifolds: The Neural Geometry of Stories Over Time
June 23, 2026 Eric Bigelow, Raphaël Sarfati, Daniel Wurgaft, Owen Lewis, Thomas McGrath, Jack Merullo, Atticus Geiger, Ekdeep Singh Lubana
Predictive Data Debugging: Reveal and Shape What Your Model Learns, Before You Train
June 11, 2026 Leon Bergen, Usha Bhalla, Sidharth Baskaran, Max Loeffler, Raphaël Sarfati, Dhruvil Gala, Ryan Panwar, Santiago Aranguri, Thomas Fel, Atticus Geiger, Matthew Kowal, Siddharth Boppana, Daniel Balsam, Owen Lewis, Jack Merullo, Thomas McGrath, Ekdeep Singh Lubana