Mapping the Latent Space of Llama 3.3 70B

We have trained sparse autoencoders (SAEs) on Llama 3.3 70B and released the interpreted model for general access via an API.

Authors

Thomas McGrath*
Daniel Balsam*
Liv Gorton*
Murat Cubuktepe*
Myra Deng*
Nam Nguyen*
Akshaj Jain*
Thariq Shihipar*
Eric Ho*

Published

Dec. 23, 2024

* Goodfire

We have trained sparse autoencoders (SAEs) on Llama 3.3 70B and released the interpreted model for general access via an API. To our knowledge, this is the most capable openly available model with interpretability tooling. We think that making interpretability tools easily available on a powerful model will enable both new research and new products.

This post explores the feature space of Llama 3.3-70B at an intermediate layer - you can browse an interactive map of features that you can then use in the API, and we also demo the steering effects of some of our favorite features.

We have also introduced a range of new features that make SAE-based steering much easier to use and more reliable. You can learn how to use them in our API docs and experiment with them in our playground. We'll be releasing a research post covering our improvements in steering methodology in the new year.

Feature explorer

Feature map

We used DataMapPlot to create an interactive UMAP [1]Umap: Uniform manifold approximation and projection for dimension reduction [link]

Interestingly, many features related to special formatting tokens or repetitive elements of chat data (such as the knowledge cutoff date) appear as isolated points or small clusters away from the central component. There are two potential explanations for this: special tokens (e.g., beginning of text) often have a very large magnitude, so we might expect that SAE features for them also have a large magnitude and are thus not close to other points. A second explanation is that because many of these features are repeated so frequently (every chat has the knowledge cutoff, for example) the SAE and base model may have memorized them, which has consequences for their representation [2]Superposition, memorization, and double descent [HTML]

Feature examples

Here we show some examples of feature clusters we found interesting - this is by no means exhaustive and there is a lot left to discover in this latent space.

Our SAE has learned a surprisingly broad range of concepts given that it was trained purely on internet chat data. Latents for precise distinctions in terms of types of behaviour appear regularly, including the cluster shown above, although we have not yet verified that they do in fact have distinct effects. In addition to the biomedical knowledge cluster we have also seen multiple physics and programming clusters.

Feature steering

You can also steer the model using SAE latents. Our AutoSteer functionality automatically finds SAE latents to elicit a desired behaviour and sets their weights (you can read more about this in our API). Here we showcase a simpler setting, which is what you get when you call variant.set(feature_id, z). In this case, we simply increase the selected feature's value.

Methods

Our methodology was broadly consistent with our approach to training SAEs on smaller models, which you can read about in our earlier research post [4]Understanding and Steering Llama 3 with Sparse Autoencoders [link]

We also believe there are research use cases where having access to the unmoderated SAEs is valuable. If you're a safety researcher, you can request access by emailing contact@goodfire.ai.

Limitations and areas for improvement

Feature steering and using features as classifiers appear to be in tension: feature steering benefits from narrower, sparser SAEs around the middle of the model, whereas classification tasks are likely to benefit from broader SAEs early or late in the model (depending on the classification task). This could plausibly be resolved with a combination of crosscoders (to capture features at all layers) [6]Sparse Crosscoders for Cross-layer Features and Model Diffing [HTML]

Citation

McGrath, et al., "Mapping the latent space of Llama 3.3 70B", Goodfire, 2024.

References

Umap: Uniform manifold approximation and projection for dimension reduction [link]\n02. Superposition, memorization, and double descent [HTML]\n03. A is for absorption: Studying feature splitting and absorption in sparse autoencoders [PDF]\n04. Understanding and Steering Llama 3 with Sparse Autoencoders [link]\n05. Lmsys-chat-1m: A large-scale real-world llm conversation dataset [PDF]\n06. Sparse Crosscoders for Cross-layer Features and Model Diffing [HTML]\n07. Group Crosscoders for Mechanistic Analysis of Symmetry [PDF]\n08. Matryoshka Sparse Autoencoders [link]\n09. Learning Multi-Level Features with Matryoshka SAEs [link]\n10. Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models\n11. Decomposing The Dark Matter of Sparse Autoencoders [PDF]