Causal Attribution & Patching (Part 2): Pathscopes

Recap: From Part 1 to Part 2

In Part 1, we established causal patching as a core technique for mechanistic interpretability. By intervening on a model’s internal activations—running it on a "clean" input and patching those activations into a forward pass on a "corrupted" input—we can identify which components are sufficient to restore a desired behavior. This moves us from mere correlation to causal evidence, allowing us to ask not just what information a component encodes, but what mechanism it implements.

However, this approach faces two major limitations. First, our units of analysis—attention heads, MLP layers, or individual neurons—are often polysemantic, meaning they represent multiple, unrelated concepts simultaneously. This makes their role in a circuit hard to pin down. Second, the output of a patching experiment is typically a numeric score (e.g., a change in loss or logit difference), which tells us that a component matters but not what it represents.

This brings us to Part 2. We will explore two major advances that address these challenges. First, we will move from coarse, polysemantic components to fine-grained, interpretable "features" identified by Sparse Autoencoders (SAEs). Second, we will move beyond simple numeric probes to generating rich, natural-language (NL) descriptions of what a hidden state represents. The goal is to build a more precise, interpretable, and editable understanding of the causal mechanisms inside language models.

Background: Patching & Its Fast Approximations

At the heart of our analysis is Activation Patching (also known as causal tracing or interchange interventions). It provides a direct, causal measure of a component's importance. We take a clean prompt (e.g., "The Eiffel Tower is in Paris") and a corrupted prompt (e.g., "The Eiffel Tower is in Rome"), then run the model on the corrupted prompt while replacing a specific internal state, like an attention head's output, with the corresponding state from the clean run. If this single patch restores the model's correct output ("Paris"), we have strong evidence that the component is causally sufficient for that behavior.

While powerful, activation patching is computationally expensive. It requires a separate forward pass for every component we want to test. For a large model with thousands of attention heads and MLP layers across dozens of layers, this quickly becomes intractable.

This scalability problem motivated the development of Attribution Patching (AtP). AtP reframes the patching intervention through the lens of a first-order Taylor-series approximation. Instead of performing a full forward pass for each patch, it estimates the effect using gradients.

Let $f_\theta$ be our model, and let $m$ be a scalar metric we care about, such as the logit of the target token $t^\star$. Let $s_{\text{clean}}$ be the activation of a component on a clean input and $s_{\text{corr}}$ be the activation on a corrupted input. Activation patching measures the exact change in the metric, $\Delta m$. Attribution Patching approximates this change:

$\Delta m \approx \nabla_{s} m \big|{s{\text{corr}}}^\top \, (s_{\text{clean}} - s_{\text{corr}})$

This calculation requires only one forward pass on the clean input (to get $s_{\text{clean}}$), one forward pass on the corrupted input (to get $s_{\text{corr}}$), and one backward pass from the metric on the corrupted run (to get the gradient $\nabla_{s} m$). With these three passes, we can estimate the effect of patching every component in the model simultaneously, offering a massive speedup.

However, this is an approximation. The paper introducing AtP* identified two key failure modes where this linearization breaks down:

Caveat
AtP and AtP* are powerful for rapidly identifying candidate components for a circuit. However, they are not foolproof. Their reliability depends on how well the model's local behavior can be approximated by a linear function. When gradients are small or zero, these methods can miss crucial components. It is best practice to use AtP/AtP* for an initial, wide search and then verify the top-scoring candidates with exact activation patching.

Patchscopes: A Unifying Framework for Inspecting Representations

While AtP helps us find where causality flows, it doesn't tell us what information is flowing. This is where Patchscopes comes in. Introduced by Ghandeharioun et al. (ICML 2024), Patchscopes is a framework that uses the language model itself to explain its internal representations in natural language.

The core idea is simple but profound: "patch" a hidden state from a source computation into a target computation that is designed to elicit a natural language description. For example, to understand what information is encoded at the final token of the prompt "The Colosseum is in the city of," we can take the residual stream vector at that position and patch it into a templated prompt like "The city is [MASK]." The model's subsequent prediction for "[MASK]" ("Rome") reveals the information stored in that vector.

Patchscopes unifies and extends several prior interpretability techniques:

• Unifying Framework: The paper shows that methods like Logit Lens (projecting hidden states to the vocabulary) and Tuned Lens are special cases of the Patchscopes framework. This provides a common ground for understanding and comparing these methods.
• Mitigating Early-Layer Gaps: Prior methods often failed to produce coherent explanations for representations in early layers of a network, as these states are not yet "aligned" with the final vocabulary space. Patchscopes can overcome this by patching a state into the same layer of a different computation, allowing the model's own later layers to process and "translate" the representation effectively.
• Cross-Model Explanations: A fascinating capability introduced by Patchscopes is using a more powerful model (e.g., GPT-4) to explain the internal representations of a smaller, weaker model (e.g., GPT-2). This leverages the advanced reasoning and linguistic capabilities of the larger model to interpret the more primitive representations of the smaller one.
• Multi-Hop Error Correction: Patchscopes can be used not just for inspection but for intervention. In multi-hop reasoning tasks where a model makes an intermediate error, one can patch in a corrected hidden state to steer the model back toward the correct final answer, demonstrating a direct application of the insights gained.

Intuition
Think of Patchscopes as providing human-readable labels for the intermediate variables in a model's computation. Instead of seeing a raw vector and a numeric attribution score, you get a natural language phrase like "the city of Rome" or "a plural subject." This provides a powerful conceptual scaffold that can guide further investigation. The patch queries themselves become testable hypotheses about what a given representation means and does.

Sparse Feature Circuits: Causally Implicated Feature Graphs

Patchscopes gives us better descriptions, but we are often still working with coarse components like the full residual stream or attention head outputs. The problem of polysemanticity remains. Sparse Feature Circuits (SFCs), introduced by Marks et al. (ICLR 2025), tackle this head-on by changing the fundamental unit of analysis from neurons or heads to features.

The approach relies on Sparse Autoencoders (SAEs). An SAE is a small neural network trained to reconstruct a model's internal activations (e.g., an MLP layer's output) using a highly sparse, learned dictionary of features. The goal is to find a basis where each dimension corresponds to a more monosemantic, or conceptually singular, concept. For example, instead of a neuron that activates for both Python code and English lists, an SAE might learn separate features for "Python [ token" and "start of an English list."

With SAEs in hand, SFCs build a causal graph of the model's behavior in three main steps:

This feature-level view enables a powerful new editing capability called Shift. The SFC paper demonstrates this on a classifier trained to determine a person's profession from a biography. When the training data has a spurious correlation (e.g., doctors are always male, nurses are always female), the model learns to rely on gender-associated features. With an SFC, these gender features can be identified and manually ablated from the circuit. The result is a model that no longer relies on the spurious feature and generalizes better to a balanced test set. This is a significant step beyond simple fact-editing, as it alters a model's underlying reasoning strategy.

Caveat
The quality and interpretability of a Sparse Feature Circuit are fundamentally dependent on the quality of the underlying SAEs. Training SAEs is an active area of research, and challenges remain in ensuring features are consistently monosemantic and that the SAE reconstruction is faithful to the original model's activations. The choice of approximation for indirect effects (linear vs. IG) also involves trade-offs between speed and accuracy, particularly in early layers where non-linearities can be more pronounced.

Synthesis: From Components to Features, from Numbers to Language

Patchscopes and Sparse Feature Circuits are not competing methods; they are complementary tools for dissecting a model's internal algorithm. Patchscopes offers a top-down, exploratory lens, while SFCs provide a bottom-up, mechanistic one.

• Patchscopes asks: "What does this representation mean in natural language?" It frames our understanding.
• SFCs ask: "Which specific features in this representation causally drive the output?" It quantifies the mechanism.

When used together, they create a powerful loop: propose a hypothesis about a mechanism, use Patchscopes to find representations that seem to correspond to concepts in that hypothesis, then use SFCs to validate the causal role of the features within those representations and assemble the full circuit.

When to choose which tool:

• Exploration and Hypothesis Generation: Start with Patchscopes. Its natural language outputs are ideal for initial exploration and for communicating findings to a broader audience.
• Scalable Triage: Use AtP/AtP* to quickly scan thousands of components (or features) to find the most influential ones for a specific behavior. This narrows the search space dramatically.
• Mechanism Discovery and Editing: Once you have candidate components, use SFCs to decompose them into interpretable features and map their causal pathways. For editing, the Shift technique is the tool of choice for removing reliance on spurious features. For verifying a key feature's role, a targeted activation patch on that feature is the gold standard.

A Practical Protocol for Modern Causal Patching

Here is a conceptual workflow for conducting a causal analysis project that integrates these modern tools:

The most effective and robust workflow today combines these techniques in a layered approach: Patchscopes for high-level conceptual framing, AtP* for rapid, scalable triage, and Sparse Feature Circuits for deep mechanistic analysis and editing. Each tool shores up the weaknesses of the others. Patchscopes makes the numeric outputs of AtP interpretable, AtP narrows the immense search space for SFCs, and SFCs provide the granular, editable substrate that fully realizes the promise of causal interventions. The gold standard for confirming a critical finding remains the targeted, exact activation patch on the component or feature in question.

Intuition
Many local behaviors in transformers linearize surprisingly well, which is why AtP/AtP* are so effective as a ranking tool. The complexity of the overall function emerges from the composition of many simpler, near-linear steps. The feature lens provided by SAEs is transformative because it refactors the model’s internal ontology into one that is more aligned with human concepts. This turns the question from "What did patching this opaque vector do?" into "What was the causal effect of the 'plural noun' feature?"

Limits & Open Questions

This frontier of research is moving fast, but significant challenges remain:

• Faithfulness of Approximations: How can we establish tighter theoretical bounds on when linear approximations like AtP* and IE estimates are reliable?
• SAE Quality and Scale: Training high-quality, perfectly monosemantic SAEs for all components in a frontier model is a massive engineering and research challenge. Standardization and best practices are still emerging.
• Early-Layer Fidelity: While Patchscopes and better IE approximations have improved our view of early layers, these representations remain harder to interpret and analyze than those in later layers.
• Porting Circuits: How well do feature circuits generalize across model sizes and families? Discovering universal circuits or principles of their construction is a key open goal.
• Dataset-Level Corroboration: Current methods focus on specific input pairs. Bridging these instance-level causal findings to dataset-level statistics and phenomena (e.g., connecting a feature circuit to a specific pre-training data distribution) is a crucial next step for building a complete picture of model behavior.

References

• Ghandeharioun, A., Caciularu, A., Pearce, A., Dixon, L., Geva, M., et al. (2024). Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models. In Proceedings of the 41st International Conference on Machine Learning (ICML). PMLR.
• Marks, S., Rager, C., Michaud, E. J., Belinkov, Y., Bau, D., & Mueller, A. (2024). Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models. arXiv preprint arXiv:2405.00172. To appear at ICLR 2025.
• Syed, F., et al. (2023). Attribution Patching. In Proceedings of the 7th BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP.
• Kramár, J., Lieberum, T., Nanda, N., & Shah, R. (2024). AtP*: Efficient and scalable methods for localizing LLM behaviour to components. arXiv preprint arXiv:2403.00745. to components. arXiv preprint arXiv:2403.00745.