Decoding the Web of Interactions: Scalable Methods for LLM Interpretability

Introduction

Large Language Models (LLMs) have become central to modern AI, but their inner workings remain opaque. Understanding how these models arrive at decisions is essential for building trust, ensuring safety, and debugging performance. Interpretability research tackles this by analyzing models through multiple lenses, yet a common challenge persists: complexity at scale. LLMs synthesize countless relationships among input features, training data, and internal components, producing behavior that emerges from intricate webs of interactions. Identifying these interactions is computationally daunting because the number of potential combinations grows exponentially. In this article, we explore three core interpretability perspectives and introduce SPEX and ProxySPEX—algorithms that efficiently pinpoint influential interactions even in massive models.

The Three Faces of Interpretability

Interpretability research generally falls into three categories, each shedding light on a different aspect of model behavior.

Feature Attribution

Feature attribution methods isolate the specific input tokens or segments that drive a prediction. By masking parts of the prompt and measuring the change in output, researchers can identify critical words or phrases. Pioneering work by Lundberg & Lee (2017) on SHAP and Ribeiro et al. (2022) on LIME established foundational approaches. These methods assume that removing a feature reveals its contribution, but in practice, features often interact—masking one can amplify or diminish the effect of others.

Data Attribution

Data attribution links model outputs to influential training examples. Koh & Liang (2017) and Ilyas et al. (2022) showed that removing or adding certain data points can drastically alter predictions. This perspective helps understand how training data shapes model knowledge and biases. The challenge lies in the fact that models learn from shared patterns across many examples; attribution must capture not just single examples but synergistic contributions from multiple data points.

Mechanistic Interpretability

Mechanistic interpretability aims to reverse-engineer the internal circuits of neural networks. By intervening on specific neurons, attention heads, or layers, researchers like Conmy et al. (2023) and Sharkey et al. (2025) explain how computations propagate. This white-box approach reveals detailed cause-effect pathways, but the sheer number of components makes exhaustive mapping infeasible.

The Interaction Problem

Across all three lenses, the same obstacle emerges: interactions. Model behavior rarely stems from isolated parts. A prediction may depend on a combination of input features, a cluster of training examples, or a coordinated set of internal components. As the model scales—more features, more data, more parameters—the number of possible interactions skyrockets. For instance, with just 100 features, the number of pairwise interactions is nearly 5,000; triple interactions exceed 160,000. Exhaustive analysis is computationally impossible.

Interpretability methods must therefore be scalable and selective, focusing on the interactions that truly matter. This is where ablation-based approaches shine.

Ablation-Based Attribution

Ablation is a fundamental technique: systematically remove a component (input feature, training data point, or internal unit) and measure the effect on the output. The difference between original and ablated outputs quantifies that component’s influence. Applied to interactions, we can ablate combinations of components to see if their joint effect exceeds the sum of individual effects—a hallmark of interaction.

However, each ablation incurs a cost. In feature attribution, it requires an extra inference call. In data attribution, it may require retraining models on subsets. In mechanistic interpretability, it involves surgical interventions in the forward pass. To make the approach tractable, we need algorithms that minimize the number of ablation trials.

Types of Ablation

• Feature ablation: Mask input segments (e.g., substitute tokens with [MASK]) and observe prediction shift.
• Data ablation: Train models on different data subsets (e.g., leave-one-out) and assess test point behavior.
• Component ablation: Remove the contribution of specific neurons or attention heads by zeroing out activations or masking gradients.

Each type reveals a different interaction landscape, but all share the goal of efficient discovery.

SPEX and ProxySPEX: Scalable Interaction Discovery

To tackle exponential complexity, we developed SPEX (Sparse Probing for EXplanations) and its enhanced variant ProxySPEX. These algorithms leverage the insight that most interactions are weak or irrelevant; only a sparse subset drives behavior. By focusing computational resources on promising candidates, they achieve scalable attribution.

SPEX works by first performing a quick screening pass: it evaluates individual components and low-order interactions to identify a candidate set. Then, using a greedy or optimization-based strategy, it probes higher-order interactions among those candidates. This prunes the search space dramatically. ProxySPEX goes further by incorporating a proxy model—a faster, approximate model that mimics the LLM’s behavior for certain inputs. The proxy model is used to filter many potential interactions before expensive ablations are performed on the full LLM.

Both methods output a ranked list of influential interactions, with interaction strength measured by the drop in output probability when the interacting components are ablated together. This allows researchers to focus on the most impactful dependencies.

Conclusion and Future Directions

Identifying interactions at scale is a critical step toward trustworthy LLMs. Feature attribution, data attribution, and mechanistic interpretability each offer unique insights, but all benefit from scalable interaction detection. SPEX and ProxySPEX provide a practical path forward, balancing computational cost with accuracy. As models grow, future work will likely incorporate causal inference and automated hypothesis testing to further streamline interpretability. The ultimate goal is to make LLMs not just powerful, but also transparent and accountable.