Summary
- Goodfire researchers introduced an unsupervised method to discover recurring "neuron communities" in language models by combining gradient-based attribution, neuron graph construction, and graph community detection.
- Tested on models like Llama-3.1-8B and Qwen 3.6 35B-A3B using millions of prompts, small groups of neurons (often fewer than ten) were found to act as computational building blocks supporting specific behaviors.
- Causal interventions demonstrated that these communities are both necessary (ablation alters predictions) and sufficient (retaining them preserves predictions when other MLP neurons are removed).
Why it matters
Identifying functional neuron communities offers an intermediate level of abstraction between individual neurons and higher-order computations, advancing mechanistic interpretability in large language models.