Polysemanticity
term_id: polysemanticity
Category: basic_concepts
Definition
Polysemanticity is a characteristic observed in deep neural networks, particularly in transformers, where a single neuron may activate in response to several unrelated or semantically distinct features. This contrasts with monosemantic neurons, which respond to only one specific concept. Understanding polysemanticity is crucial for interpretability research, as it complicates efforts to map specific network components to human-understandable concepts, necessitating advanced techniques like sparse autoencoders for disentanglement.
Summary
The phenomenon where individual neurons in neural networks respond to multiple distinct concepts.
Key Concepts
- Neuron Interpretability
- Sparse Autoencoders
- Feature Disentanglement
- Transformer Architecture
Use Cases
- Mechanistic interpretability research
- Debugging model behavior
- Improving model transparency and safety