Native Geometry vs. Linear Recoverability: How Alignment Changes Platonic Scaling
Summary
- Researchers examine the Platonic Representation Hypothesis to see if larger models learn increasingly similar representations.
- Using matched Legacy Survey and Hyper Suprime-Cam embeddings across five model families, the study evaluates native cosine geometry versus linear recoverability.
- Results show that Ridge alignment strengthens cross-dataset neighbourhood correspondence as model size increases, outperforming native cosine geometry.
- The learned maps remain far from rotations or uniform scaling, and sparse features offer no distinct advantage over dense embeddings.
Why it matters
Understanding how model scale and alignment affect internal representations helps researchers clarify whether neural networks converge toward shared geometric structures.