TReC: Evaluating Move-Induced State Changes in Instruction-Conditioned Go
Summary
- Researchers introduce Transition Representation Consistency (TReC) to evaluate move-induced state changes in instruction-conditioned Go models.
- TReC computes the cosine similarity between board-state embedding changes resulting from predicted versus reference moves executed from the same position.
- Experiments using a frozen KataGo network on 680 mid-to-late-game positions show TReC correlates only weakly with traditional coordinate distance.
Why it matters
TReC offers a novel semantic metric for evaluating model move predictions in complex games, moving beyond simple coordinate distances to better capture state representation dynamics.