TReC: Evaluating Move-Induced State Changes in Instruction-Conditioned Go

Impact 4 arXiv cs.AI

Summary

  • Researchers introduce Transition Representation Consistency (TReC) to evaluate move-induced state changes in instruction-conditioned Go models.
  • TReC computes the cosine similarity between board-state embedding changes resulting from predicted versus reference moves executed from the same position.
  • Experiments using a frozen KataGo network on 680 mid-to-late-game positions show TReC correlates only weakly with traditional coordinate distance.

Why it matters

TReC offers a novel semantic metric for evaluating model move predictions in complex games, moving beyond simple coordinate distances to better capture state representation dynamics.

Who should care