Modeling LLM Reasoning Dynamics with Structural Representations

Impact 7 arXiv cs.AI

Summary

  • Researchers introduce T/E/S, combining text, local events, and activity spans to model language model reasoning dynamics.
  • Using events and spans improves next-outcome prediction across reasoning and information-seeking tasks for Transformers and Qwen with LoRA.
  • Bounding shared continuation mass recovers optimal actions while reducing mean inference CPU time by 77.7% in finite-process experiments.
  • A compact structural observer reduces Brier loss from 0.0836 to 0.0756 when predicting whether inference finishes within a CPU allowance on HotpotQA and MuSiQue.

Why it matters

Modeling LLM reasoning dynamics with structural representations helps optimize computational resource allocation and inference time during complex reasoning tasks.

Who should care