Modeling LLM Reasoning Dynamics with Structural Representations
Summary
- Researchers introduce T/E/S, combining text, local events, and activity spans to model language model reasoning dynamics.
- Using events and spans improves next-outcome prediction across reasoning and information-seeking tasks for Transformers and Qwen with LoRA.
- Bounding shared continuation mass recovers optimal actions while reducing mean inference CPU time by 77.7% in finite-process experiments.
- A compact structural observer reduces Brier loss from 0.0836 to 0.0756 when predicting whether inference finishes within a CPU allowance on HotpotQA and MuSiQue.
Why it matters
Modeling LLM reasoning dynamics with structural representations helps optimize computational resource allocation and inference time during complex reasoning tasks.