Post-Training Large Language Models with LLMON

Impact 8 arXiv cs.AI

Summary

  • Researchers introduce LLMON (LLM Object Notation), a post-training framework that separates executable instructions from passive data to prevent prompt injection vulnerabilities.
  • The approach uses a 4.6B token training corpus with distractor-infused variants, applied to two Granite-4.1 base models (3B and 8B) via supervised fine-tuning and reinforcement learning.
  • LLMON reduces indirect prompt injection attack success rates from 53-100% down to 0.02-5.6% without sacrificing general model capabilities.
  • The method uses standard post-training without requiring architectural modifications and introduces attention-based interpretability metrics to diagnose layer failure modes.

Why it matters

LLMON offers a practical post-training defense against prompt injection and execution ambiguity in LLMs without altering core model architectures.

Who should care