MARQ: A Modular, LLM-Agnostic Requirement Quality Assessor Infused with Systems Engineering Knowledge
Summary
- Researchers introduced MARQ, a modular, LLM-agnostic prototype that infuses large language models with expert-derived systems engineering knowledge for requirement quality assessment.
- MARQ operationalizes INCOSE requirement quality criteria using a criterion-specific knowledge stack and is evaluated against off-the-shelf baselines using a NASA requirement set.
- The study tests Claude Opus 5, GPT-5.6 Sol, and Gemma 3 12B over 100 executions per model, showing median F1 score improvements ranging from model to model.
- Claude Opus 5 achieved a median F1 score of 0.80, GPT-5.6 Sol reached 0.62, and Gemma 3 12B reached 0.53, compared to baseline F1 scores of 0.55, 0.52, and 0.18 respectively.
- Gains are concentrated in Correctness and Conformance criteria, though performance impacts and trade-offs between misses and false alarms remain model-dependent.
Why it matters
Infusing structured domain knowledge into LLMs significantly enhances their defect detection capabilities for requirements engineering, offering a reliable pathway for trustworthy automated evaluation.