MedConstBench: Benchmarking How Constitutions Steer LLMs in Medical Ethics
Summary
- Researchers introduce MedConstBench, a benchmark designed to evaluate how deployment constitutions steer large language models in medical ethics.
- The benchmark incorporates MedDilemma, a dataset of clinical dilemmas structured around Beauchamp's four principles, and uses human-calibrated multi-model peer review.
- Evaluating ten open-source LLMs under four constitutions shows that constitutional effects are dimension-selective and model-dependent without a universally optimal choice.
- The authors propose a do-no-harm adaptive selector that matches per-model oracles, guarantees a worst-case floor, and generalizes to unseen models.
Why it matters
Understanding how deployment constitutions influence medical ethics behavior helps practitioners safely govern LLMs in sensitive healthcare applications where competing ethical values must be balanced.