RxnExBench: Benchmarking Vision-Language Models for Structured Chemical Reaction Extraction from Scientific Literature
Summary
- Researchers introduced RxnExBench, a multimodal benchmark designed to evaluate vision-language models on structured chemical reaction extraction from scientific literature.
- The dataset contains 3,565 expert-annotated reactions and 2,361 molecular structures sourced from high-impact chemistry journals.
- Evaluation of nine state-of-the-art vision-language models indicates that performance drops significantly as schema complexity and reasoning requirements increase.
- Results using the Oracle-MolRec protocol show that accurate molecular recognition alone is insufficient, highlighting multimodal integration and chemistry-aware reasoning as primary bottlenecks.
Why it matters
RxnExBench provides a rigorous evaluation framework for assessing vision-language models on complex chemical literature tasks, highlighting current limitations in multimodal integration and chemistry-aware reasoning.