RxnExBench: Benchmarking Vision-Language Models for Structured Chemical Reaction Extraction from Scientific Literature

Impact 6 arXiv cs.AI

Summary

  • Researchers introduced RxnExBench, a multimodal benchmark designed to evaluate vision-language models on structured chemical reaction extraction from scientific literature.
  • The dataset contains 3,565 expert-annotated reactions and 2,361 molecular structures sourced from high-impact chemistry journals.
  • Evaluation of nine state-of-the-art vision-language models indicates that performance drops significantly as schema complexity and reasoning requirements increase.
  • Results using the Oracle-MolRec protocol show that accurate molecular recognition alone is insufficient, highlighting multimodal integration and chemistry-aware reasoning as primary bottlenecks.

Why it matters

RxnExBench provides a rigorous evaluation framework for assessing vision-language models on complex chemical literature tasks, highlighting current limitations in multimodal integration and chemistry-aware reasoning.

Who should care