AscendGYM: A Cost-Aware Kernel Optimization Benchmark for Real-World Workloads on Ascend NPUs

Impact 6 arXiv cs.AI

Summary

  • Researchers introduce AscendGYM, a cost-aware kernel optimization benchmark designed for real-world workloads on Ascend NPUs.
  • The benchmark includes an initial suite of 48 kernels across three difficulty levels, paired with test cases derived from practical LLM training and inference tasks.
  • AscendGYM supports model evaluation and model-harness evaluation, offering a complete execution environment that mimics practical pipelines.
  • A new metric called the Speedup-Token Score (STS) is proposed to quantify performance gains relative to token cost under a budget.

Why it matters

AscendGYM provides a standardized environment and metric to evaluate how well models and custom agents optimize hardware kernels for real-world workloads while considering resource costs.

Who should care