AscendGYM: A Cost-Aware Kernel Optimization Benchmark for Real-World Workloads on Ascend NPUs
Summary
- Researchers introduce AscendGYM, a cost-aware kernel optimization benchmark designed for real-world workloads on Ascend NPUs.
- The benchmark includes an initial suite of 48 kernels across three difficulty levels, paired with test cases derived from practical LLM training and inference tasks.
- AscendGYM supports model evaluation and model-harness evaluation, offering a complete execution environment that mimics practical pipelines.
- A new metric called the Speedup-Token Score (STS) is proposed to quantify performance gains relative to token cost under a budget.
Why it matters
AscendGYM provides a standardized environment and metric to evaluate how well models and custom agents optimize hardware kernels for real-world workloads while considering resource costs.