A Team of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels
Summary
- Researchers from Columbia University and CertiK introduced TensorLean, a Lean-based formal verification framework for GPU kernels at the PTX level.
- TensorLean checks compiled kernels against tensor-level math specifications using Lean, Z3, and cvc5, verifying properties like data-race freedom and correct output.
- Out of 149 evaluated kernel instances across FlashAttention, DeepGEMM, and other projects, 83 were proved correct and 38 bugs were identified.
- The framework successfully caught a data race in FlashAttention-4 that did not appear across 17,760 traditional test runs.
Why it matters
TensorLean provides a scalable formal verification method for complex GPU kernels like FlashAttention, helping AI and infrastructure engineers catch deep concurrency and numerical bugs that standard testing misses.