Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips

Impact 7 AWS Machine Learning Blog

Summary

  • Exa optimized embedding inference on AWS Inferentia2 and Trainium2 hardware using custom NKI kernels.
  • The optimization achieved a 47% gain in performance and cost per million documents.
  • Exa's neural search engine relies on an in-house vector database to index billions of web documents.

Why it matters

Optimizing embedding inference on custom hardware significantly improves the unit economics of indexing billions of web documents for neural search engines.

Who should care