Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips
Summary
- Exa optimized embedding inference on AWS Inferentia2 and Trainium2 hardware using custom NKI kernels.
- The optimization achieved a 47% gain in performance and cost per million documents.
- Exa's neural search engine relies on an in-house vector database to index billions of web documents.
Why it matters
Optimizing embedding inference on custom hardware significantly improves the unit economics of indexing billions of web documents for neural search engines.