Using NVIDIA Run:ai GPU Fractioning on Amazon EKS

Impact 7 AWS Machine Learning Blog

Summary

  • AWS published a walkthrough on using open-source NVIDIA Run:ai GPU Fractioning on Amazon EKS.
  • The guide demonstrates running two memory-bounded Kubernetes Pods on a single NVIDIA A10G GPU.
  • It covers building a custom Amazon Linux 2023 image, installing the KAI Scheduler and GPU Fractioning Operator, and validating per-Pod GPU-memory contracts.

Why it matters

GPU fractioning enables higher utilization of expensive hardware like the NVIDIA A10G by allowing multiple memory-bounded workloads to share a single device securely on Amazon EKS.

Who should care