Introducing Amazon SageMaker HyperPod Global Inference Gateway: One endpoint across Regions and clusters

Impact 8 AWS Machine Learning Blog

Summary

  • Amazon SageMaker HyperPod has introduced the Global Inference Gateway, providing a single managed endpoint to route inference requests across HyperPod clusters within a single Region or multiple Regions.
  • The gateway automatically selects the optimal cluster and handles failover.
  • It secures all requests using SigV4 and IAM authentication over AWS PrivateLink.

Why it matters

This feature simplifies distributed AI deployments by providing unified, secure multi-region inference routing and automatic failover for large-scale workloads.

Who should care