Introducing Amazon SageMaker HyperPod Global Inference Gateway: One endpoint across Regions and clusters
Summary
- Amazon SageMaker HyperPod has introduced the Global Inference Gateway, providing a single managed endpoint to route inference requests across HyperPod clusters within a single Region or multiple Regions.
- The gateway automatically selects the optimal cluster and handles failover.
- It secures all requests using SigV4 and IAM authentication over AWS PrivateLink.
Why it matters
This feature simplifies distributed AI deployments by providing unified, secure multi-region inference routing and automatic failover for large-scale workloads.