SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints

原文摘要 · 中英对照

Amazon SageMaker Inference now supports GPU partitioning with NVIDIA Multi-Instance GPU (MIG), which divides one GPU into hardware-isolated partitions so you can run a different model in each or take a right-sized fraction of a newer accelerator. This post shows how two optional API fields place one model per partition, and what we measured against a whole older GPU.

Amazon SageMaker Inference现在支持使用NVIDIA多实例GPU（MIG）进行GPU分区，它将一块GPU划分为硬件隔离的分区，因此你可以在每个分区中运行不同的模型，或取用较新加速器合适比例的一部分。本文展示了两个可选API字段如何实现每个分区放置一个模型，以及我们与整块旧GPU对比的实测结果。