Experiments have shown that offloading Engram to DRAM results in up to 50% better performance on H200, B200, B300, and even GB300 NVL72. Our technical team has collaborated with 10x engineers from @EmadBarsoumPi & @AnushElangovan to upstream support for @vllm_project ROCm Engram DRAM offloading too, and we are already seeing up to 50% better perf too in PR 57491. As such, in PRs 985, 1002, 1005, 1006 @NVIDIA, @AIatAMD and @SemiAnalysis_ are updating the default recommended recipes to use Engram offloading.
Engram 卸载到 DRAM 在 H200、B200、B300 及 GB300 NVL72 上最高提升 50% 性能
37
AI 编辑部评分,满分 100Engram 卸载到 DRAM 在 H200、B200、B300 及 GB300 NVL72 上最高提升 50% 性能
来源:SemiAnalysis· x.com