AMD Instinct MI350P: PCIe Add-In Card For High Performance Open-Source AI/Compute
While there is the AMD Instinct MI400 series coming this year, today AMD announced an interesting and arguably overdue offering for the Instinct MI350 series: the MI350P. The AMD Instinct MI350P is a PCIe add-in-card to add Instinct MI350 compute capabilities to existing PCIe 5.0 air-cooled servers as an alternative to the Open Accelerator Module (OAM) currently used by the Instinct MI350 series.
👁 AMD Instinct MI350P PCIe card features
Similar to the earlier AMD Instinct products available in PCIe card form factor, the Instinct Mi350P is finally an option for those wanting to add in up to eight Instinct MI350 class PCIe cards to server platforms not equipped to handle OAM modules. This makes it easy to handle upgrading existing PCIe server platforms with new AI/compute capabilities while leveraging the common capabilities of the MI350X/MI355X accelerators.
The AMD Instinct MI350P measures in at 10.5-inches and uses an FHFL dual-slot design. It's passively air-cooled via typical air cooled server platforms and rated for 600 Watts. For those concerned about power budgets, power capping is permitted to drop the Instinct MI350P down to 450 Watts. The AMD Instinct MI350P offers 128 compute units and using the CDNA4 architecture like the rest of the Instinct MI350 series. There is 144GB of HBM3E memory with the Instinct MI350P.
👁 AMD Instinct MI350P specifications
AMD is promoting the Instinct MI350P as capable of handling around 200~250 billion parameter large language models per GPU and support for up to eight GPUs per node. Intended use-cases cover SLM / MLM / LLM inference / RAG workloads. The Instinct MI350P uses all the common ROCm stack inline with the other Instinct and Radeon products.
👁 AMD enterprise AI software stack
For those wondering, no list pricing was provided with the pre-launch assets. It's great seeing AMD introduce an MI350 class PCIe card product but a bit unfortunate it's taken so long to come to market with the Instinct MI400 series coming this year. This Instinct MI350P launch comes ahead of Intel expected to formally launch their Crescent Island AI accelerator later this year in PCIe card form as well. For Intel Crescent Island their design is using 160GB of LPDDR5X memory as previously reported compared to the Instinct MI350P having 144GB but using HBM3E high bandwidth memory.
👁 AMD Instinct MI350P for enterprise AI
Interesting times ahead but that's all for now with no hardware for testing at the moment nor any other interesting Linux details to share at this time.
If you enjoyed this article consider joining Phoronix Premium to view this site ad-free, multi-page articles on a single page, and other benefits. PayPal or Stripe tips are also graciously accepted. Thanks for your support.