Intel LLM-Scaler vllm-0.14.0-b8.2 Released With Official Arc Pro B70 Support
As part of Intel's LLM-Scaler initiative for AI inferencing on Intel Arc hardware, out today is their vllm-0.14.0-b8.2 update that includes officially supporting the Arc Pro B70 graphics card.
Intel LLM-Scaler provides the Docker-ized approach for deploying large language models on Intel Arc hardware with a particular focus on latest-generation Battlemage graphics hardware, including multi-GPU configurations as part of the Project Battlematrix initiative that began last year.
With today's update for their LLM-Scaler stack with vLLM, they have updated the platform image to intel/llm-scaler-platform:26.18.8.2. The only other listed change is now officially supporting the Intel Arc Pro B70 GPU, the BMG-G31 graphics card that recently debuted with 32GB of vRAM and a sub-$1000 price point.
The Intel Arc Pro B70 continues running well in tests at Phoronix and I will have out some more benchmarks soon.
The new release is tagged on GitHub and also available via Docker Hub. Though since the release announcement notification, they've dropped mentioning the highlights and the Arc Pro B70 support, presumably some release process snafu.
Intel LLM-Scaler provides the Docker-ized approach for deploying large language models on Intel Arc hardware with a particular focus on latest-generation Battlemage graphics hardware, including multi-GPU configurations as part of the Project Battlematrix initiative that began last year.
With today's update for their LLM-Scaler stack with vLLM, they have updated the platform image to intel/llm-scaler-platform:26.18.8.2. The only other listed change is now officially supporting the Intel Arc Pro B70 GPU, the BMG-G31 graphics card that recently debuted with 32GB of vRAM and a sub-$1000 price point.
The Intel Arc Pro B70 continues running well in tests at Phoronix and I will have out some more benchmarks soon.
The new release is tagged on GitHub and also available via Docker Hub. Though since the release announcement notification, they've dropped mentioning the highlights and the Arc Pro B70 support, presumably some release process snafu.