SMT Proves Very Advantageous For AMD Ryzen AI MAX Strix Halo Performance
AI is another big area with Strix Halo thanks to the powerful integrated graphics with unified memory and Ryzen AI NPU. For CPU-based inferencing, SMT also helps deliver even better performance. With PyTorch just across the CPU cores/threads there was better performance thanks to SMT. Though with PyTorch the CPU use was higher when using SMT than without, but on a performance-per-Watt basis it still proved comparable or slightly better.
Llama.cpp on the CPU also benefited from SMT. Power use was up slightly when using SMT but on a performance-per-Watt basis was still beneficial.