MemryX MX3 M.2 Module Delivers Nice AI Performance With A Great Software Experience
👁 MemryX MX3 vehicle detection example
When initially setting up the MemryX M.2 module and following their getting started guide for Ubuntu Linux use, it all went smooth and took just a few minutes. On reboot in the kernel dmesg I saw "memx_init: pcie init success" so assumed all was well but didn't end up seeing any /dev/memx* devices exposed or any /sys/memx* nodes either... When reaching out to MemryX, it turns out some M.2 slots can be funky especially around the BAR resource requirements of the MX3-2280-M-4. So the suggestion was to try a different M.2 slot, which I did and sure enough then the software stack worked out fine. This was my main critique thus far throughout all of my MemryX MX3 testing was just the initial setup confusion: seeing the MemryX kernel driver report "init[ialization] success" but seemingly not so successful and without any other messaging from the driver about failures or recommending an alternative M.2 slot. I relayed my recommendations to MemryX that having a bit more verbose kernel messages would be helpful for that initial experience or better outlining it in the getting started guide around possible M.2 slot compatibility issues and seeing the basic message reporting "success" doesn't necessarily mean success.
Once having moved the MemryX MX3-2280-M-4 to another M.2 slot, all went well and no awkward troubles since that point. Most of my testing to date has been on Ubuntu 24.04 LTS and it's been a smooth experience. Via sysfs (/sys/memx0/*) there is temperature reporting for each MX3 chip plus reporting the overall utilization level, version information, and other metrics. There is not any real-time power reporting but MemryX is said to have a new revision coming that will contain an onboard sensor for self-reporting total power consumption. MemryX though has found that it's around 6.5 Watts for the MX3-2280-M-4 under load or around 8 Watts with models like ResNet50-V2 and Yolov6m-640.
Via the MemryX Developer Hub there are Python code examples for leveraging the MX3 as well as documentation on various tools for testing and making use of the MemryX AI chips.
👁 MemryX MX3 Linux benchmarking
Running some benchmarks with the MemryX MX3-2280-M-4 showed nice performance and this 6~8 Watt M.2 module delivering better performance than the AMD Ryzen 9 9950X. More benchmarks and comparing to other NPUs/GPUs hardware will come with enough interest level from Phoronix readers.
👁 MemryX MX3 Linux benchmarking tinyyolov2
👁 MemryX MX3 Linux benchmarking googlenet
👁 MemryX MX3 Linux benchmarking resnet50
For having not heard of MemryX previously and being frustrated by many AI accelerator startups largely being vaporware or hardware that is incredibly expensive and often having half-baked software stacks and sparse documentation, it's been a very pleasant time experimenting with the MemryX MX3 M.2 module. Aside from the initial confusion during the setup process that was ultimately fixed by just plugging the MX3-2280-M-4 into a different M.2 slot, there weren't any other show-stopping issues encountered and a nice software experience too especially for being from a startup I previously hadn't heard of. The Linux support was great, is largely open-source already and the remaining bits will hopefully be opened up in 2025 as noted. There is Microsoft Windows support too.
At $149 USD the MemryX MX3-2280-M-4 is an interesting M.2 module for edge AI uses with smaller sized models. This M.2 module can be useful for desktops without any NPU support currently, assisting in AI software development and testing more non-CPU code paths, and low-power AI edge computing scenarios. It will also be very interesting to see what more comes out of MemryX in the future and their follow-on products with the MX3 chip being able to be interconnected up to 16 chips.
👁 MemryX MX3-2280-M-4 M.2 module AI accelerator
Thanks to MemryX for sending over a MX3-2280-M-4 review sample for this Linux testing on Phoronix. For those wanting to learn more about the MX3 or for links to order the $149 USD M.2 module via WPG Americas, visit MemryX.com.
If you enjoyed this article consider joining Phoronix Premium to view this site ad-free, multi-page articles on a single page, and other benefits. PayPal or Stripe tips are also graciously accepted. Thanks for your support.