With most next-gen processors including dedicated neural processing units (NPUs), AI is well on its way to becoming mainstream. However, you don’t need cutting-edge AI PCs to utilize the newest artificial intelligence facilities; even a tiny SBC can provide enough horsepower to run the latest LLM models and image generation tools.

In fact, there are quite a lot of SBCs that can pull their own weight in AI tasks. So, we’ve compiled a list of the best SBCs for anything and everything related to AI to help you pick the ideal board for your next project.

Our favorite SBCs for AI workloads in 2024

Team Green has always been a key player in the artificial intelligence landscape, and with the popular GPU manufacturer doubling down on chips centered around high-performance AI computations, it shouldn’t come as a surprise that Team Green designed the best SBC for AI workloads. Priced at nearly $500, the Jetson Orin Nano is more expensive than most SBCs on this list, but you get what you pay for.

Although the Orin Nano’s 6-core ARM CPU is decent on its own, the Ampere GPU is what you’re here for. With a max frequency of 625MHz and a whopping 32 Tensor core count, Nvidia’s pint-sized SBC can provide superb AI processing capabilities. As per Team Green, the Orin Nano can achieve an AI performance metric of 40 TOPS, which is rock solid for a tiny system. You also get a decent port selection for other DIY projects, as the SBC possesses four USB-A 3.2 Gen 2 ports, one RJ45 connection, one DisplayPort socket, and a USB Type-C Upstream-Facing Port.

As for the headers and interfaces, the Orin Nano is armed with the usual 40-pin expansion header, two M.2 Key M interfaces, one M.2 key E socket, two MIPI/CSI ports, and a 12-pin power header. The SBC can be powered via a barrel connector, and Nvidia provides a robust set of tools, APIs, and libraries with Orin Nano-compatible JetPack SDK. If you’re someone looking to kickstart your journey into deep learning but aren’t willing to shell out thousands of dollars on a huge system, the Nvidia Jetson Orin Nano is hands-down the best SBC on the market.

If you’ve been a part of the DIY projects landscape, you may have seen BeagleBoard’s set of sleek white SBCs. The BeagleBone AI-64 is one of the newest boards from the company, and it’s a solid option for those who wish to build AI projects with an SBC. On the processor side, the BeagleBone AI-64 rocks two Arm Cortex-A72 processors, though it's the programmable C7x Digital Signal Processor (DSP) chip that sets the AI-64 apart from most SBCs.

For starters, the C7x has a huge 512-bit SIMD data path on top of a neural network accelerator. Combine these with the SBC’s matrix multiply and vision processing accelerators, and it’s easy to see how it can hit up to 8TOPS. The max 4GB DDR4 memory is a bit of a buzzkill, especially if you’re planning to run complex AI tasks that require at least 16 GB RAM.

The port and interface selection, despite not being as diverse as the Orin Nano or even the Raspberry Pi 5, is still pretty decent. You get two USB 3.0 Type-A ports, two CSI connectors, a (separate) DSI interface, a microSD card slot, an M.2 E-key connection, a mini-DisplayPort socket, an eMMC slot and a 1GbE jack alongside two UART debug pins, which offer solid connectivity for most DIY enthusiasts.

One of the biggest reasons to invest in SBCs is that they won’t blow a hole in your pocket. So, if you’re in the market for a tiny and affordable system that can process AI tasks with relative ease, then you should look no further than the Libra Computer Alta. The Amlogic A311D SoC on the SBC pairs four 2.01GHz Arm Cortex-A73 processors with two slightly slower Arm Cortex-A53 chips and a Mali-G52 MP4 graphics card.

For the artificial intelligence workloads, you get a 5 TOPS AI Accelerator with four Tensor Cores and eight Neuro Cores, which are more than enough for an SBC this inexpensive. Like the BeagleBone AI-64, the Alta’s memory is also capped at 4GB, but it should be more than enough for most SLMs and some of the simpler LLMs.

IO-wise, you’re looking at four USB 3.0 Type-A, an RJ45 socket, a 40-Pin GPIO interface, a UART header for debugging, a microSD card slot, and an HDMI 2.0 socket capable of 4K output. Its Amlogic Video Engine 10 decoder is another useful feature of the SBC, though it’s not something you’ll use a lot in AI tasks. All-in-all, the Libre Computer Alta may not be the fastest device for AI workloads, but it’s easily the most affordable option on this list with its $70 price tag.

The Raspberry Pi series is easily the most talked-about family of boards in the SBC landscape. However, as someone who previously attempted to run LLMs via Ollama on the Raspberry Pi 5, I can confidently state that on its own, the fastest RPi board still has a lot of ground left to cover on the AI front. XDA’s SBC maestro Daniel Allen also arrived at a similar conclusion when he tinkered with some chatbots on the SBC.

Sure, the 2.4GHz quad-core Arm Cortex-A7 processor and VideoCore VII GPU can provide decent performance on most DIY SBC projects, but AI workloads are a different story altogether. However, the tide turns in the RPi5’s favor once you arm the SBC with the unique accessories developed for the SBC. For instance, connecting the Coral Edge AI TPU to the Raspberry Pi 5 can boost its capabilities in machine learning tasks to the next level.

That’s before you include the new official AI accelerator HAT the Raspberry Pi Foundation released for the RPi 5 a few weeks ago. According to the Cambridge-based firm, the M.2 HAT can boost the Raspberry Pi 5’s computational prowess to 13 TOPS, making it the second most powerful SBC on this list! The 8GB of LPDDR4X RAM and the humongous set of IO options, including 4 USB Type-A ports, Ethernet jack, dual micro-HDMI connections, PCIe 2.0 interface, and 40-GPIO pins, among others, also cement its standing as a solid SBC.

Without its “crutches,” the Raspberry Pi 5 would easily fall over in AI tasks. But if you’re willing to invest in extra accessories, then the newest Raspberry Pi board can become an absolute powerhouse at running LLMs.

Despite its superior performance, the Orange Pi family is often overlooked because of the uber-popular Raspberry Pi series. But once you compare the AI processing capabilities of the Orange Pi 5 Pro and the barebones Raspberry Pi 5, the former is the undisputed victor. It’s powered by an 8-core RK3588S chip and an Arm Mali-G610 GPU, and both rank pretty high up on the performance tier.

You also get up to 16GB of LPDDR5 memory, which is the second-highest amount of RAM on this list. What’s more, the Orange Pi 5 Pro possesses an embedded NPU that maxes out at 6 TOPS. It also supports a decent number of operating systems right-out-of-the-box, including Unbuntu, Debian, Android, and even its own Arch and Droid-flavored Orange Pi OS.

On the ports front, the Orange Pi 5 Pro ships with four USB Type-A ports, an audio jack, an Ethernet socket, two full-sized HDMI connections, a MIPI CSI slot, 40 GPIO headers, a micro-SD card slot, and an M.2 M-Key interface. In short, the Orange Pi 5 Pro serves as a solid option for those who want a more powerful version of the Raspberry Pi 5 without investing in additional peripherals.

Most SBCs, be it the premium Nvidia Jetson Orin Nano or the budget Libre Computer Alta, are equipped with Arm chipsets. Unfortunately, most Arm processors can still compete with desktop-grade CPUs on an even footing when it comes to performance. Luckily, the Odroid H3+ is a blazing-fast SBC that forgoes energy-efficient Arm processors for a full-fledged Intel Pentium processor.

To be more specific, the H3+ ships with a quad-core Intel Pentium N6005 CPU that can hit a burst frequency of 3.30GHz. Assuming you have adequate cooling provisions, you can even toggle an Ultra Processing Mode on the H3+ and force the N6005 to operate at max frequency.

Thanks to its 64GB memory capacity, you don’t have to concern yourself with the high RAM requirements of the more complex LLMs. Its storage provisions are just as interesting: instead of featuring a microSD card slot, the H3+ comes with a 4-lane PCIe 3.0 socket for NVMe SSDs alongside two SATA connections, where you can plug additional storage drives.

The rest of the port layout is just as solid, with two 2.5GbE ports, one DisplayPort socket, one HDMI port, four USB Type-A connections, a 24-pin expansion header, and dedicated audio jacks providing ample IO connections for you to plug in your favorite accessories.

If you thought Coral’s USB accelerator for the Raspberry Pi 5 was neat, then you’ll be pleased to know that the Google-owned firm has a robust set of AI tools, including a full-blown SBC! Built to help in the development of machine-learning projects, the Dev Board leverages the onboard Google Edge TPU coprocessor to provide a horsepower of 4 TOPS.

At its core, the Coral Dev Board houses an NXP i.MX 8M SoC alongside the GC7000 Lite integrated graphics processor. Sadly, RAM isn't the Dev Board’s finest aspect, as you’re limited to a maximum of 4GB LPDDR4 memory. The same goes for the device’s ports, as you only get one USB Type-C connection, a USB Type-A 3.0 port, a USB Type-B micro serial console interface, an RJ45 socket, a 40-pin GPIO interface, a full-size HDMI 2.0a port, a 3.5mm audio jack, and dedicated eMMC and SD card slots.

Nevertheless, the Coral Dev Board’s plethora of APIs, pre-trained TensorFlow modules, and rich documentation should more than makeup for its drawbacks.

Khadas is yet another SBC manufacturer that has created some great boards that specialize in AI tasks. One such device is the Khadas VIM4, which packs solid deep-learning capabilities with an affordable price and a bevy of supported operating systems. The Khadas VIM4 has an Amlogic A311D2 SoC powering the SBC, which is an upgraded version of the SoC on the Libre Computer Alta. However, that’s where the similarities end, because the VIM4 supports up to 8GB of LPDDR4X memory.

On the AI side of things, the VIM4’s NPU can only manage to hit 3.2 TOPS. On paper, the VIM4 should be weaker than its predecessor in AI-intensive workloads, since the latter can achieve 5 TOPS with its built-in NPU. However, the larger memory bandwidth, combined with the superior processor and GPU specs, allows the VIM4 to outperform every other Khadas SBC at more complex deep-learning algorithms.

The Khadas VIM4 also rocks a solid collection of ports, including two USB Type-A connections, an M.2 slot, a microSD card, an RJ45 socket, a 40-pin header, two CSI, and one DSI interface. Between its multitude of first-party accessories, inexpensive price, and support for tons of OS, there’s a lot to love about this SBC.

Modern SBCs are more than capable of running AI

Besides the options we’ve mentioned so far, there are plenty of other boards that are worth checking out if you wish to incorporate AI into your projects. For example, the Udoo Bolt lineup houses ultra-fast Ryzen Embedded chips and Vega iGPUs, making it great for AMD lovers. Likewise, the Ultra96-V2 makes for a fine addition to any ML enthusiast’s SBC collection.

But for those who want sheer processing power, then there’s no beating the Nvidia Jetson Orin Nano and its Tensor cores. If budget is an issue, then you can go for the cheaper Libre Computer Alta instead. Then there’s the Raspberry Pi 5, a highly versatile SBC that gets more powerful once you start outfitting it with the proper accessories.