Summary
- Geekbench AI measures both speed and accuracy for AI workloads
- It generates three scores: Single Precision, Half Precision, and Quantized.
- It's available on all major platforms and supports multiple frameworks.
Primate Labs, the team behind the popular Geekbench benchmark, has announced its first-ever AI-focused test: Geekbench AI 1.0. As you might be able to glean from the name, this test is all about dtermining the AI performance of your devices.
It's impossible to have a product launch in 2024 without some kind of mention of AI, whether that's premium laptops, phones, or even a random little gadget you put in your pocket. It's fair to say AI has become something of a buzzword, but this benchmark will allow you to truly compare the AI capabilities of different kinds of hardware.
How Geekbench AI tests your device
Geekbench AI is obviously focused completely on determining the AI capabilities of different hardware, whether that's an NPU, CPU, or GPU. Yes, all those devices can provide some kind of AI processing. Gekbench AI measures performance with three distinct scores: Single Precision, Half Precision, and Quantized. Single precision is what's often referred to as 32-bit precision or INT32. Half precision is 16-bit precision, while the Quantized Score is what's often been referred to as INT8, or 8-bit precision. However, performance scores aren't displayed in TOPS, so don't expect to see the same numbers as what's advertised by companies.
What are AI TOPS? Explaining a largely meaningless term
If you've seen companies marketing based on "AI TOPS", here's what that means and why it's largely meaningless.
The tests include the following workloads and measurements:
|
Test |
Metric |
|---|---|
|
Image Classification |
Top-1 Accuracy |
|
Image Segmentation |
Pixel Accuracy |
|
Object Detection |
F1 Score |
|
Face Detection |
F1 Score |
|
Pose Estimation |
Object Keypoint Similarity |
|
Depth Estimation |
Root Mean Square Error (RMSE) |
|
Super Image Resolution |
Structural Similarity Index Measure (SSIM) |
|
Style Transfer |
Structural Similarity Index Measure (SSIM) |
|
Machine Translation |
BiLingual Evaluation Understudy |
|
Text Classification |
Top-1 Accuracy |
Geekbench AI doesn't just measure the speed of an AI workload, but it also measures the accuracy of the output. That is to say, it takes into account how close the output is to the truth, so you're not just testing speeds at random. For developers, this can help determine whether a larger or smaller data type should be used in a specific isntance, for example, increasing performance in exchange for accuracy.
The first full official version of Geekbench AI also comes with support for all the major AI Frameworks right now. That includes TensorFlow Lite on Android and Linux, CoreML on iOS and macOS, and ONNX and OpenVINO on both Windows and Linux PCs.
If you want to learn more about the AI performance of your PC or phone, you can download Geekbench AI 1.0 starting today. This kind of tool should contribute to greater transparency when it comes to AI performance claims from companies, especially now that we have Copilot+ PCs about to enter the market. You can download Geekbench AI for Windows, Linux, macOS, Android, and iOS.