Arm Eyes AI with Its Latest Neoverse Cores and Subsystems

Arm last week unveiled the latest addition to its Neoverse family of CPU cores that extend the chip designer’s reach into data centers, the edge, and the cloud and are the latest step in proving its decade-plus-old promise of becoming a player in the server chip market.

The company introduced Neoverse microarchitecture in 2018 and has been building it out since. Neoverse comprises 64-bit Arm processor cores in the E-Series for edge computing, N-Series for core data center jobs, and V-Series for high-performance computing (HPC).

Last year, Arm introduced its Neoverse Compute Subsystems (CSS), which offers chip makers a faster and easier way to create custom Arm-based specialized silicon for an infrastructure sector undergoing rapid changes like the massive amounts of data being created, expansion of the Internet of Things, and the ongoing onslaught of AI.

Unlike Intel, AMD, and other silicon vendors, Arm doesn’t make chips. It designs and licenses systems-on-a-chip (SoCs) that others can use to build their processors. Neoverse CSS is aimed at expanding what Arm can offer chip makers, in this case optimized, integrated, and verified platforms.

New Chips, with Some CSS

What Arm did this week was unveil the V3 CPU architecture, codenamed “Poseidon,” for HPC and the N3 architecture (“Hermes”) for data center — or “balanced” – systems and introduced CSS offerings for each CPU core.

In announcing the new offerings, Arm didn’t get into deep architectural details but did say the Neoverse CSS N3 offers 20 percent higher performance-per-watt than the CSS N2, while the CSS V3 brings 50 percent better performance-per-socket than previous CSS offerings. Both also deliver higher performance for crucial AI workloads, with the N3 showing 196 percent better performance over its predecessor and the V3 and 84 percent improvement with AI data analytics workloads.

In addition, CSS N3, built on Arm v9.2, offers up to 32 cores as low as 40 watts of thermal design power (TDP) and is aimed at applications like telecommunications, networks, data processing units (DPUs), and scale-out cloud configurations.

CSS V3 can scale up to 128 cores per socket and supports the newest high-speed memory and IO standards, according to Arm.

Chip Makers Get More Options

This gives chip makers a broader range of options when designing custom chips, a growing necessity in a rapidly transforming compute space that is seeing companies that once relied on general-purpose CPUs building their own chips. That includes organizations like cloud giant Amazon Web Services (AWS) with Graviton and even Microsoft with Cobalt for general-purpose workloads in the cloud. Nvidia turned to Arm when it developed its Grace Hopper chip.

Dermot O’Driscoll, vice president of product solutions for Arm’s infrastructure line of business, told journalists ahead of the launch that there are two trends driving infrastructure.

“Firstly, there is a desire to optimize compute for key workloads that underpin the cloud,” O’Driscoll said. “Secondly, big companies are building their own custom silicon and need an accessible way to do so. With Arm’s collaborative IP model and the compute subsystems, we are providing them the technical control and the logistical flexibility to do both.”

There are other moves Arm is making with chip-making partners, including finding ways to reuse chiplets, which is difficult when designs from different teams are paired together, he said. Arm recently announced the idea of a chiplet architecture, which is being reviewed by 15 chip makers, adding that “CSS makes it easier for you to build a compute chiplet and we expect many people to need compute chips to sit alongside their AI accelerators and CSS simplifies that co-design experience.”

Some Pluses and Minuses for Developers

Developers have been keeping a close eye on Arm’s push into data center compute and adapting as more chip makers have embraced the microarchitecture and OEMs have brought Arm-based chips into their systems. Bob O’Donnell, principal analyst with TECHnalysis Research, told The New Stack that the “slow and steady march to run Arm-based processors in the data center” has grown the momentum of developers who want to port their applications onto the Arm architecture.

“The thing about porting their apps to Arm is they now have more reason to,” O’Donnell said. “This is a long, slow transition [for organizations to move to Arm-based systems in the data center]. This stuff takes years. Now more companies are using Arm-based CPUs.”

That will only increase as AI technologies mature and more workloads come to the market. Companies are spending a lot of money on AI applications and many developers are anxious to move in that direction, he said.

That said, porting software is never easy, which could make the growing popularity of Neoverse a pain, according to Rob Enderle, principal analyst at The Enderle Group.

“It is an attractive platform for their customers but will result in additional costs to port to this platform for the software developer,” Enderle told The New Stack. “Choices come with costs and those costs come in the form of differences between platforms and the related software development and support costs. So, as it is, Neoverse is an initial negative for software developers because it creates the potential initially for additional support and porting costs.”

Benefits that can come from more platform diversity go to the user or hardware owner, not the developer, he said.

“For instance, right now AI is largely being done — with the exception of inference — on GPUs,” Enderle said. “Moving from GPUs to another AI hardware architecture costs time and money for developers but, again, the performance and cost benefits don’t flow to them.”

In addition, software development is often done on workstations and not servers, but Neoverse and similar technologies are focused on the cloud and not workstations, which also adds to the burden on software developers.

“Now, long term, this could help expand the AI market, which would have positive revenue benefits for the developers,” he said. “But generally, for them, the fewer the platforms, the happier they are.”