Coming to ISO C++ 26 Standard: An AI Acceleration Edge

If you listen to the experts developing C++ standards, it’s clear that being part of a committee isn’t easy. This was evident at an ISO C++ 26 panel discussion at the recent Supercomputing 2024 conference.

Some proposals can take more than a decade to become an ISO C++ standard. Reaching consensus can be challenging as differing priorities, disagreements and corporate interests derail proposals.

For example, there are three competing proposals on memory safety, but there are disagreements over implementations. As a result, memory safety isn’t a top priority as committee members work on C++ 26.

However, the committee members agreed on one thing: C++ 26 needs to add standards that boost computing speed and parallelism, which is the cornerstone of artificial intelligence.

C++ 26 committee members are pushing concurrency across GPUs, FPGAs and other AI chips. GPUs are vital to mainstream computing, but the goal is to support a wide range of accelerators coming in the future.

They reminded the audience that C++ code is permanent. The installed base is so large that modernizing code to newer languages or tuning it to hardware is becoming wishful thinking.

Here are some features that panelists indicated as most likely to be part of C++ 26 standard, which is ongoing. Work for C++ 29 will begin in early 2026.

Concurrency Is Important

C++ 26 will have an important parallelism feature called senders/receivers, which was characterized as the most important piece of work for the standard, despite being controversial.

Senders/receivers orchestrate parallel execution of code regardless of hardware and can easily port the code to new hardware.

“That allows you to do things like scheduling asynchronous algorithms on thread pools, creating task graphs, scheduling full task graphs onto thread pools or, for example, other types of executors like CUDA streams — so you can actually dispatch a task graph into a GPU,” said Gonzalo Brito, a GPU architect at Nvidia.

Generally speaking, abstraction is not a strong suit of the C++ programming language. Parallelism has become difficult with newer AI accelerators, and it usually helps for coders to point out the separate memory subsystems in CPUs and GPUs for meaningful interaction.

“It’s very exciting that we can just think about writing our parallel codes in the standard language rather than bolting on some other model, which is kind of cool,” said Tom Deakin, a lecturer at the University of Bristol.

That said, it’s just one feature. C++ is built for backward compatibility, and it’s not built to look forward to complex systems architected for AI.

For example, the existing USM (Unified Shared Memory) standard in C++ is firmly focused on thinking of a system as one giant memory module, and that won’t change anytime soon.

“We just see the world as one big memory space, no matter what type of accelerator or processor we’re on,” Deakin said.

Senders/receivers also don’t mesh with all the parallel algorithms that have been in use since 2017, Deakin noted. Senders/receivers were proposed by Nvidia, and the company will benefit, given its GPUs are driving AI development.

Speed Up Execution on SIMD

There’s hope that C++ 26 will include std::simd, a proposal for standard libraries for SIMD (Single Instruction Multiple Data). The P1928 standard is on its way to the committee.

SIMD allows programmers to perform multiple operations in a single instruction. It is mostly associated with vector processors, which have been around for decades dating back to the oldest supercomputers. Now they are in general-purpose processors.

“It’s really kind of amazing that SIMD’s been around for decades now and we still don’t have a standard way to use it, but hopefully we’re getting to finally see a light at the end of the tunnel on that with SIMD,” said James Brodman, principal engineer at Intel.

There is no need to rewrite code for new instruction set architecture. SIMD is on every chip, and vendors have their own versions of SIMD on chips.

“This is a proposal to abstract data parallelism in a nice way and finally get rid of all those compiler-specific or vendor-specific special vector types, or even just using intrinsics directly in your code,” Brodman said.

Reflection

It looks like reflection will make it into ISO C++ 26.

Reflection allows code to examine and understand the structure of classes, functions and other program elements during compilation. The reflection feature generates metadata about the data types and structure. Developers can write code that works with existing structures, reducing the need to manually rewrite code.

“Seems like we’re out to finally get something to the standard that can help us with reflection,” said Bruno Cardoso Lopes, who works on the compiler team at Meta.

Other Features

C++ 26 will have linear algebra as a standard feature, which is significant for AI operations. The linear algebra will include BLAS (Basic Linear Algebra Subprograms) levels 1, 2 and 3, which allows for various types of matrix multiplications and matrix-vector calculations by default.

C++ 2023 had multidimensional arrays, but C++ 26 has already improved on them with “submdspan,” which allows coders to do slices of multidimensional spans.

“Multidimensional spans allow you to customize the layout of your multidimensional tensor data structure so you have column major, row major and so on,” Brito said.

C++ 26 also includes padded layouts so coders can align parts of their matrix to a cache line.

Minor Safety Features

Panelists largely agreed that safety in C++ 26 was important but wasn’t advancing quickly. One proposal called Circle has caught some interest, but there are more important priorities.

“The safety proposals that are going on are pretty big. They are pretty radical changes and are going to require multiyear, dozens of people group efforts to ship. If those people don’t show up, then they just won’t happen,” Brito said.

But some basic security features will be in C++ 26.

“We have a few safety improvements like initialization of automatic variables or infinite loops that don’t perform any steps are no longer undefined behavior,” Brito said.