Since 2019, the semiconductor industry has been gradually moving to a new design philosophy for chips: chiplets. On the surface, it might seem like this is a fairly minor change, because all that's really happening is that a chip is being split up into smaller pieces. Plus, not every company is doing it, and even the companies that are doing it aren't moving completely towards chiplets. Are chiplets really all that important?
Well, despite all that, chiplets are going to be very important for the semiconductor industry. Not only is it a counter to the recent woes many companies have been experiencing, but now that AMD and Intel are seeing great success with chiplets, their rivals will definitely have to consider following their lead in order to not be disadvantaged.
Chiplets and the two companies using them to revolutionize processors
A chiplet is exactly what it sounds like: a little chip with just partial functionality. The whole idea with chiplets is that instead of making a processor on one single piece of silicon (also called monolithic), you have multiple chips that each contain some part of the CPU. While it's natural to have one chiplet for every function (like one for cores, one for connectivity, one for graphics, etc.) it's also sometimes desirable to put many of the same chips into one processor, which is good for adding more cores for example.
AMD was the company that really coined (or at least popularized) and introduced the concept of chiplets. It had a basic multi-chip module design with its original Zen processors in 2017, of which the higher-end models utilized multiple CPU chips to increase core count, rather than using a single bigger chip. But with Zen 2 in 2019, AMD split its CPUs in half: a chiplet for CPU cores and another for everything else, like PCIe lanes and RAM connectors.
Meanwhile, Intel has been trying to catch up with its own implementation of chiplets, which the company calls tiles. Despite getting into the game much later than AMD, its first chiplet processors have finally hit the scene this year, and they're pretty complex. The Ponte Vecchio datacenter GPU has several tiles full of GPU cores, a couple of tiles for cache, a tile for HBM2 VRAM, and two more tiles for connectivity. Meteor Lake is a four-tile solution for the mainstream, and while it's laptop-exclusive, its successor Arrow Lake will hit the desktop next year, and it's fairly similar.
Other companies such as Fujitsu and Broadcom have signalled their intent to make processors with chiplets, but AMD and Intel are the only ones thus far launching chiplet-based products and producing them in volume. However, particularly for the higher-end of computing, it seems moving to chiplets will become necessary to keep in the game.
Chiplets allow for smarter processor design
What's so fascinating about chiplets is how varied their usefulness is. Chiplets aren't a one-trick pony like ray tracing, nor are they something extremely vague or non-specific like AI. Chiplets have clear and distinct advantages that in many cases make monolithic processors totally obsolete.
One of the things AMD and Intel talk about often with chiplets is how it makes it easier to offer more specific solutions for certain markets and customers. It's very simple to increase and decrease core counts, or to swap out one chiplet for another that's more suitable. For instance, AMD's server CPUs not only have more CPU chiplets than the desktop models, but also a bigger and better IO die (for connectivity stuff). AMD can also add another layer with its 3D V-Cache chiplets for both consumer and server processors, further offering more options to buyers.
You may also consider that reusing chiplets across generations is possible as long as they're still good enough, and this is a key strength in Intel's tile system. While AMD has CPU core and IO chiplets (plus the cache chiplets), Intel's tiles include one for cores, one for graphics, one for SOC functions, and one for IO functions. While this is useful for offering multiple versions of these tiles, Intel's approach allows the company to not replace tiles until it's absolutely necessary since it has more functions spread across more tiles. For example, if Intel wants to update its AI hardware, it just needs to replace the SOC tile.
Although keeping old tiles around for longer is about saving money, it also makes it easier to justify adding new features more incrementally than before. We've been used to generational upgrades every year or two and getting a whole load of things all at once; chiplets can speed up the upgrade cycle quite significantly.
Chiplets also improve manufacturing too
Those are just design considerations though, we haven't even gotten to manufacturing, which is much cheaper with chiplets. This is because of how defects occur in the production of processors, and the short story is that bigger chips are more prone to defects, which decreases production. By the same token, smaller chips are much less prone to defects, to the point that chiplets actually save quite a bit on manufacturing. It's even more pronounced of an effect in brand-new process nodes with high defect rates, which make big chips nigh impossible to be commercially viable.
But perhaps the biggest deal for chiplets when it comes to manufacturing is Moore's Law, which is a prediction that the transistor count in a chip should double every two years. What this actually means in the real world is kind of nebulous, but it's very applicable in high-end computing for the kinds of processors that break the record of having the most transistors. If Moore's Law is still correct as it has been for 50 years, in two years from now we should see a chip that has double the transistors of the biggest chip today.
It's very hotly debated between companies and analysts whether Moore's Law is dead, but it's unquestionable that it's getting harder to improve process nodes, the things that largely facilitate Moore's Law by increasing transistor density. While increasing transistor count can also be achieved by making physically larger processors, there's a practical limit to how large chips can be, and we've already hit it. So, when TSMC's 3nm failed to increase the density for cache by even 1%, that was pretty bad news for the industry and signaled that Moore's Law is dying, or even dead.
Chiplets can't increase density, but they can get around the size limitation since no single chip is even close to the limit. Generally speaking, ~750mm2 is the absolute largest a chip can be on the latest processes, but with chiplets the limit is practically the size of the PCB. AMD's latest Zen 4 Genoa server CPUs are up to 1,271mm2 for models with the full 96 cores.
And speaking of that whole cache problem (which could become industry-wide), chiplets also mitigate that too. Using cache chiplets instead of adding more cache to CPU or graphics chiplets of course has the benefit of making those core chiplets smaller and cheaper, plus the whole specialization angle with things like 3D V-Cache, but there's also a third benefit for manufacturing. If newer nodes aren't going to really improve cache density, then cache chiplets can be made on much older and cheaper nodes without actually losing much if any performance.
Chiplets aren't for everyone, but they will be a big deal
While chiplets are great, they probably won't be very applicable for certain cases, like for really tiny processors like smartphone chipsets or intentionally simple chips for microwaves and home assistants (at least the lower-end ones). The whole semiconductor world will not be built on chiplets; many generic but important chips are still made on some of the oldest and cheapest processes available.
But for our laptops, desktops, servers, cars, and consoles, it's looking more and more likely that chiplets are the future. Of course, the great counterpoint to chiplets from companies like Nvidia is that it's a waste of time because AI is so much better, to the point where doubling performance every two years is slow. Thus, Nvidia still makes its GPUs (and now CPUs) the old-fashioned way, because Moore's Law just doesn't even matter anymore.
Still, in the here and now, AI isn't fully developed as a technology, and if it flops or if the space becomes extremely competitive, then companies that use chiplets will be advantaged against those that do not. Whatever the future has in store for the semiconductor industry, it's hard to imagine that chiplet technology won't be a part of it.