Build your chip like lego.
As semiconductor foundries keep making transistors smaller and smaller, it’s getting much more difficult and expensive to do so, so the cost of making a chip of a given size is increasing drastically.
As well, as the size of a chip increases, the cost of that chip will increase roughly quadratically: this is because when a batch of chips is made on a big flat wafer, there will be small random defects on the surface of said wafer. If that defect is on a chip, the chip will be defective. With smaller chips, you can fit many more on the surface of a wafer, so the cost of losing say 20 out of 264 chips (8%) is much less severe than losing say 16 out of 54 chips (30%).
J. Rabaey et al.
This means that larger chips cost more to make, and you get fewer useful chips you can sell due to defects, so the cost of larger chips increases roughly quadratically with size [1].
So as the cost of a batch of chips has gotten higher and higher with recent generations, the market has responded with making smaller chips and connecting them together like lego: this is the so-called chiplet method. Nowadays many processors are made using chiplets: AMD’s server CPUs and high end gaming CPUs since 2019, and their newest high end GPUs; Intel’s newest server CPUs and GPUs, and upcoming laptop CPUs; and Amazon’s newest server CPU.
But in all these examples, the processor uses only chiplets made by one company. As more and more companies start to use chiplets, there’s an emerging open market for chiplets, where someone who wants to make a custom processor can make some chiplets of their own, and buy some chiplets from others, and connect them together to make their own product. This allows them to complete their processor faster, and have a faster time-to-market, and it’s also often cheaper than trying to do it all themselves.
This chiplet market is still in its infancy, but some startups are targeting this new market, notably Tenstorrent and Ventana. Tenstorrent currently makes machine learning accelerators, and they plan to make their future accelerators using chiplets, and also sell their Ascalon RISC-V CPU chiplet in the mass market.
And Ventana is targeting the chiplet market right from the outset: now that they’ve completed their first CPU design, a big RISC-V core, they will both license the design of their core to others to build into their own chips, and they will sell a chiplet containing many of their cores on the mass market.
There is even a new industry standard for connecting chiplets to each other, called UCIe: a chiplet that conforms to UCIe can be connected to any other chiplet that also conforms to UCIe. This new standard is managed by an industry consortium with members such as Google Cloud, Meta (Facebook), Microsoft, Intel, AMD, etc., and presumably these companies see themselves benefitting from this standard via making or using processors with chiplets.
The chiplet market is still in its infancy, and chiplets are very useful in some applications but are power-wasting in others, but the chiplet market has the potential to shake up how many processors are designed and introduce some competitive new blood into the industry.
Footnotes:
[1]: Often when a chip has a defect, you can disable the part the defect lands on and sell the mostly-functional chip as a cheaper product: many six core CPUs are actually eight core CPUs that had a defect in one or two of their cores. This process is called “harvesting”. So the number of chips you lose to defects isn’t quite as dire as the above image suggests.
