Tying CPU and GPU Together: The Norm for Laptops, Coming to Servers
Goodbye Intel + Nvidia, hello single vendor platforms.
Nearly every laptop and smartphone is powered by a chip that has both CPU and GPU [1] on the same chip. This is done for several reasons: it’s usually cheaper, saves power, and sometimes needs less RAM.
Servers did not follow this trend, they would rather have a big CPU chip and a big GPU chip, rather than a small CPU and GPU on one chip. But now that the interconnect between chips has improved a lot, and the market is favouring chiplets [2], tying CPU and GPU chips closely together in a single unit is seen as the future: Intel, Nvidia and AMD all have plans to turn their server GPUs into CPU+GPU units.
Nvidia Grace Hopper Superchip
Nvidia’s H100 “Hopper” server GPU has been out for almost a year now. It has so far been used along with separate Intel or AMD CPUs, which is the norm for GPU servers.
Meanwhile Nvidia has been making their own CPU using cores they licensed from Arm, preferring to take control of their own destiny by making a CPU full of their custom features, rather than just buying one from Intel. Many in the industry are starting to go this route, as explained in my CPU wars article. This new Nvidia CPU is called Grace, and it’s a 72 core Arm CPU made for servers and machine learning (AI).
Which brings us to the Grace Hopper Superchip, a Grace CPU (left side) and H100 “Hopper” GPU (right side) on a single mezzanine board, named in honour of the computer pioneer Grace Hopper. The CPU and GPU are linked by a high speed 900 GB/s interconnect, that allows the GPU to access the CPU’s huge memory pool, which is useful for large workloads that don’t fit in the GPU’s small-but-fast HBM memory pool, such as large machine learning models. Nvidia is also making a Grace CPU Superchip, which is two Grace CPUs on one board and linked with the same high speed interconnect. Both superchips are scheduled for an H1 2023 release, so pretty soon.
AMD Instinct MI300 CPU+GPU
Meanwhile AMD’s upcoming CPU+GPU product ties the two even more closely together: the MI300 is composed of many CPU and GPU chiplets [2] connected together with very high speed interconnect on a single package. All chiplets also share the same high-speed HBM memory pool, rather than the split fast and slow memory pools on the Nvidia Grace Hopper Superchip. The MI300 is expected to be available near the end of 2023, and will power the El Capitan supercomputer, also expected in H2 2023 and it may be the most powerful supercomputer in the world on release.
Intel Falcon Shores XPU
Last and furthest away is Intel’s Falcon Shores XPU, an ambitious platform that allows mixing and matching CPU, GPU and other custom chiplets onto a single package, similar to AMD’s MI300 but more flexible. Unfortunately Intel recently announced a delay: the first product on this platform will come in 2025 and will be GPU only, meaning Intel won’t have a CPU+GPU product until 2026 or later. This gives AMD and Nvidia a three year head start in selling CPU+GPU products and gaining market share.
The GPU industry is shifting towards single-vendor systems
An Intel CPU paired with an Nvidia GPU (or many) has been the norm for a while, for gaming laptops and for GPU servers. This made sense, Intel was the top CPU vendor, Nvidia was the top GPU vendor. But now Intel is seriously investing in its own GPUs, and is bundling them along with its CPUs to try to convince laptop and GPU server makers to make products with only Intel CPUs and GPUs, cutting Nvidia out of the picture. Nvidia is doing the same thing, making their own CPUs to cut Intel out of the picture. And AMD has been making CPUs and GPUs for a while, but only now are their server products competitive enough to convince server makers to make AMD-only systems. This trend towards single vendor systems is intensified by these new CPU+GPU products, and each vendor has their own software ecosystem to lock you into. However most servers are CPU only, so they’ve always been single vendor systems, this is just extending that to GPU servers as well.
This is part of a larger industry shift towards processor makers becoming more vertically integrated: Apple, Qualcomm, Intel, AMD and Nvidia sell platforms now, with CPU, GPU and interconnect, often with a machine learning accelerator, DSP, FPGA or networking included. And this town may not be big enough for five platform companies to survive.
Footnotes:
[1] A CPU (Central Processing Unit) is “the boss”, it runs the operating system, most programs, and oversees everything. A GPU (Graphics Processing Unit), often called a video card, creates the images that are shown on a screen, plays video games, and is assigned certain tasks by the CPU because the GPU can do them faster.
[2] Chiplets are small chips that are closely connected to other chips on a package. The economics of the semiconductor market now favour chiplets in many applications, as explained in my chiplet market article.
