Tying CPU and GPU Together: The Norm for Laptops, Coming to Servers

Ian C
5 min readMar 17, 2023

--

Goodbye Intel + Nvidia, hello single vendor platforms.

Nearly every laptop and smartphone is powered by a chip that has both CPU and GPU [1] on the same chip. This is done for several reasons: it’s usually cheaper, saves power, and sometimes needs less RAM.

AMD “Raven Ridge” CPU+GPU; the CCX is a four core CPU cluster. Die labeled by Wikichip, die image from AMD.

Servers did not follow this trend, they would rather have a big CPU chip and a big GPU chip, rather than a small CPU and GPU on one chip. But now that the interconnect between chips has improved a lot, and the market is favouring chiplets [2], tying CPU and GPU chips closely together in a single unit is seen as the future: Intel, Nvidia and AMD all have plans to turn their server GPUs into CPU+GPU units.

Nvidia Grace Hopper Superchip

Nvidia’s H100 “Hopper” server GPU has been out for almost a year now. It has so far been used along with separate Intel or AMD CPUs, which is the norm for GPU servers.

H100 “Hopper” server GPU

Meanwhile Nvidia has been making their own CPU using cores they licensed from Arm, preferring to take control of their own destiny by making a CPU full of their custom features, rather than just buying one from Intel. Many in the industry are starting to go this route, as explained in my CPU wars article. This new Nvidia CPU is called Grace, and it’s a 72 core Arm CPU made for servers and machine learning (AI).

Which brings us to the Grace Hopper Superchip, a Grace CPU (left side) and H100 “Hopper” GPU (right side) on a single mezzanine board, named in honour of the computer pioneer Grace Hopper. The CPU and GPU are linked by a high speed 900 GB/s interconnect, that allows the GPU to access the CPU’s huge memory pool, which is useful for large workloads that don’t fit in the GPU’s small-but-fast HBM memory pool, such as large machine learning models. Nvidia is also making a Grace CPU Superchip, which is two Grace CPUs on one board and linked with the same high speed interconnect. Both superchips are scheduled for an H1 2023 release, so pretty soon.

AMD Instinct MI300 CPU+GPU

Meanwhile AMD’s upcoming CPU+GPU product ties the two even more closely together: the MI300 is composed of many CPU and GPU chiplets [2] connected together with very high speed interconnect on a single package. All chiplets also share the same high-speed HBM memory pool, rather than the split fast and slow memory pools on the Nvidia Grace Hopper Superchip. The MI300 is expected to be available near the end of 2023, and will power the El Capitan supercomputer, also expected in H2 2023 and it may be the most powerful supercomputer in the world on release.

The CPU chiplets are in the bottom right quarter, while the GPU chiplets are in the other quarters

Intel Falcon Shores XPU

Last and furthest away is Intel’s Falcon Shores XPU, an ambitious platform that allows mixing and matching CPU, GPU and other custom chiplets onto a single package, similar to AMD’s MI300 but more flexible. Unfortunately Intel recently announced a delay: the first product on this platform will come in 2025 and will be GPU only, meaning Intel won’t have a CPU+GPU product until 2026 or later. This gives AMD and Nvidia a three year head start in selling CPU+GPU products and gaining market share.

The GPU industry is shifting towards single-vendor systems

An Intel CPU paired with an Nvidia GPU (or many) has been the norm for a while, for gaming laptops and for GPU servers. This made sense, Intel was the top CPU vendor, Nvidia was the top GPU vendor. But now Intel is seriously investing in its own GPUs, and is bundling them along with its CPUs to try to convince laptop and GPU server makers to make products with only Intel CPUs and GPUs, cutting Nvidia out of the picture. Nvidia is doing the same thing, making their own CPUs to cut Intel out of the picture. And AMD has been making CPUs and GPUs for a while, but only now are their server products competitive enough to convince server makers to make AMD-only systems. This trend towards single vendor systems is intensified by these new CPU+GPU products, and each vendor has their own software ecosystem to lock you into. However most servers are CPU only, so they’ve always been single vendor systems, this is just extending that to GPU servers as well.

This is part of a larger industry shift towards processor makers becoming more vertically integrated: Apple, Qualcomm, Intel, AMD and Nvidia sell platforms now, with CPU, GPU and interconnect, often with a machine learning accelerator, DSP, FPGA or networking included. And this town may not be big enough for five platform companies to survive.

Footnotes:

[1] A CPU (Central Processing Unit) is “the boss”, it runs the operating system, most programs, and oversees everything. A GPU (Graphics Processing Unit), often called a video card, creates the images that are shown on a screen, plays video games, and is assigned certain tasks by the CPU because the GPU can do them faster.

[2] Chiplets are small chips that are closely connected to other chips on a package. The economics of the semiconductor market now favour chiplets in many applications, as explained in my chiplet market article.

Sign up to discover human stories that deepen your understanding of the world.

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

--

--

Ian C
Ian C

Written by Ian C

Computer processors, computer history, and other fixations

No responses yet