Why the chip industry is shifting to 'memory by design'

The useful part

Transistors were drawn on a flat surface, wires were run across it, and progress was measured by how small a distance could be maintained between two features. The most consequential decision in computing was itself a piece of flat-world planning. On a plane, two things cannot occupy the same place, so anything that must be large and anything that must be fast are built side by side and connected.

How it works

• Flash went vertical more than a decade ago because its cells had stopped shrinking usefully, and stacked DRAM followed.
• Partly because more money is available for this research than at any point in the industry's history.
• Apparently, the folks at OpenAI heard a rumor that two mathematicians, one of them an Anthropic researcher, were close to a Millennium Problem.
• The table shows the moment the lever changes, and it is worth pausing on the column that names the mechanism.
• The exhibit below shows the two directions as they are usually drawn: bandwidth across the plane and capacity up the stack.

What to take from it

Partly because fields that used to advance separately — memory process technology, logic foundry technology, packaging, thermal engineering, and design software — have all arrived at the same problem in the same year. Readers who have difficulty accessing or viewing the exhibits are encouraged to reach out to our team. A giant aside: the week the fluid blew up This may not be the E equals MC squared of our era, but the week the Navier-Stokes problem falls, or claims to have fallen, to a machine is not a week to open a note with anything else.

Example or evidence

• The stack was a taller building in a two-dimensional town, joined to the processor next door by a road.
• Nvidia is pulling the memory controller and the interface upward into its own platform, and selling that platform to everyone building custom silicon.
• SK Hynix is building the same capability through partnerships, not ownership, and by putting advanced packaging close to its customers.
• Seven decades of engineering then went into widening that channel and shortening it, which is what caches, wider buses, and eventually stacked memory placed beside the processor have all been doing.

Details worth keeping

The chip industry's future is memory by design. Semiconductors have lived in that world for 70 years. The single number, quoted in microns and then nanometers, became the industry's scoreboard, its marketing, and eventually its mythology, for the simple reason that in a flat country there is only one direction in which to get better.

Related coverage

• Marketwatch: Semiconductor industry revenue is expected to reach $1.5 trillion this year, driven mostly by memory chips.
• Wired: Kepler Computing claims a new approach to chip design—and a proprietary material—can help end the supply bottlenecks that have sent memory prices surging.
• Nikkei: China's CXMT tops SK Hynix, Micron to take memory profit margin crown