At the Open Compute Project Global Summit in San Jose this week, Samsung Electronics and AMD staged a demonstration that addressed a problem familiar to anyone running large models in production: one server runs out of memory while a rack mate sits with capacity going unused.
The two companies showed a system built on Compute Express Link, or CXL, an interconnect that lets a processor reach memory sitting outside its own chassis. In the demonstration, multiple servers drew from a shared pool assembled from Samsung’s CMM-D memory modules, borrowing capacity when a workload demands it and returning it afterward. Samsung said the arrangement was validated in a live system with AMD, which supplies the processors on the other end of the link.
The mechanism rests on an awkward fact about AI hardware. Computing power has scaled far faster than the memory attached to it, and inference, the stage where a trained model answers a request, is bound by memory capacity and bandwidth. Every conversation a model holds requires a key-value cache, a running store of intermediate results that grows with context length, and that cache consumes high-bandwidth memory that is both costly and in short supply. Operators overprovision today because they cannot predict which server will fill first.
Pooling attacks the provisioning problem. Samsung’s MD310 platform lets several servers share a pool of CMM-D modules, so capacity can be assigned to the workload that needs it rather than stranded in the machine that owns it. The company pitched the approach as a way to cut capital and operating costs in data centers where memory is the scarcest component.
The standard behind it is not new. AMD shipped native support for CXL type-3 memory devices with its EPYC Genoa processors in 2022, and the current Turin parts keep it. Intel added the capability with Emerald Rapids after leaving it out of Sapphire Rapids. Suppliers including Micron and SK Hynix have shipped modules, and Microsoft opened CXL-equipped cloud instances to preview late last year. Samsung itself displayed CXL hardware at earlier summits, along with a management platform developed in house.
Deployment has lagged the specifications. Samsung has pushed back mass production of CXL 3.1 memory modules while continuing to ship the 2.0 generation, according to industry reports, and the software that decides which pages live in the pool remains immature. Buyers have been willing to test and slow to commit, analysts said, because a shared memory pool is also a shared failure domain.
Samsung presented the demonstration as evidence that the technology survives contact with production systems. Shim Ho-jun, a vice president on the company’s application engineering team, told the summit that raw memory speed is no longer sufficient on its own and set out three priorities: speed, capacity expansion and data persistence. The exhibition halls were organized around those ideas, with one zone given to high-bandwidth memory generations, including HBM4E and HBM5 concept products and a low-power chip with processing-in-memory, and another to the cache problem, showing CXL memory, enterprise solid-state drives, vector databases and direct storage.
The commercial logic behind that second zone is where the money may be. If a pooled tier can hold caches that would otherwise occupy the most expensive memory in the rack, the savings show up in the total cost of ownership rather than in a benchmark. Several vendors have reported large multiples of concurrent model instances and lower energy per token when caches move into CXL memory, though those figures come from controlled tests rather than from fleets carrying customer traffic.
The competitive dimension is harder to ignore. High-bandwidth memory is the product that sets a memory maker’s profitability this cycle, and it is sold in tight allocations to accelerator vendors. CXL memory uses ordinary DRAM behind a controller, which opens a demand channel that does not compete for the same scarce supply and gives Samsung a way to sell a memory tier alongside a design service. The company has opened a lab where customers can test emerging memory under their own workloads, and it funds an open-source group working on cache optimization and direct storage.
Set against that is a market that has heard CXL promises before. Memory pooling requires switches, controllers, firmware and orchestration to work together, and each layer has its own vendors with their own schedules. Analysts who track the segment expect the market to grow quickly from a small base, with much of the volume arriving late in the decade. The technology also has to justify itself against a simpler alternative that vendors discuss less often: buying more memory than needed and leaving part of it idle.
For AMD, the demonstration is part of a broader effort to position its processors as the control point of an AI rack rather than a component inside one. For Samsung, a memory maker that has watched accelerator vendors capture the margin in AI systems, the argument is that the next gain comes from how memory is organized, not from how fast a single chip runs.
Both companies said they would continue the collaboration. Neither said when a customer would run production traffic on a pooled memory tier, which is the only test that will settle whether the demonstration describes a product or a possibility.
