At the Hot Chips conference on August 25, Samsung detailed a new kind of memory chip built to tackle one of the AI industry’s biggest headaches: the soaring cost of the high-bandwidth memory that powers most AI chips today. The chip, called LPDDR5X-PIM, takes a different approach by moving some computing work directly inside the memory itself.
Normally, a processor has to constantly fetch data from memory, do the math, and send the result back — a round trip that costs both time and power. Samsung’s chip adds small processing units directly onto the memory die, so basic calculations happen right where the data already lives. Using Meta’s Llama 3.1 8B model as a test case, Samsung says the approach pushed inference speed from 27 tokens per second up to 81.3 tokens per second, roughly three times faster, while cutting the time to complete a task from 12.3 seconds down to 5.4 seconds.
Unlike the pricier HBM (high-bandwidth memory) chips used in large AI data centers, LPDDR5X-PIM is built on the same low-power memory standard already used in phones and laptops, and Samsung designed it as a drop-in replacement — it uses the same physical package as standard LPDDR5X, so device makers wouldn’t need to redesign their hardware to use it. Samsung is positioning it as a complement to HBM rather than a replacement, aimed at situations where cost, power efficiency, and inference speed matter more than raw bandwidth, such as future AI-enabled phones, laptops, and edge devices.
The timing lines up with a real problem across the industry: memory chip prices have climbed sharply through 2026 as AI data centers snap up available HBM supply, a shortage that’s already pushed up prices on phones and other consumer electronics that also rely on DRAM. Samsung’s pitch is that spreading some AI workloads onto cheaper, more efficient memory like LPDDR5X-PIM could ease that pressure over time, especially for on-device AI features that don’t need a full data-center-grade chip.
The technology isn’t sitting on store shelves yet — JEDEC, the industry group that sets memory standards, is still working through formal specifications for this style of chip. But Samsung has already released a simulator and developer kit for companies that want to start testing it, suggesting real products could follow sooner rather than later.