Overview

The Penguin Solutions MemoryAI KV Cache Server is the industry's first production-ready KV cache server built on CXL memory, announced by Penguin Solutions (NASDAQ: PENG) to attack the memory wall that now governs AI inference economics. The premise is straightforward: training and tuning are episodic and compute-bound, but always-on inference and agentic workloads are memory-bound — Penguin puts the split at roughly 30% compute-driven and 70% memory-driven. By storing and reusing computed key/value pairs in a dedicated high-capacity CXL memory appliance rather than inside GPU memory, the server expands the RAM available to GPUs across the cluster, enables memory disaggregation so each node draws what it needs, and removes the re-compute penalty that shows up as GPU idle time. The measured result is faster time-to-first-token, less time per output token and higher end-to-end token throughput, with more consistent latency against tight SLA targets. The production configuration, model Altus XE4318GT-KVC, is a 4U rackmount built on dual AMD EPYC 9005 processors with 88 DDR5 DIMM slots at 6400 MT/s. It carries 3 TB of DDR5 main memory and up to eight 1 TB CXL add-in cards for 8 TB of additional CXL-attached DDR5 memory, with total system memory reaching 11 TB; the CXL AIC in the reference configuration is the SMART CXA-8F2W. Expansion is eight PCIe Gen5 x16 full-height full-length slots plus two low-profile PCIe Gen5 x16 slots, which keeps the appliance usable as both a memory target and a PCIe host for accelerators or storage. Penguin states customers are already deploying the MemoryAI KV cache server in production clusters, and has shown it at NVIDIA GTC in San Jose (March 2026) and at AI Infra Summit 2026 in Santa Clara. It sits inside Penguin's Full-Stack AI Factory Platform alongside the company's compute systems, infrastructure software and design-and-build services for enterprise, sovereign and neocloud data centers.

Specifications

Availability

Available. Contact our sales team for current lead times, volume pricing, and integration support.