Intel’s missed AI opportunity for Optane – could it return?

Today I’ve been doing some reading up on NVIDIA ICMSP, or Inference Context Memory Storage Platform.  The technology is a new solution that uses NVIDIA BlueField-4 intelligent NICs to store AI inferencing model context data in a key-value store on persistent storage, such as NVMe SSDs.

I won’t go into much detail in this article about the ICMSP technology, but suffice it to say, the idea is to create a shared persistent storage pool across many NVIDIA GPUs in a cluster that can all share context data generated from LLM conversations.  Think of it like any storage caching technology that benefits from long-term retention of active data, where queries could come from multiple sources.