From NVMe Standards to AI Infrastructure: Eran Kirzner's Two-Decade Journey

When you’ve spent 20+ years building infrastructure for data centers, you start to see patterns that others miss. For Eran Kirzner, one of those patterns became the foundation for Lightbits Labs—a company that’s now reshaping how AI workloads run at scale.

The Origin Story: Spotting an Outdated Model

It all started in 2016 when Avigdor Willenz, a veteran founder from Israel’s tech scene, approached Kirzner with a deceptively simple question: what’s the next big problem data centers will face?

The two didn’t come in as fresh-faced Stanford dropouts with a half-baked idea. Instead, they assembled a small team of three to four industry veterans—people who’d spent decades in networking, compute, and storage. Their collective experience became their superpower.

What they discovered was striking: the storage solutions dominating the market were built on an outdated model. Major vendors like Dell, IBM, and Hitachi were selling monolithic appliances. Meanwhile, cloud giants like Amazon, Microsoft, and Google had figured out something different—they’d built custom, proprietary solutions that gave them massive competitive advantages. But those solutions weren’t accessible to anyone else.

“Our goal was to help private clouds and data centers build the next generation storage solution,” Kirzner explains. The answer? A new protocol called NVMe over TCP.

Building a Standard, Not Just a Product

Here’s what makes Lightbits’ story different from your typical startup: they didn’t just invent a protocol. They made it a standard.

That’s a massive distinction. Any startup can build cool technology. But getting an entire industry to adopt and standardize around your invention? That’s another level entirely.

NVMe over TCP was an extension of the existing NVMe standard, designed to carry storage over the network using standard Ethernet and TCP/IP. In 2017-2018, this was genuinely visionary thinking. Most of the storage world was still running on SATA drives, and everything was built around Fiber Channel protocols. Kirzner’s team was betting that the future would be different.

The timing was crucial. They had major technology investors backing them—Cisco, Intel Capital, Dell Capital, Microsoft, Lenovo—companies that could help evangelize the protocol throughout the industry ecosystem. But capital alone doesn’t make a standard. You need adoption.

So Lightbits built the reference implementation for Linux, partnered with VMware to integrate it into their platform, and worked to get it embedded into Open Stack, Kubernetes, and other critical infrastructure. They built the entire ecosystem around the protocol.

“Even some of our investors said it wouldn’t work,” Kirzner recalls. “They thought it wouldn’t be fast enough, that you couldn’t get the quality of service you need from storage over TCP/IP.”

They were wrong. Today, a decade later, every major storage vendor supports NVMe over TCP. If you Google it, you’ll find Lightbits credited as the inventors. It’s in Wikipedia. It’s the standard.

The Pivot: From Storage to AI Inference

But here’s where the story gets even more interesting. By the time Lightbits had conquered the storage world, the industry was shifting seismically. GPUs were everywhere. AI workloads were exploding. And a new problem was emerging.

The problem is called the GPU memory wall.

Think about it: every GPU has a fixed amount of high-bandwidth memory (HBM) built directly onto the chip. This memory is incredibly fast, but it’s limited. An H100 GPU might have 80GB of HBM. If you want to run a large language model with hundreds of billions of parameters, you quickly run out of space.

What do companies do? They buy more GPUs—often spending millions just to get access to more memory. It’s wasteful and economically irrational, but it’s been the only option.

Lightbits saw an opportunity to apply their infrastructure expertise to this new problem. What if you could extend GPU memory over the network? What if you could create a “virtual HBM” that spans across multiple GPUs, pulling data from high-speed NVMe storage when needed?

The Math Gets Interesting

The results they’re achieving are compelling. On a standard H100 GPU, Lightbits’ solution enables:

• 10x increase in model parameters that can run simultaneously (from 1 million to 10 million input tokens)
• 10x increase in concurrent sessions (from 10 to 100 simultaneous inference requests)
• 10-20x improvement in time-to-first-token (the latency users actually experience)
• GPU utilization jumping from 30% to 80%

These aren’t marginal improvements. These are the kinds of gains that fundamentally change the economics of AI inference.

The solution works by doing two things: first, it extends the available memory pool beyond what’s physically on the GPU. Second, it intelligently predicts what data the GPU will need and brings it “just in time”—minimizing stalls and keeping the processor fully utilized.

Scaling Complexity

But here’s what separates infrastructure companies from everyone else: when you’re running thousands of nodes, everything changes.

Kirzner asks candidates he interviews a simple question: “Do you know what tail latency is?” Most people haven’t thought deeply about it. But when you’re running at scale—when you have 1,000 or 10,000 nodes—tail latency becomes everything. Sure, your average latency might be great. But that one node that’s slow? That’s the one that determines your user experience.

This is why Lightbits had to build for enterprise-grade reliability from day one. You can’t lose data in financial services. You can’t have fake results. Storage isn’t like networking—if your network drops a packet, you might recover it. But storage? That’s where your data lives. It has to work, every single time.

The Shift in AI Workloads

What’s fascinating is watching the market transition in real-time. Two years ago, GPU workloads were dominated by training (80%) versus inference (20%). Today, that’s flipping. More and more of the compute is going toward inference—running models that have already been trained.

And unlike training, which happens once, inference happens continuously, at scale. A financial institution might run thousands of inference requests per second. A cloud provider might be serving millions.

This is where Lightbits is positioned. They’re not just solving a technical problem. They’re solving an economic one. By increasing GPU utilization from 30% to 80%, they’re making AI inference dramatically more cost-effective. That changes the entire business model for companies building AI services.

The Takeaway

What’s remarkable about Eran Kirzner’s journey is that it spans two different eras of infrastructure problems. He solved one (storage networking), made it a standard, and then pivoted to solve the next one (GPU memory wall) using the same deep infrastructure thinking.

That’s the mark of a true infrastructure founder: the ability to see problems before they become obvious, and the persistence to build not just products, but entire ecosystems around solutions.

Want to hear the full story directly from Eran? Listen to the complete episode on Code Story to dive deeper into how Lightbits is powering the next generation of AI inference infrastructure, the challenges of building standards, and what’s next for the company. You’ll also get insights into his journey from Israel’s tech scene to building a company that’s trusted by some of the world’s largest financial institutions and cloud providers.