Steadcast
NVIDIA AI Podcast cover art
NVIDIA AI Podcast

Powering the AI Inference Wave with EPRI's Ben Sooter - Ep. 292

March 4, 202632 min · 5,848 words

Show notes

AI is reshaping electricity demand. What does increased demand, and the shape of that demand, mean for the electric grid? Ben Sooter, Director of R&D at EPRI joins the podcast to explain why most of an AI model’s lifetime energy use comes from inference rather than training, and how micro data centers located near underutilized substations can help deliver low‑latency AI services while strengthening grid resilience.

Highlighted moments

only about 20% of its like compute capacity and thus its power consumption is in the training side. 80% of it is in the inference side.
the training loads would slam hundreds of megawatts of, of demand, you know, nearly instantly within milliseconds. And they can also fall off once that job is done.
we seem to be coalescing somewhere around, um, this idea of, of 20 megawatts. Um, but that's actually sort of, I, I hadn't gotten into some of the, the electrical aspects of all this, but as we're looking at where to place these micro data centers, 20 megawatts is, can be the not insignificant ask, um, of just dropping, dropping a load somewhere onto the grid.
you're probably going to find three to five megawatts, maybe up to 10 megawatts of available capacity in a single substation.

Transcript

Transcript not available for this episode yet.

More from NVIDIA AI Podcast

Inside Instacart's AI-Powered Smart Shopping Cart | NVIDIA AI Podcast Ep. 302

Jun 24, 202639 min

How Mistral Is Building Frontier AI for the Enterprise | NVIDIA AI Podcast Ep. 301

Jun 10, 202621 min

Everyone Can Build a Robot: Open Source Embodied AI With Seeed Studio | NVIDIA AI Podcast Ep. 300

May 27, 202629 min

Inside AI Tokenomics: How to Profitably Turn Tokens Into Business Value | NVIDIA AI Podcast Ep. 299

May 21, 202633 min

Snap’s Secret to Processing 10 Petabytes a Day: GPU-Accelerated Spark | NVIDIA AI Podcast Ep. 298

May 13, 202623 min