Dwarkesh Podcast
Reiner Pope – The math behind how LLMs are trained and served
April 29, 20262h 13m · 22,975 words
Show notes
Did a very different format with Reiner Pope - a blackboard lecture where he walks through how frontier LLMs are trained and served. It’s shocking how much you can deduce about what the labs are doing from a handful of equations, public API prices, and some chalk. It’s a bit technical, but I encourage you to hang in there – it’s really worth it.
Highlighted moments
if you do not batch together many users, the cost and the economics you get can be like a thousand times worse than if you do batch many two users together.
“The fact that they are charging 5x less for pre-fill than decode does suggest that they are bottlenecked on memory bandwidth to quite a degree”
Transcript
Transcript not available for this episode yet.
More from Dwarkesh Podcast
8 Predictions for the Era of Continual Learning
Aug 7, 20268 min
Why smarter AI models could drive up compute prices 10x
Aug 3, 202611 min
Adam Brown – A deep but accessible introduction to general relativity
Jul 10, 20261h 38m
Grant Sanderson – AI and the future of math
Jun 30, 20261h 33m
The next big breakthrough will be AIs learning on the job
Jun 26, 202619 min