No Priors · Sep 18, 10:00 AM UTC
Diffusion LLMs can match custom hardware speed on Nvidia GPUs
“And then they switched over to to our diffusion based LLMs because they can essentially get the same speed as what you would get if you were to run an auto reggressive model on custom hardware. If you have a diffusion based LLM that it's built to be parallel, it's accelerating at the software level, then you can get the same speed on Nvidia GPUs, uh, which means much more, um, availability.”
Quoted source · 18:17–18:44 · 12 source segments
- [1] 18:17–18:21
- [2] 18:19–18:23
- [3] 18:21–18:25
- [4] 18:23–18:27
- [5] 18:25–18:29
- [6] 18:27–18:31
- [7] 18:29–18:32
- [8] 18:31–18:34
- [9] 18:32–18:37
- [10] 18:34–18:40
- [11] 18:37–18:42
- [12] 18:40–18:44
Model assessment
OpenCall, a voice-agent customer, previously served its LLMs on Cerebras custom chips but switched to Inception's diffusion-based LLMs, which achieve comparable speed on Nvidia hardware, indicating a loss of business for Cerebras.
17 exact transcript citations