Nvidia noted that cost per token went from 20 cents on the older Hopper platform to 10 cents on Blackwell. Moving to Blackwell’s native low-precision NVFP4 format further reduced the cost to just 5 cents, so a basic upgrade gave a 4x improvement in cost per token while maintaining the accuracy that customers expect.
Nvidia outlined four industry deployments in a blog post showing how this combination of Blackwell infrastructure, NVFP4, optimized software stacks and open-source models delivers significant cost reductions. They break down like this:
- Healthcare — In healthcare, tedious, time-consuming tasks like medical coding, documentation and managing insurance forms cut into the time doctors can spend with patients. Sully.ai helps tackle this problem through AI agents to handle routine tasks that take up time.
The problem is that Sully.ai’s proprietary, closed source models didn’t scale well. So Sully.ai used Baseten’s open-source Model API on Blackwell GPUs with NVFP4 data format, the TensorRT-LLM library and the Dynamo inference framework .The result was a 90% drop in inference costs dropped by 90%, representing a 10x reduction compared with the prior closed source implementation, while response times improved by 65% for critical workflows like generating medical notes.
