Follow us
Add us as a preferred source on Google
Newsletter
Subscribe to our newsletter
A lawyer with a private AI server built on eight used Nvidia V100 GPUs returned with an update, moving about 80% of its work up from 2-bit to 4-bit NVFP4 precision and bringing in 1TB of RAM and an RTX 3090. The server uses DeepSeek V4.1 Flash with a legal harness, the workflows and checks he wraps around the model, for his law practice. The cost? Performance, with code output dropping from 160–215 to 150–180 tokens per second, although the crashes that hit the new build are gone. The culprit behind the crashes, originally blamed on PCIe, was likely a bad PSU.
Lawyer’s V100 homelab, update 2: the 8x V100 box now runs DeepSeek at near-NVFP4 precision with a 524K context, a 3090 moved in, RAM went to 1 TB, and I blamed PCIe for what was probably one bad PSU from r/homelab
Latest Videos FromTom’s Hardware
You may like
What to read next
Swipe to scroll horizontally
DeepSeek V4.1 Flash on eight Tesla V100s: 2-bit vs. near-NVFP4 build, by prompt length
Prompt length
Wait for first token, 2-bit / near-NVFP4
Code (tokens/s), 2-bit / near-NVFP4
Legal writing (tokens/s), 2-bit / near-NVFP4
1K tokens
1.2 / 1.0 seconds
215.0 / 150.1
117.6 / 105.5
32K tokens
17.2 / 16.0 seconds
200.5 / 165.1
116.5 / 93.4
128K tokens
59.2 / 54.3 seconds
182.8 / 151.4
110.9 / 87.3
256K tokens
118.7 / 108.4 seconds
160.0 / 154.6
103.1 / 93.5