Add us as a preferred source on Google

Newsletter

Subscribe to our newsletter

A lawyer with a private AI server built on eight used Nvidia V100 GPUs returned with an update, moving about 80% of its work up from 2-bit to 4-bit NVFP4 precision and bringing in 1TB of RAM and an RTX 3090. The server uses DeepSeek V4.1 Flash with a legal harness, the workflows and checks he wraps around the model, for his law practice. The cost? Performance, with code output dropping from 160–215 to 150–180 tokens per second, although the crashes that hit the new build are gone. The culprit behind the crashes, originally blamed on PCIe, was likely a bad PSU.

Lawyer’s V100 homelab, update 2: the 8x V100 box now runs DeepSeek at near-NVFP4 precision with a 524K context, a 3090 moved in, RAM went to 1 TB, and I blamed PCIe for what was probably one bad PSU from r/homelab

Latest Videos FromTom’s Hardware

You may like

What to read next

Swipe to scroll horizontally

DeepSeek V4.1 Flash on eight Tesla V100s: 2-bit vs. near-NVFP4 build, by prompt length

Prompt length

Wait for first token, 2-bit / near-NVFP4

Code (tokens/s), 2-bit / near-NVFP4

Legal writing (tokens/s), 2-bit / near-NVFP4

1K tokens

1.2 / 1.0 seconds

215.0 / 150.1

117.6 / 105.5

32K tokens

17.2 / 16.0 seconds

200.5 / 165.1

116.5 / 93.4

128K tokens

59.2 / 54.3 seconds

182.8 / 151.4

110.9 / 87.3

256K tokens

118.7 / 108.4 seconds

160.0 / 154.6

103.1 / 93.5