12 Used Nvidia V100s: Lawyer Builds Home AI Cluster to Run DeepSeek, Hits Up to 184 Tokens per Second
According to IT Home on November 21, a lawyer posting as u/TumbleweedNew6515 said in a Reddit community post on November 20 that he spent $9,450 to build a private AI server using 12 second-hand Nvidia V100 GPUs, running DeepSeek-V4.1-Flash locally, with the cluster reaching a maximum throughput of 184 tokens per second. On the hardware side, the lawyer used 12 Nvidia Tesla V100 SXM2 32GB cards, linked via NVLink and PCIe into a local AI cluster capable of collaborative inference, with the whole hardware purchase costing about $9,450. The system uses a structure of 3 pipeline stages (PP3) × 4-card tensor parallelism per stage (TP4), runs DeepSeek-V4.1-Flash, and uses 4-bit NVFP4 quantization; the poster said each expert was compressed to about 15,360 parameters, allowing the 12 V100s to each carry expert shards of the model.
Only about 69 characters could be extracted — the rest did not come through (common on sites that need scrolling or a login). Read the original for the full piece.