Newshot

12 Used Nvidia V100s: Lawyer Builds Home AI Cluster to Run DeepSeek, Hits Up to 184 Tokens per Second

AI summary

According to IT Home on November 21, a lawyer posting as u/TumbleweedNew6515 said in a Reddit community post on November 20 that he spent $9,450 to build a private AI server using 12 second-hand Nvidia V100 GPUs, running DeepSeek-V4.1-Flash locally, with the cluster reaching a maximum throughput of 184 tokens per second. On the hardware side, the lawyer used 12 Nvidia Tesla V100 SXM2 32GB cards, linked via NVLink and PCIe into a local AI cluster capable of collaborative inference, with the whole hardware purchase costing about $9,450. The system uses a structure of 3 pipeline stages (PP3) × 4-card tensor parallelism per stage (TP4), runs DeepSeek-V4.1-Flash, and uses 4-bit NVFP4 quantization; the poster said each expert was compressed to about 15,360 parameters, allowing the 12 V100s to each carry expert shards of the model.

Translation(model translation, not the source's own text)

12 Used Nvidia V100s: Lawyer Builds Own AI Cluster to Run DeepSeek,

Only about 69 characters could be extracted — the rest did not come through (common on sites that need scrolling or a login). Read the original for the full piece.

Published by IT之家
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗