Newshot Original text(via HN AI)
Qwen/Qwen3.8-27B on 1 × NVIDIA GeForce RTX 5090, with TP 1
This is a theoretical maximum. Read how the estimate works
Memory per device Memory needed per device, sized for the larger of prefill and decode.If it does not fit, the speed shown is only a theoretical ceiling.Host RAM and SSD bars are per machine.
- Weights 53.8GB
- KV + state 691MB
- Activations 856MB
max 0 sequences per replica @ 8k
Bottleneck estimation Time per kind of op, added up before any overlap.Comm is the delay of each message plus the time to send its data.Offload transfer is the time to read offloaded weights.
- comm latency
- comm bandwidth
| Operation | Time | Share | Bound |
|---|---|---|---|
| mlp.gate_up | 12 . 7ms | 44% | memory |
| mlp.down | 6 . 37ms | 22% | memory |
| lin.in_proj | 4 . 52ms | 16% | memory |
| lin.out_proj | 1 . 69ms | 5 . 8% | memory |
| lm_head | 1 . 42ms | 4 . 9% | memory |
| attn.qkv_proj | 1 . 31ms | 4 . 5% | memory |
| attn.o_proj | 0 . 56ms | 1 . 9% | memory |
| attn.core | 0 . 30ms | 1 . 0% | memory |
| lin.recurrent | 0 . 17ms | 0 . 6% | memory |
| lin.conv1d | 6 . 58µs | 0 . 0% | memory |
| mlp.act | 3 . 73µs | 0 . 0% | memory |
| input_norm | 1 . 10µs | 0 . 0% | memory |
Decode throughput by batch size
- Per stream (left axis)
- Aggregate (right axis)
- language model only; vision/audio towers excluded
- Language decoder only: MTP and vision tower excluded; their parameters are estimated from config.
- Partial RoPE and output-gate activation variants are approximated; shared-expert sigmoid gates are omitted.