Same question. One loops, one answers.
A real recording of both models on the same prompt with the same seed. It plays by itself — drag the timeline, change the speed, or pick another case.
recorded · llama.cpp · Q4_K_M
Unrolling the recording…
Fetching the recorded token streams (about 1 MB). Everything else on the page already works.
Press play. Thinking text appears in grey, the final answer in dark ink.
Is this cherry-picked? Here's the whole batch. loading…
The replay cases come from a batch of MATH-500 problems recorded with the same settings.
The base model gets stuck in a loop
With thinking on, small Qwen keeps re-checking itself — “wait, let me verify…” — until it runs out of budget instead of answering.
An LLM edits 0.016% of the weights
It subtracts one direction from 64 neurons' output weights (ffn_down) in layers 15 and 19 — 131,072 of ~800M weights.
Stripes drawn ~25× wider than true scale — at real size they would be invisible.
The edited model escapes
Same model, same prompts — it now stops looping and finishes its answers.
Where this might fit
Model souping - diffusion-LoRA merging is one case of it - is weight-space editing too. The difference is the target: souping picks merge weights (and sometimes merge order) with a broad benchmark; weight-space programming zeroes in on a few specific failure cases and the neurons behind them. And here's the fun part: it's a promising fit for agents - propose an edit, test it, feed the result back - the same edit-test-repeat loop behind the recent coding-agent wins.
Inside the edit
Where the 131,072 changed weights sit, and what the edited neurons do while the model thinks.
Weight diff loading…
Which of the 3,584 neurons feeding each edited ffn_down tensor were changed, and by how much.
Key neurons during inference loading…
The 64 edited neurons of layers 15 and 19, token by token, on the recorded runs. Synced to the replay above: hover to peek, drag to scrub.
- Loops light them upWhen the original model starts repeating itself, these neurons fire harder: the push the edit adds is about 2–4× bigger inside loops than in the rest of the same run.
- The edit pushes backThat push points against what the neurons were writing, so the harder they fire, the harder the edit steers away from the loop.
- Layer 19 calms downRun the edited weights over the same looping text (dashed line) and layer 19 is about half as active (≈ 2.1 instead of ≈ 4.0).
- The loop gets wobblyIts next tokens stop being a sure thing: 15.9% of loop tokens fall below 90% probability, against 0.4% for the original.
Waiting for the replay…
how hard the edit pushes · |push| per token · L15: the dashed line equals base by construction (layers below 15 are unchanged)
n = 4 loops + 1 control · correlation, not proof · bf16 forward over the recorded tokens · int8-quantised for the web
Footnote: the recorder marks only non-overlapping repeats, so a loop can start later than the text actually repeats; your browser re-checks every repeat and hatches the missed part. “Push” is what the edit would add given the base activations — its sign is a convention, so the chart shows its size.
Run it in your browser
Both models run locally via llama.cpp → WebAssembly + WebGPU. Nothing is sent to a server.
Sequential runs unload each model by default to limit GPU memory. n_ctx / n_gpu_layers apply on the next load.
Output streams
How long each model talks, and how it ends: grey = thinking, colour = answer, ✓ = stopped on its own, ⚠ = hit the cap. x = stream chunk (≈ one token).