LLM's Play Zachtronics' SpaceChem

Released by Zachtronics in 2011, SpaceChem is a puzzle game about building chemical factories that turn raw input molecules into target compounds. Inside each 10×8 reactor, you lay down looping tracks and commands for two moving grabbers (called waldos, one red and one blue) that pick up atoms, break and form bonds, and carry finished molecules to the output. It is widely considered one of the hardest puzzle games ever made: both waldos and the molecules they carry share the same cramped grid, so one mistimed turn or wide rotation causes a collision, and later levels chain multiple reactors together where a single bottleneck jams the whole pipeline. To test how well LLMs handle that kind of 2D spatial and timing reasoning, we gave 16 models all 55 non-boss puzzles through a text interface, withheld the 15-year community archive, and told them to keep optimizing.

Within days, players picked up the models’ reactors and improved on them.

Watch the launch video.

The game scores a working solution on cycles, reactors, and symbols: how long it runs, how many reactors it uses, and how many arrows and instructions it places. Lower is better, but the scores pull against each other. A faster machine usually needs more symbols, so each puzzle has a frontier of tradeoffs instead of one best answer. Players have been trading solutions on a public community archive for years.

Leaderboard

We ran 16 LLMs, covering leading frontier and open-weight models. Every model attempted every non-boss puzzle in the main campaign, the 63 Corvi expansion, and the three Team Fortress 2 levels, in one scored attempt each (pass@1), with 10 to 60 minutes per puzzle, depending on difficulty. 37 of the puzzles run inside a single fixed reactor. In the other 18 (SpaceChem’s production levels), the model routes pipes between tanks and reactors on a terrain map, from a zero-reactor pipeline tutorial up to six-reactor chemical plants.

Model ranking

Average proximity to community records · 100 matches the archive

All 55 puzzles · 37 single-reactor / 18 multi-reactor · 19 easy / 18 medium / 18 hard

• 74.3 record proximityGPT-6 Astra

• 69.7 record proximityGPT-6.1 Sol

• 66.2 record proximityOpus 5.5

• 65.2 record proximityGemini 4 Argon

• 53.7 record proximitySonnet 5.5

• 49.8 record proximityFable 5.1

• 42.6 record proximityGemini 3.8 Flash

• 41.7 record proximityKimi K3.1 Pro

• 36.8 record proximityMuse Wonder 1.0

• 31.8 record proximityDeepSeek V4.1 Flash

• 29.4 record proximityMuse Spark 1.3

• 29.4 record proximityGrok 4.7

• 24.5 record proximityHaiku 5.5

• 19.3 record proximityGPT-6 Luna

• 16.4 record proximityGLM-5.3

• 9.8 record proximityMiMo-V2.6-Pro

Scroll horizontally to compare all models →

GPT-6 Astra leads on every Pareto-aware metric: 74.3 record proximity, 67.8 sum score, 69.7 3D hypervolume, and 57.4 frontier coverage, solving 54 of 55 puzzles and missing only Ω-Pseudoethyneorganic_5, which no model solved. Record proximity compares a model’s best cycle, reactor, and symbol scores against the community records (100 matches them), even when those three come from different solutions; sum score, following Opus Magnum Bench, compares the single solution with the lowest combined cycles + reactors + symbols; and 3D hypervolume measures how much of the community Pareto frontier’s dominated volume the model’s frontier covers. GPT-6.1 Sol is second across all three (69.7 proximity, 62.4 sum, 63.2 hypervolume, 52 solved), ahead of Opus 5.5 (66.2, 60.9, 61.8, 48 solved). On frontier breakthroughs, Gemini 4 Argon ties GPT-6 Astra at the top with 7 new frontier points across 5 puzzles (evaluated per model against the frozen archive; pooling across all models yields 20 nondominated points), despite ranking fourth overall (65.2 proximity, 46 solved) after losing 8 of the 18 multi-reactor puzzles.

Compared with TIS-100 Bench, where the same 16 models solved 82% of episodes (724 of 880), found 133 new frontier points, and Opus 5.5 tied GPT-6 Astra at 55 of 55 solved, SpaceChem cuts overall solve rate to 56% (495 of 880). Multi-reactor puzzles drive most of that drop: 68% of single-reactor episodes passed (404 of 592), against 32% of multi-reactor ones (91 of 288), even though multi-reactor puzzles are split evenly across easy, medium, and hard. Fable 5.1 solved 52 of 55 in TIS-100 Bench and 32 of 37 single-reactor puzzles here, but only 4 of 18 multi-reactor ones.

Counts are out of 55 puzzles. Point totals are in parentheses and are evaluated per model against the frozen archive (28 model points in total, which pool to 20 nondominated points across models). Methods

RL environments & traces are available upon request for research teams. Get in touch ↗

How it works: harness design

SpaceChem has no text format to type into. Players drag arrows and instructions onto the grid with a mouse. We run the unmodified Linux release of the game on a virtual display and give models text tools instead. A model’s edits are written straight into the game’s save file, and every score in this article comes from the real game.

Why check for precognition? Fourteen puzzles draw some inputs at random, but the game always uses the same seed. A solution can pass by assuming the sequence it will see instead of sensing each atom, and the community doesn’t accept that. We run SChem’s precognition check, which replays a solution on other seeds, after every passing run and tell the model if it fails. It flagged one of 7,497 passing runs: on The Blue Danuberesearch_1, Gemini 4 Argon’s 191-cycle solution passed in the game, but only 40 of 312 runs on other seeds succeeded. Gemini 4 Argon’s best clean solution stayed at 219 cycles.

The search over time

A passing solution is only the starting point: models keep editing and running until the 10- to 60-minute clock expires, trading symbols for cycles or recovering an earlier checkpoint with get_best when a rewrite crashes. On No Employment Record Foundresearch_6, for example, Sonnet 5.5’s first passing reactor two minutes in took 1,113 cycles; twenty minutes later it reached 465 cycles before finishing at 464/1/23 and 504/1/22 (examined up close below).

Loading puzzle trajectories…

Search trajectories by puzzle

Best in-game score so far, before replay. Each score can come from a different solution, and lines stop when the episode ends.

Where 2D spatial programming breaks the models

2D layout reshuffles the TIS-100 leaders, and price still doesn’t buy rank

In TIS-100 Bench, Opus 5.5 and GPT-6 Astra finished in a statistical tie. Moving from 1D assembly to a 10×8 spatial grid separates them: GPT-6 Astra leads at 74.3 record proximity for an estimated $5.55 per puzzle, while GPT-6.1 Sol takes second at 69.7 for $1.17, 4.7 times cheaper than GPT-6 Astra and 4.2 times cheaper than Opus 5.5 (66.2 at $4.90). Muse Wonder 1.0 is the most expensive model tested at $9.94 per puzzle and finishes ninth at 36.8.

Performance versus resources

One point per model. Upper left is better.

Thrashing on the grid doesn’t buy spatial intuition. Across all episodes, models made 15,562 waypoint-path edits (set_reactor), 10,786 single-cell placements, and 8,674 full-solution loads. Gemini 3.8 Flash averages 239 game calls per puzzle, the most of any model, for 42.6 record proximity; Gemini 4 Argon averages 52 calls for 65.2 and ties for the most new frontier points.

How models use the game

Mean game-tool calls per puzzle

Physical collisions, not wrong chemistry, kill 57% of failed runs

Models ran their reactors 23,745 times, and 32% passed. Of the 16,248 failed runs, 57% crashed from physical collisions: two atoms meeting in one cell, a rotated molecule clipping a wall, or a new input spawning onto an atom that hadn’t cleared the intake zone. Another 26% deadlocked in loops that never reached the target count, and only 12% actually output the wrong molecule (5% failed static checks before simulation). Models usually figure out the chemistry; they lose on 2D traffic control.

Where simulation runs fail

Mean failed simulation runs per puzzle, by cause

Unlike TIS-100, frontier gains cluster on medium puzzles rather than hard ones

In TIS-100 Bench, two-thirds of the puzzles with new frontier points were hard (12 hard, 6 medium, 0 easy). In SpaceChem, breakthroughs cluster on 7 medium puzzles, against 3 hard ones and 1 easy one. Single-reactor medium puzzles were solved as often as easy ones, in 77% of episodes, and their 40-minute clock left time to tighten waldo loops after passing. On hard single-reactor puzzles, solve rate fell to 49%, and on multi-reactor puzzles no model beat the archive at any difficulty.

Solving versus extending the frontier

Full bar = solve rate · dark segment = new-frontier rate

图表：Select a dot to inspect its puzzle and community records.

Every puzzle by its community records

Position = community records · circle area = fewest reactors

Loading the puzzle map…

Every new point trades symbols for cycles, and single-solution sum scores expose split frontiers

Whereas TIS-100 Bench produced three outright cycle records, no SpaceChem model beat the community’s best cycle, reactor, or symbol count on its own. Across all 16 models, mean proximity is 54 on reactors, 38 on symbols, and 32 on cycles. Because models often save separate fast-but-sprawling and compact-but-slow reactors on the same puzzle, requiring a single machine to minimize the combined sum score (cycles + reactors + symbols) drops GPT-6 Astra from 74.3 record proximity to 67.8 sum score and Gemini 4 Argon from 65.2 to 57.0.

Cycles, reactors, and symbols

Mean proximity to each community record · higher is better

图表：Select an episode to inspect its measurements.

The spread across puzzles

Every dot is an episode · box = middle 50% · line = median · whiskers = minimum and maximum

图表：Hover or focus a cell for its puzzle, outcome, and record proximity.

Every model, every puzzle

Choose a cell to inspect the scores in the Pareto explorer.

What the models built

Across all 16 models, 28 model-frontier points beat the frozen archive; pooling across models deduplicates shared discoveries and drops six points superseded by another model on No Employment Record Foundresearch_6, An Introduction to Sensingsensing_1, and Ice to Meet Youfusion_1, leaving 20 nondominated points across 11 puzzles. None of the 20 beats the community’s best cycle, reactor, or symbol count outright (each is a new tradeoff), and 11 of them knock an existing community solution off the frontier by matching or beating it on all three scores. All 20 are on single-reactor puzzles.

All 20 new frontier points

Cycles/reactors/symbols · lower is better · select a puzzle to open it in the explorer

One record up close: random inputs and fusion

No Employment Record Foundresearch_6 feeds in boron, aluminum, or silicon at random and wants hydrochloric acid. The reactor has to fuse each atom up to chlorine, check it with a sensor, and bond it to hydrogen. When we froze the archive, the best community solution with 23 symbols took 501 cycles. Sonnet 5.5’s first passing solution, two minutes in, took 1,113 cycles. Twenty minutes in, it had a 465-cycle solution, and it finished with two new frontier points: 464/1/23, which knocks the community’s 501/1/23 and 465/1/24 off the frontier, and 504/1/22, 36 cycles faster than the best 22-symbol solution. Opus 5.5 added 548/1/21, between the community’s 22- and 20-symbol solutions (and GPT-6 Astra’s 464/1/24 and Opus 5.5’s 473/1/23 and 508/1/22 also beat the frozen archive at 22–24 symbols before being superseded by Sonnet 5.5’s points in the pooled frontier).

The 464-cycle reactor

Sonnet 5.5 · 1 reactor · 23 symbols · replayed in a fresh game

Layout

Running in the game

A tutorial with a cycle to spare

Slightly Differentresearch_tutorial_1 is an early tutorial: move ten hydrogen molecules from the β input to the ω output. The community’s best solutions, 52/1/11 and 53/1/10, already shuttle both waldos back and forth along one row instead of running them around a loop. GPT-6 Astra kept that idea, moved the track one column left, changed where the waldos start, and found 51/1/11. GPT-6 Astra, GPT-6.1 Sol, and Muse Wonder 1.0 each found 52/1/10, which beats both old solutions. Muse Wonder 1.0’s is now in the archive.

The 51-cycle shuttle

GPT-6 Astra · 1 reactor · 11 symbols · the box on the right of the recording is the tutorial’s own hint

Layout

Running in the game

Two old solutions beaten at once

The Plot Thickensmining_7 builds polyethylene chains from carbon and hydrogen and outputs them into a tall output zone. Gemini 4 Argon’s 720/1/18 is faster and uses fewer symbols than two community solutions at once: 729/1/21 by Haxton and 805/1/19 by Zemyla. Its red waldo runs a 2×6 loop near the left wall, taking in atoms, bonding, and outputting. The blue waldo shuttles along one row with just four instructions: sync, bond, grab, and rotate. At the fast end of the same frontier, GPT-6.1 Sol added 468/1/31 and 469/1/29, between the community’s 449/1/34 and 551/1/27.

The 18-symbol polymer line

Gemini 4 Argon · 1 reactor · 12 red and 6 blue symbols

Layout

Running in the game

Fusion, 21 cycles faster

Under the Icefusion_2 makes nitrous oxide from hydrogen and nitrogen, and the reactor has a fuser, which merges two atoms into one with their combined atomic number. Gemini 4 Argon’s 105/1/18 beats busdriver’s 126/1/18 at the same symbol count, 17% faster. Its red waldo runs a 2×5 loop that takes in both inputs, and the blue waldo shuttles along one row through the fuser. Gemini 4 Argon also found 102/1/19, between the community’s 100/1/22 and 126/1/18.

The 105-cycle fusion reactor

Gemini 4 Argon · 1 reactor · 18 symbols

Layout

Running in the game

Performance by puzzle mechanic

Fraction solved · puzzles with new points below

Co-discovery

SpaceChem Bench started with a challenge on the Zachtronics Discord. After early LLM benchmarks came out on Opus Magnum and TIS-100, @12345ieee, who holds hundreds of records in the SpaceChem archive, asked when models would come to SpaceChem:

When our runs finished, two of the models’ new frontier points beat @12345ieee’s own solutions in the frozen archive: 465/1/24 on No Employment Record Foundresearch_6 and 314/1/10 on Moustachium 604tf2_2. We submitted selected frontier solutions to the community archive on October 27, including Sonnet 5.5’s, GPT-6 Astra’s, and Opus 5.5’s four new points on No Employment Record Foundresearch_6.

Three days later, @12345ieee came back and posted 463/1/23, 462/1/24, 503/1/22, and 544/1/21 on No Employment Record Foundresearch_6, each one to four cycles faster than the model solution it replaced. The new files share most of their lines with the model files: 463/1/23 keeps Sonnet 5.5’s layout, moving one red bond instruction and shifting a three-arrow corner by one cell (4 of the 29 MEMBER lines).

Just like on TIS-100, neither side got there alone. The models found new layouts and symbol tradeoffs across the archive, and players read those reactors, saw where a corner or instruction could tighten by a cycle, and pushed the frontier further.

Pareto explorer

Choose a puzzle and model, then select a point to see its scores and the reactor that produced it. Every puzzle also has its own page with records and the best solutions: all 55 puzzles.

Pareto explorer

The plot shows cycles and symbols; reactor counts can differ. New frontier points are judged on all three scores.

Download all 1,169 model Pareto points (CSV) · all 1,169 solutions (ZIP), also on GitHub and Hugging Face. 28 rows are marked as new frontier points, which pool to 20 across models.

All 55 puzzles

Grouped by main mechanic. Select a puzzle to open it in the explorer, or browse all 55 puzzle pages.

• Multiple Outputseasy · single-reactorInputs: α Carbon Monoxide (CO). Output: ψ Carbon (C) ×10, ω Oxygen (O) ×10.

Inputs: α Carbon Monoxide (CO). Output: ψ Carbon (C) ×10, ω Oxygen (O) ×10.

• Crossovereasy · single-reactorInputs: α Gold (Au), β Platinum (Pt). Output: ψ Platinum (Pt) ×10, ω Gold (Au) ×10.

Inputs: α Gold (Au), β Platinum (Pt). Output: ψ Platinum (Pt) ×10, ω Gold (Au) ×10.

• Every Day is the First Dayeasy · single-reactorInputs: α Hydrogen (H), β Carbon (C). Output: ψ Acetylene (C2H2) ×10.

Inputs: α Hydrogen (H), β Carbon (C). Output: ψ Acetylene (C2H2) ×10.

• Best Left Unansweredeasy · single-reactorInputs: α Nitrogen (N). Output: ψ Nitrogen (N2) ×10.

Inputs: α Nitrogen (N). Output: ψ Nitrogen (N2) ×10.

• Double Bondseasy · single-reactorInputs: α Oxygen (O). Output: ψ Oxygen (O2) ×10.

Inputs: α Oxygen (O). Output: ψ Oxygen (O2) ×10.

• Split Before Bondingeasy · single-reactorInputs: α Hydrogen (H2), β Oxygen (O2). Output: ψ Hydrogen Peroxide (H2O2) ×10.

Inputs: α Hydrogen (H2), β Oxygen (O2). Output: ψ Hydrogen Peroxide (H2O2) ×10.

• No Stomach for Lunchhard · single-reactorInputs: α Ethane (C2H6). Output: ψ Ethylene (C2H4) ×10, ω Hydrogen (H2) ×10.

Inputs: α Ethane (C2H6). Output: ψ Ethylene (C2H4) ×10, ω Hydrogen (H2) ×10.

• An Introduction to Bondingeasy · single-reactorInputs: α Silver (Ag), β Fluorine (F). Output: ψ Silver (I) Fluoride (AgF) ×10.

Inputs: α Silver (Ag), β Fluorine (F). Output: ψ Silver (I) Fluoride (AgF) ×10.

• Slightly Differenteasy · single-reactorInputs: β Hydrogen (H2). Output: ω Hydrogen (H2) ×10.

Inputs: β Hydrogen (H2). Output: ω Hydrogen (H2) ×10.

• Like a Bossmedium · single-reactorInputs: α Hydrogen (H), β Carbon (C). Output: ψ Methane (CH4) ×10.

Inputs: α Hydrogen (H), β Carbon (C). Output: ψ Methane (CH4) ×10.

• I Told You Sohard · single-reactorInputs: α Carbon (C), β Hydrogen (H). Output: ψ Benzene (C6H6) ×10.

Inputs: α Carbon (C), β Hydrogen (H). Output: ψ Benzene (C6H6) ×10.

• Moustachium 608hard · single-reactorInputs: α Australium (Av), β Australium (Av). Output: ψ Moustachium 608 (Av8) ×10.

Inputs: α Australium (Av), β Australium (Av). Output: ψ Moustachium 608 (Av8) ×10.

• It Takes Threeeasy · single-reactorInputs: α Hydrogen (H), β Nitrogen (N). Output: ψ Ammonia (NH3) ×10.

Inputs: α Hydrogen (H), β Nitrogen (N). Output: ψ Ammonia (NH3) ×10.

• A Brief History of SpaceChemeasy · single-reactorInputs: α Hydrogen (H), β Chlorine (Cl). Output: ω Hydrochloric Acid (HCl) ×10.

Inputs: α Hydrogen (H), β Chlorine (Cl). Output: ω Hydrochloric Acid (HCl) ×10.

• Special Assignmenthard · single-reactorInputs: α Ethylene (C2H4), β Hydroxyl Radical (OH). Output: ψ Ethanol (C2H5OH) ×10, ω Acetaldehyde (CH3CHO) ×10.

Inputs: α Ethylene (C2H4), β Hydroxyl Radical (OH). Output: ψ Ethanol (C2H5OH) ×10, ω Acetaldehyde (CH3CHO) ×10.

• Of Pancakes and Spaceshipseasy · single-reactorInputs: α Oxygen (O2). Output: ψ Oxygen (O2) ×10.

Inputs: α Oxygen (O2). Output: ψ Oxygen (O2) ×10.

• Moustachium 604medium · single-reactorInputs: α Australium (Av), β Australium (Av). Output: ψ Moustachium 604 (Av4) ×10.

Inputs: α Australium (Av), β Australium (Av). Output: ψ Moustachium 604 (Av4) ×10.

• Moustachium 602easy · single-reactorInputs: α Australium (Av), β Australium (Av). Output: ψ Moustachium 602 (Av2) ×10.

Inputs: α Australium (Av), β Australium (Av). Output: ψ Moustachium 602 (Av2) ×10.

• Removing Bondseasy · single-reactorInputs: α Fluorine (F2). Output: ψ Fluorine (F) ×10.

Inputs: α Fluorine (F2). Output: ψ Fluorine (F) ×10.

• Sacré Bleu!medium · single-reactorInputs: α Hydrogen Cyanide (HCN), β Methane (CH4). Output: ψ Hydrogen (H2) ×10, ω Acetonitrile (CH3CN) ×10.

Inputs: α Hydrogen Cyanide (HCN), β Methane (CH4). Output: ψ Hydrogen (H2) ×10, ω Acetonitrile (CH3CN) ×10.

• Random Oxidesmedium · single-reactor · random inputsRandom inputs: α Oxygen (O), β Zinc (Zn) / Titanium (Ti). Output: ψ Titanium Dioxide (TiO2) ×10, ω Zinc Oxide (ZnO) ×10.

Random inputs: α Oxygen (O), β Zinc (Zn) / Titanium (Ti). Output: ψ Titanium Dioxide (TiO2) ×10, ω Zinc Oxide (ZnO) ×10.

• Right All Alonghard · single-reactorInputs: α Carbon (C), β Fluorine (F). Output: ψ Polytetrafluoroethylene ((C2F4)n) ×10.

Inputs: α Carbon (C), β Fluorine (F). Output: ψ Polytetrafluoroethylene ((C2F4)n) ×10.

• Prelude to a Migrainemedium · single-reactor · random inputsRandom inputs: α Oxygen (O) / Oxygen (O2). Output: ψ Oxygen (O) ×20.

Random inputs: α Oxygen (O) / Oxygen (O2). Output: ψ Oxygen (O) ×20.

• Danger Zonemedium · single-reactor · random inputsRandom inputs: α Hydrogen (H) / Hydrogen (H2). Output: ψ Hydrogen (H2) ×20.

Random inputs: α Hydrogen (H) / Hydrogen (H2). Output: ψ Hydrogen (H2) ×20.

• Accidents Happenhard · single-reactor · random inputsRandom inputs: α Phosphorus (P) / Silicon (Si), β Hydrogen (H2). Output: ψ Phosphine (PH3) ×10, ω Silane (SiH4) ×10.

Random inputs: α Phosphorus (P) / Silicon (Si), β Hydrogen (H2). Output: ψ Phosphine (PH3) ×10, ω Silane (SiH4) ×10.

• An Introduction to Sensingmedium · single-reactor · random inputsRandom inputs: α Neon (Ne) / Argon (Ar). Output: ψ Neon (Ne) ×10, ω Argon (Ar) ×10.

Random inputs: α Neon (Ne) / Argon (Ar). Output: ψ Neon (Ne) ×10, ω Argon (Ar) ×10.

• Suspicious Behaviorhard · single-reactorInputs: α Carbon (C), β Δ-Dihydrogen (ΔH2). Output: ψ Δ-Pseudopolyethylene ((Δ2C2H4)n) ×10.

Inputs: α Carbon (C), β Δ-Dihydrogen (ΔH2). Output: ψ Δ-Pseudopolyethylene ((Δ2C2H4)n) ×10.

• The Plot Thickensmedium · single-reactorInputs: α Carbon (C), β Hydrogen (H). Output: ψ Polyethylene ((C2H4)n) ×10.

Inputs: α Carbon (C), β Hydrogen (H). Output: ψ Polyethylene ((C2H4)n) ×10.

• The Blue Danubehard · single-reactor · random inputsRandom inputs: α Hydrogen (H) / Hydrogen (H2). Output: ψ Hydrogen (H2) ×20.

Random inputs: α Hydrogen (H) / Hydrogen (H2). Output: ψ Hydrogen (H2) ×20.

• No Employment Record Foundhard · single-reactor · random inputsRandom inputs: α Boron (B) / Aluminum (Al) / Silicon (Si), β Hydrogen (H). Output: ψ Hydrochloric Acid (HCl) ×20.

Random inputs: α Boron (B) / Aluminum (Al) / Silicon (Si), β Hydrogen (H). Output: ψ Hydrochloric Acid (HCl) ×20.

• Unknown Sendermedium · single-reactorInputs: α Hydrogen (H). Output: ψ Neon (Ne) ×10.

Inputs: α Hydrogen (H). Output: ψ Neon (Ne) ×10.

• Under the Icemedium · single-reactorInputs: α Hydrogen (H), β Nitrogen (N). Output: ψ Nitrous Oxide (N2O) ×10.

Inputs: α Hydrogen (H), β Nitrogen (N). Output: ψ Nitrous Oxide (N2O) ×10.

• Ice to Meet Youmedium · single-reactorInputs: α Hydrogen (H), β Iron (II) Oxide (FeO). Output: ψ Cobalt (II) Oxide (CoO) ×10.

Inputs: α Hydrogen (H), β Iron (II) Oxide (FeO). Output: ψ Cobalt (II) Oxide (CoO) ×10.

• QT-3hard · single-reactorInputs: α Acetylene (C2H2). Output: ω Acetylene (C2H2) ×10.

Inputs: α Acetylene (C2H2). Output: ω Acetylene (C2H2) ×10.

• QT-1medium · single-reactorInputs: α Helium (He). Output: ω Helium (He) ×10.

Inputs: α Helium (He). Output: ω Helium (He) ×10.

• QT-2hard · single-reactorInputs: α Chlorine (Cl2). Output: ω Chlorine (Cl2) ×10.

Inputs: α Chlorine (Cl2). Output: ω Chlorine (Cl2) ×10.

• QT-4hard · single-reactorInputs: α Methane (CH4), β Carbon (C). Output: ω Methane (CH4) ×10.

Inputs: α Methane (CH4), β Carbon (C). Output: ω Methane (CH4) ×10.

• An Introduction to Pipelineseasy · 0 reactors (pipes only)Inputs: α Benzene (C6H6), β Acetonitrile (CH3CN). Output: ψ Acetonitrile (CH3CN) ×10, ω Benzene (C6H6) ×10.

Inputs: α Benzene (C6H6), β Acetonitrile (CH3CN). Output: ψ Acetonitrile (CH3CN) ×10, ω Benzene (C6H6) ×10.

• No Ordinary Headachemedium · multi-reactor · random inputsRandom inputs: α Nitrogen (N2) / Oxygen (O2). Output: ψ Nitric Oxide (NO) ×10.

Random inputs: α Nitrogen (N2) / Oxygen (O2). Output: ψ Nitric Oxide (NO) ×10.

• Challenge: Applied Fusionmedium · multi-reactorInputs: α Water (H2O). Output: ψ Phosphoric Acid (H3PO4) ×10.

Inputs: α Water (H2O). Output: ψ Phosphoric Acid (H3PO4) ×10.

• Ω-Pseudoethynehard · multi-reactor · random inputsRandom inputs: α Carbomega (Ω / C), β Hydrogen (H2). Output: ψ Ω-Pseudoethyne (ΩH3CH) ×40.

Random inputs: α Carbomega (Ω / C), β Hydrogen (H2). Output: ψ Ω-Pseudoethyne (ΩH3CH) ×40.

• No Thanks Necessarymedium · multi-reactor · random inputsRandom inputs: α Nitrogen (N2) / Oxygen (O2), β Water (H2O) / Sodium Chloride (NaCl). Output: ψ Nitric Acid (HNO3) ×5, ω Sodium Nitrate (NaNO3) ×5.

Random inputs: α Nitrogen (N2) / Oxygen (O2), β Water (H2O) / Sodium Chloride (NaCl). Output: ψ Nitric Acid (HNO3) ×5, ω Sodium Nitrate (NaNO3) ×5.

• Precursor Compoundshard · multi-reactor · random inputsRandom inputs: α Carbomega (Ω / C), β Hydrogen (H2). Output: ψ Ω-Acetylene (Ω2H2) ×10, ω Carbon Ring (C6) ×10.

Random inputs: α Carbomega (Ω / C), β Hydrogen (H2). Output: ψ Ω-Acetylene (Ω2H2) ×10, ω Carbon Ring (C6) ×10.

• Challenge: Going Greenmedium · multi-reactor · random inputsRandom inputs: α Carbon Dioxide (CO2) / Hydrogen Sulfide (H2S). Output: ψ Sulfuric Acid (H2SO4) ×10, ω Carbon (C) ×10.

Random inputs: α Carbon Dioxide (CO2) / Hydrogen Sulfide (H2S). Output: ψ Sulfuric Acid (H2SO4) ×10, ω Carbon (C) ×10.

• Teleportershard · multi-reactorInputs: α Tungsten Trioxide (W / O). Output: ψ Tungsten Nanospike ((W)n) ×10.

Inputs: α Tungsten Trioxide (W / O). Output: ψ Tungsten Nanospike ((W)n) ×10.

• Challenge: In-Place Swapeasy · multi-reactorInputs: α Nitrogen (N2), β Methane (CH4). Output: ψ Ammonia (NH3) ×10, ω Hydrogen Cyanide (HCN) ×10.

Inputs: α Nitrogen (N2), β Methane (CH4). Output: ψ Ammonia (NH3) ×10, ω Hydrogen Cyanide (HCN) ×10.

• There's Something in the Fishcakeeasy · 1 reactor + pipesInputs: α Hydrogen (H), β Chlorine (Cl). Output: ψ Hydrochloric Acid (HCl) ×10.

Inputs: α Hydrogen (H), β Chlorine (Cl). Output: ψ Hydrochloric Acid (HCl) ×10.

• Fallingmedium · multi-reactorInputs: α Water (H2O), β Krypton (Kr). Output: ψ Yttrium (Y) ×5, ω Zirconium (Zr) ×5, ξ Niobium (Nb) ×5.

Inputs: α Water (H2O), β Krypton (Kr). Output: ψ Yttrium (Y) ×5, ω Zirconium (Zr) ×5, ξ Niobium (Nb) ×5.

• Sleepless on Sernimir IVeasy · multi-reactorInputs: α Hydrogen (H), β Oxygen (O), γ Carbon (C). Output: ψ Formaldehyde (CH2O) ×10.

Inputs: α Hydrogen (H), β Oxygen (O), γ Carbon (C). Output: ψ Formaldehyde (CH2O) ×10.

• Nothing Workseasy · multi-reactorInputs: α Methane (CH4), β Nitrogen (N2). Output: ψ Hydrogen Cyanide (HCN) ×10, ω Hydrogen (H2) ×10.

Inputs: α Methane (CH4), β Nitrogen (N2). Output: ψ Hydrogen Cyanide (HCN) ×10, ω Hydrogen (H2) ×10.

• Σ-Ethylenehard · multi-reactorInputs: α Hydrogen (H2), β Diatomic Sigma (Σ2). Output: ψ Σ-Ethylene (Σ2H4) ×20, ω Ethylene (C2H4) ×20.

Inputs: α Hydrogen (H2), β Diatomic Sigma (Σ2). Output: ψ Σ-Ethylene (Σ2H4) ×20, ω Ethylene (C2H4) ×20.

• Settling into the Routineeasy · multi-reactorInputs: α Methane (CH4). Output: ψ Hydrogen (H2) ×10, ω Ethane (C2H6) ×10.

Inputs: α Methane (CH4). Output: ψ Hydrogen (H2) ×10, ω Ethane (C2H6) ×10.

• Molecular Foundrymedium · multi-reactor · random inputsRandom inputs: α Iron Oxide (Fe / O) / Carbon (C). Output: ψ Cementite (Fe / C) ×10.

Random inputs: α Iron Oxide (Fe / O) / Carbon (C). Output: ψ Cementite (Fe / C) ×10.

• Gas Works Parkhard · multi-reactor · random inputsRandom inputs: α Carbon (C), β Water (H2O). Output: ψ Methane (CH4) ×10, ω Carbon Dioxide (CO2) ×10.

Random inputs: α Carbon (C), β Water (H2O). Output: ψ Methane (CH4) ×10, ω Carbon Dioxide (CO2) ×10.

• Challenge: KOHCTPYKTOPhard · multi-reactorInputs: α Silicon Dioxide (Si / O), β Arsenic (As), γ Boron (B). Output: ψ Silicon Fragment (Si) ×10, ω Silicon Fragment (+) (Si / B) ×10, ξ Silicon Fragment (-) (Si / As) ×10.

Inputs: α Silicon Dioxide (Si / O), β Arsenic (As), γ Boron (B). Output: ψ Silicon Fragment (Si) ×10, ω Silicon Fragment (+) (Si / B) ×10, ξ Silicon Fragment (-) (Si / As) ×10.

Methods

Suite. Every non-boss puzzle in the main campaign (46), the 63 Corvi expansion (6), and the Team Fortress 2 levels (3): 37 single-reactor research puzzles and 18 multi-reactor production puzzles, split into 19 easy, 18 medium, and 18 hard. Easy, medium, and hard puzzles had 10-, 40-, and 60-minute time limits. Every model had the same limit on a given puzzle. Time limits differ by difficulty, so difficulty and search time can’t be separated.

Scoring. Each model–puzzle pair contributes one selected attempt, with no best-of-attempts scoring, so solve rates are pass@1: a puzzle counts as solved when that one episode saved a solution that replays in a fresh game. Unsolved puzzles stay in every average and score zero.

New frontier points. The reference is the community archive as of October 18, 2026, without solutions the community flags as precognitive or as relying on bugs. Ties don’t count. Our count pools and deduplicates points across models: 20 on 11 puzzles.

Replay. Every published point is rebuilt from the model’s edits and replayed in a fresh game. SChem’s precognition check then plays it on other random seeds and rejects it if it is likely to succeed on fewer than 20% of them. The check is statistical, so it is strong evidence, not proof.

Record proximity, sum score, hypervolume, and frontier coverage. Record proximity is the community record divided by the model’s best score, averaged across the three dimensions and all puzzles, ×100; each dimension’s best can come from a different solution. Sum score, adapted from Opus Magnum Bench, compares the best single solution’s combined cycles + reactors + symbols against the lowest sum in the community archive, ×100. 3D hypervolume measures the volume of score space dominated by the model’s Pareto frontier up to a reference box at 2× the community’s maximum scores, divided by the community frontier’s hypervolume, ×100. Frontier coverage (reciprocal multiplicative ε) measures how closely the model’s frontier covers every point on the community Pareto curve at once: for each community point, it finds the model point with the highest minimum coordinate ratio across the three scores, then takes the worst case across the community frontier, ×100. One tutorial’s records are zero reactors and zero symbols; matching a zero counts as 1 and anything else as 0.

Limits. One episode per pair compares the configurations as tested and doesn’t measure repeated-run variability. Spread between puzzles is variation across tasks. Public puzzles and solutions may overlap with training data. Mechanic groups are our own labels.

Solutions. Every model Pareto point’s solution is public under the MIT License, in the community’s SpaceChem Community Edition export format and named like the archive’s files (<puzzle>.<cycles>-<reactors>-<symbols>.txt). They are mirrored as one file per solution on GitHub and as a table on Hugging Face.

Resources. Costs are estimated from recorded October 2026 API list prices, including cache discounts. Tokens accumulate across responses, so repeated input counts again, and tokenizers differ. Game calls count individual dispatched tools, including errors, and exclude the automatic final submission.

Settings by model

Reasoning effort, for all 55 episodes

Related work

TIS-100 Bench is our companion benchmark on Zachtronics’ assembly game, using the same 16 models, 55-puzzle pass@1 structure, and Pareto-archive evaluation. Where models solved 82% of TIS-100 episodes, found 133 new frontier points, and broke three cycle records outright, the shift to 2D waldo routing and multi-reactor pipelines in SpaceChem cut overall solve rate to 56%, yielded 20 new tradeoff points (with no outright 1D records), and separated models that tied on 1D assembly. Opus Magnum Bench evaluates agents on a third Zachtronics puzzle game using a single-solution sum score (cost + cycles + area), which we include alongside record proximity, 3D hypervolume, and frontier coverage.

Cite

        @misc{wincon2026spacechembench,
  title  = {SpaceChem Bench: Can LLMs Beat Humans at 2D Spatial Programming?},
  author = {{Wincon}},
  year   = {2026},
  month  = nov,
  url    = {https://spacechem.wincon.dev/},
  note   = {Benchmark snapshot 2026-11-04-v1}
}
      

Credits

Thanks to Zachtronics, the SpaceChem players, and the maintainers of the community archive, SChem, and the leaderboard bot. Their solutions and tools make this comparison possible. Thanks also to 12345ieee for taking the models’ solutions further.

SpaceChem, TIS-100, and Opus Magnum are trademarks of Zachtronics. Team Fortress 2 is a trademark of Valve. Model names and logos are trademarks of their respective owners and appear only to identify the models tested. Wincon Research is not affiliated with or endorsed by Zachtronics, Valve, or any model provider.

Have a question or want to share feedback? Get in touch ↗