Newshot

Show HN: Grand Treasure Island – GTA-style game on a real island, modeled by AI

Original text(via HN AI)
  • 01Start from data, not from promptsThe terrain comes from USGS lidar and the road layout from OpenStreetMap. That gave us every road, path and fence, and the footprint of every building. Code bakes all of it into the island, and agents fix the parts that don't look good.USGS lidar terrain, OpenStreetMap roads and building footprints

Start from data, not from prompts

The terrain comes from USGS lidar and the road layout from OpenStreetMap. That gave us every road, path and fence, and the footprint of every building. Code bakes all of it into the island, and agents fix the parts that don't look good.

图表:USGS lidar terrain, OpenStreetMap roads and building footprints

  • 02An agent teamI talk to one agent, the foreman. The foreman doesn't build anything itself. It hands each job to a team of agents and reviews what comes back. The whole team is in the repo.Me ↗Sets the bar, playtests, takes the photosThe foreman ↗Hands out every job and reviews what comes back. Never builds.Builder ↗Models the buildings and writes the codeResearcher ↗Finds a dated photo for every sideCode reviewer ↗Reads every change for sloppy codeBar raiser ↗Compares the game with photos and sends work backProgress reviewer ↗Asks if it was the right thing to work onEfficiency reviewer ↗Watches the team for wasted workWriter ↗Keeps the rules and lessons up to dateMeet the team →

An agent team

I talk to one agent, the foreman. The foreman doesn't build anything itself. It hands each job to a team of agents and reviews what comes back. The whole team is in the repo.

Sets the bar, playtests, takes the photos

The foreman ↗

Hands out every job and reviews what comes back. Never builds.

Models the buildings and writes the code

Researcher ↗

Finds a dated photo for every side

Code reviewer ↗

Reads every change for sloppy code

Bar raiser ↗

Compares the game with photos and sends work back

Progress reviewer ↗

Asks if it was the right thing to work on

Efficiency reviewer ↗

Watches the team for wasted work

Keeps the rules and lessons up to date

Meet the team →

  • 03Agents check agentsAgents are still very willing to pass mediocre work as “fits the spec.” So every change gets looked at in the actual game by agents that didn't make it. One of them is the bar raiser: it compares the game with photos of the real place and sends work back until it gets closer. The dev box runs up to four copies of the game at once, so checking keeps up with building.Sent back“Two kept doors on photographed faces contradict the dated photos.”Passed“All three must-fixes are done, and nothing from the earlier round has come back.”The shed at 899 13th St, same camera, two rounds of the same review on 2026-11-15

Agents check agents

Agents are still very willing to pass mediocre work as “fits the spec.” So every change gets looked at in the actual game by agents that didn't make it. One of them is the bar raiser: it compares the game with photos of the real place and sends work back until it gets closer. The dev box runs up to four copies of the game at once, so checking keeps up with building.

The shed in the game with two roll-up doors on its long wall
The shed in the game with two roll-up doors on its long wall

“Two kept doors on photographed faces contradict the dated photos.”

The same shed with the doors fixed to match the photos
The same shed with the doors fixed to match the photos

“All three must-fixes are done, and nothing from the earlier round has come back.”

  • 04Breadth and depthWe had to model every building, and some buildings deserve a lot more. So each building has a level of fidelity: first the shape, then doors, windows and roofs, then details, then materials. The agents finish the whole island at a low level first, and then we keep pushing the buildings that matter most.

Breadth and depth

We had to model every building, and some buildings deserve a lot more. So each building has a level of fidelity: first the shape, then doors, windows and roofs, then details, then materials. The agents finish the whole island at a low level first, and then we keep pushing the buildings that matter most.

  • 05Systems, not hand placementWe didn't let agents hand-place every tree. Procedural systems spread plants across beds, lay out mulch and scatter vegetation. Same for buildings: a family is a group of buildings that share one design, like the townhouses with garages on Sturgeon Street. We designed each family once, and every building in it gets its own variation: colors, shutters, roofs. Windows and doors go on a grid that a system fills in, instead of an agent placing them by eye.One housing design, four instances on Sturgeon Street490 Avenue of the Palms: the window grid, measured from photos, drawn over the facadeHousing family: 113 · Midrise family: 61 · One-offs: 25

Systems, not hand placement

We didn't let agents hand-place every tree. Procedural systems spread plants across beds, lay out mulch and scatter vegetation. Same for buildings: a family is a group of buildings that share one design, like the townhouses with garages on Sturgeon Street. We designed each family once, and every building in it gets its own variation: colors, shutters, roofs. Windows and doors go on a grid that a system fills in, instead of an agent placing them by eye.

Four houses on Sturgeon Street, all built from the same housing family with different colors, shutters and roofs
Four houses on Sturgeon Street, all built from the same housing family with different colors, shutters and roofs
490 Avenue of the Palms with its window grid drawn over the facade
490 Avenue of the Palms with its window grid drawn over the facade

图表:Housing family: 113 · Midrise family: 61 · One-offs: 25

  • 06A mix of methodsPhotos never make it into the game. They're only a source. The inputs are measured open data and photos, public and my own. Then it's a mix of methods: code turns the data into the island, agents model things like the pelicans, and procedural systems vary the trees, the flower beds and the buildings. The output is the island.Agent artwork: pelicans on the breakwaterProcedural: blue gums, each one variedThe output: the island

A mix of methods

Photos never make it into the game. They're only a source. The inputs are measured open data and photos, public and my own. Then it's a mix of methods: code turns the data into the island, agents model things like the pelicans, and procedural systems vary the trees, the flower beds and the buildings. The output is the island.

Brown pelicans and gulls on the breakwater
Brown pelicans and gulls on the breakwater
A double row of young blue gums in mulch beds in Cityside Park, each a little different
A double row of young blue gums in mulch beds in Cityside Park, each a little different
The whole island in the game from high above the north-east
The whole island in the game from high above the north-east
  • 07Better input, better outputBuildings modeled only from public photos came out mediocre. Once I added my own video footage, they got a lot better.Before: public imagery onlyMy photoAfter: built with my photosMaceo May, city side

Better input, better output

Buildings modeled only from public photos came out mediocre. Once I added my own video footage, they got a lot better.

Maceo May's city side in the game in September: a blank grey wall
Maceo May's city side in the game in September: a blank grey wall

Before: public imagery only

My photo of Maceo May's city side: dark cladding, cream frames, glazed ground floor
My photo of Maceo May's city side: dark cladding, cream frames, glazed ground floor
Maceo May's city side in the game in November, matching the photo
Maceo May's city side in the game in November, matching the photo

After: built with my photos

  • 08ChesterClaude animated Chester, the main character, with a rubber-hose animation system. That worked shockingly well. It was the biggest surprise of the project.Chester's rubber-hose arms on the jetpack

Chester

Claude animated Chester, the main character, with a rubber-hose animation system. That worked shockingly well. It was the biggest surprise of the project.

图表:Chester's rubber-hose arms on the jetpack

  • 09Why GodotWe used an established game engine instead of hand-rolling a web renderer. Godot gave us physics for the player and the cars, a modern renderer, a game camera, and builds for Mac, Windows, Linux and the browser from one project. All of that would have been pretty hard to build ourselves.

Why Godot

We used an established game engine instead of hand-rolling a web renderer. Godot gave us physics for the player and the cars, a modern renderer, a game camera, and builds for Mac, Windows, Linux and the browser from one project. All of that would have been pretty hard to build ourselves.

  • 10How far can this go?I still don't know. Right now we're at a GTA 3, GTA 4 level. But every big jump so far came from better input and a higher bar, not a better model: Maceo May went from a grey box to a building you recognize once it had my photos. What runs out is time, tokens and photos. When my agents say “the models aren't there yet”, they never show it. So I haven't seen anything that says we can't push further.

How far can this go?

I still don't know. Right now we're at a GTA 3, GTA 4 level. But every big jump so far came from better input and a higher bar, not a better model: Maceo May went from a grey box to a building you recognize once it had my photos. What runs out is time, tokens and photos. When my agents say “the models aren't there yet”, they never show it. So I haven't seen anything that says we can't push further.

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by HN AI
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗