AI Agent Pipeline: A Solo Unreal Engine 5 Remake
Personal project · Solo lead · Agent orchestration, pipeline and infrastructure design
Personal project · Solo lead · Agent orchestration, pipeline and infrastructure design
I’m remaking a game on my own in Unreal Engine 5, as a private project built from my personal archive of assets and reference material. It will never be released; its purpose is the method. AI agents do most of the production work, and my job is to design and run the system they work inside: how work is planned, handed out, checked, and recorded. I find building AI pipelines for game development genuinely fascinating, and this project is where I push the idea as far as I can.
Tools: Cursor as the agent environment (one chat per task, plus a coordinator chat) · Claude Opus and Sonnet, plus a fast Composer-class model · Gemini (concept art) · Tripo (image-to-3D) · Unreal Engine 5 · Perforce · Python · PowerShell
So far: ~210 numbered tasks written · ~120 written reports · ~1,090 automated tests · a reusable backup library at v2.2.0, running on two projects
Friction: A remake needs planning, asset import, implementation, testing, documentation, builds, and backups, and that’s normally spread across a team. One person doing all of that by hand doesn’t scale. Handing it to AI agents creates new problems:
Fix: I run the project like a small studio. One coordinating agent manages it, specialist agents do the work, and nothing is accepted until it passes verification.
Impact: One person can run a game-scale production while every change stays reviewable.
Friction: One general-purpose agent doing everything is either too expensive for routine work or too unreliable for hard calls. If the same agent both does the work and judges it, nobody is really checking.
Fix: I split the work into roles and match each one to a model tier.
| Tier | Model | Used for |
|---|---|---|
| Top reasoning | Claude Opus | Design, root-cause analysis, extracting rules from many past results |
| Mid | Claude Sonnet | Most engine implementation: reliable multi-step edits with tests |
| Fast | Composer-class | Offline cataloguing, summaries, mechanical syncs, documentation |
The coordinator sets the tier in each prompt by one rule: no top-tier prices for mechanical work, and no cheap tier anywhere a miss would be hard to detect.
Impact: Expensive reasoning goes where a wrong judgment would be costly and only show up late. Routine work stays cheap and fast, and the coordinator never grades its own work.
One task, from queued prompt to verified record.
Friction: Open-ended prompts let agents wander out of scope, run past the point where they should stop, and report checks they never ran.
Fix: Every task starts from the same template. Condensed:
Kickoff: <task id> - <one-line goal>
Model / time box / what must not run at the same time
First action: rename the chat to the task id (traceability)
Why: the problem, with the evidence behind it
Read first: specific files, reports and earlier lessons
What to do: numbered parts; evidence before any change; smallest correct fix
Scope: allowed paths and tools; everything else is off-limits
Stop rules: stop and report if input is missing, a part overruns,
or the estimate runs >50% over the time box
Done check: tests, generated-doc check, clean version control, captured frames
Report: what was done · numbers (each with the command that produced it)
· files · deviations · NOT VERIFIED · open questions · assumed values
· how the human checks it
Two rules do most of the work. Every number in a report must come with the command that produced it. And “Not verified” is a required section, so a specialist can’t imply a check it didn’t run.
Impact: Each task has clear edges and a predictable report, and I can trace it back from its chat name to its changelist.
Friction: “The agent says it’s done” isn’t evidence. Even a report with passing tests can hide problems.
Fix: The coordinator runs the same checklist on every submission before accepting it:
A short list of known, tracked test failures is accepted; any new failure blocks acceptance.
Impact: Nothing enters the project on the strength of an agent’s own word, and every accepted change leaves a record I can audit.
A submission looked complete and its tests passed. The coordinator’s reconcile showed four source files modified on disk but missing from the changelist, so the shared depot head wouldn’t have compiled. The coordinator submitted them in a separate change and noted it in the review. That’s the reason step 4 exists: the gate checks what actually landed in version control, not what the report says.
Once, the coordinator created a changelist from the default list and swept another running task’s files into it. The rule since then: every changelist starts from an empty spec, and each agent moves in only its own exact paths. Every prompt now says so explicitly. When agents run in parallel, mistakes turn into rules, not one-off fixes.
A new visual feature on a level was broken, and two possible causes had been suggested. The prompt required evidence first: a labelled capture and data counts. The specialist found that both causes were real and separate, fixed each with a patch script, added tests, and measured the frame cost against a stated limit. The coordinator re-ran the test groups, checked the submitted files against the report, compared before/after frames, recorded the rules the task produced, and accepted it.
Friction: Importing archive assets by hand produces quiet errors: wrong counts, missing collision, invisible props.
Fix: Scripted export and import, with validation for:
A periodic harvest task folds every fix back into one checklist.
Impact: Each new import starts with every earlier lesson already applied.
Friction: Agent changes can break things nobody looks at again.
Fix: About 1,090 automated tests: unit and data-driven gameplay tests, import-census tests, rendering tests that capture real frames, soak and performance tests with frame-cost limits, and replay tests that check handling against measured reference data.
Impact: Regressions surface at review time, not weeks later.
Friction: Hand-written docs drift from the code, and then agents read the stale version.
Fix: The list of console commands and variables is generated from the source definitions. A --check mode fails when the list is stale, every task runs it before reporting, and the updated file goes in the same change as the code.
Impact: People and agents always read current docs.
Friction: “It works in the editor” isn’t a game.
Fix: One script cooks, stages, and packs the build, and generates a readme listing the current commands and spawnable content. It refuses to run while the editor is open. A smoke script then launches the packaged game, scans the logs for missing assets, and captures frames.
Impact: The smoke test once caught a packaged build with no HUD, because a folder was missing from the cook list. It never reached a playtest.
Nightly local, weekly offsite, weekly restore-and-verify, and a watchdog; the same library sets up each new project.
Friction: A solo project has no studio IT. One bad drive or sync mistake could lose months of work, and setting up backups from scratch for every project wastes time.
Fix: A backup design I turned into a versioned library:
Results go to a status file and the OS event log, and the scheduled tasks coordinate through a shared lock library. The library is versioned and hash-checked, and each project carries its own copy. A new project adopts it by copying the library, filling in a small config, and running the test suite that ships with it.
Impact: Backups are verified, not assumed. The library is at v2.2.0 and runs on two projects with the same behavior, logs, and recovery instructions. When a workflow proves itself, I package it for reuse; I work the same way at the studio.
Friction: A solo project has no concept artists or modelers, so any asset my archive doesn’t cover becomes a bottleneck.
Fix: I generate concept art with Gemini, hand it to Tripo to generate 3D models and textures, and bring the results into UE5 through the same validated import pipeline as every other asset.
Impact: New assets go from concept to in-game in one solo loop, and they pass the same checks as everything else in the project.