Behavior trees to control AI agents?

Two Unrest Bots under a glowing behavior tree: one stands guard, one builds a small island inside the lit leaf

You’ve probably seen the stupid stealth game guard before. It just patrols until it sees you, then chases you, then loses you, then back to patrolling. Same thing every time. It runs on a behavior tree, and game AI has been using those for about 20 years. The whole point is to get predictable behavior out of simple characters.

But coding agents are the opposite, the model is super smart but still doesn’t do the build, doesn’t even look at the error, says it’s done, etc. So what happens when we put that smart model inside the tree that a stealth game guard runs on?

GameGen's live view of the unreal-coder tree, mid-run on ImplementUnrealChanges

We actually do that, in GameGen. The screenshot is unreal-coder, our Unreal coding agent, on run #116. The highlighted node (ImplementUnrealChanges) is where the model is writing C++ right now, everything else is the tree deciding what happens before and after. Every box is a node in the behavior tree.

I’ll walk through the unreal-coder tree, compare it to behavior trees in Unreal Engine, and explain utility trees. We use the same trees to run our NPCs in Awaken, and I’ll get to where all of this fits in the world model we’re building at Unrest.

The problem with “just let the agent loop”

Most coding agents today are just one long LLM conversation where the model is given a task and some tools, and it decides everything. What to read, when to build, when it’s done, whether to commit, etc. It works for small tasks, but on a real project it’s gonna drift, forget to build, or build and ignore the errors, or declare victory on code that doesn’t compile.

Our answer: take control flow away from the model. LLM still does the hard part, which is writing C++ against a codebase it has to understand, but it’s doing it inside a leaf of a tree, and the tree is deciding what happens next.

What the unreal-coder tree does

The Unreal coder in GameGen is C++ only, no Blueprint. It writes source to disk, builds using UnrealBuildTool, checks the running editor via Unreal’s Remote Control API and editor Python, then commits to git. Here’s the tree, flattened:

sequence FetchTaskContext AnalyzeCodebase EnsureGitRepo AnalyzeUnrealTask (LLM, structured output) selector sequence selector [if QA rejected it] AnalyzeQAFeedbackAndFix (LLM) ImplementUnrealChanges (LLM) retry x3 [if a previous build failed] FixValidationFailures (LLM) stage changes BuildAndVerifyUnreal (LLM, up to 80 turns) CheckValidationPassed selector [if out of retries] escalate, report fail PrepareCommit (LLM, falls back to a template message) GitCommitChanges, GitPush AddCompletionComment, ReportVerification, ReportTriggerPass escalate, report fail

GameGen's live tree view zoomed in on a run of unreal-coder: ImplementUnrealChanges is running, with retry, BuildAndVerifyUnreal and CheckValidationPassed still waiting

The same tree live in GameGen. The blue dashed line is the path to the node that is running now.

Some nodes are LLM calls, most are not. Like the standard nodes for fetching the task, staging files, committing, pushing, reporting pass or fail. They all run the same way every time. The model can’t skip the build, the build is a node it has to pass through. It can’t commit broken code either, CheckValidationPassed is in front of the commit. If the build fails 3 times, the retry decorator gives up and the tree escalates, it doesn’t loop forever.

The tree is run in a kanban board. Coder finishes and requests review. The second agent, unreal-simulation, picks it up. It builds the branch itself, runs the automation tests, spawns the actor in the editor, reads the actor properties and the output log, doesn’t trust the coder’s report, and never edits source. If the review fails, it goes back to the coder and the coder’s tree takes the AnalyzeQAFeedbackAndFix branch on the next pass.

GameGen's agent roster: unreal-coder and unreal-simulation both running, each on its own board task

The coder and the QA agent, each on its own task from the board.

The lineage is at the bottom of the screenshot, it’s another benefit of trees. Run #116 is a fork of #112 at tick 2. #112 is a fork of #108 at tick 60. The reason it’s so powerful is because the state is explicit, state = which node is running, what’s on the blackboard, etc. Because of that we can checkpoint a run, stop the daemon, ship a change, and resume. We also do a compatibility check before resuming so we won’t silently resume into the wrong node if the tree changed under the checkpoint, we compare the recorded tree shape against the tree on disk today.

How Unreal does it

Unreal’s behavior trees are the reference most game devs know. Here’s an example from Epic’s docs:

An Unreal Engine behavior tree: a selector choosing between chasing the player and patrolling

Source: Epic Games, Behavior Tree Overview .

In this case the root selector (AI State) will try each child in order left to right. If the blackboard key HasLineOfSight is set, it will run the sequence Chase Player. Which will rotate, chase, and move to the enemy. Otherwise, it patrols. If neither, it will wait a second, and try again.

The Unreal behavior tree editor with a decorator's Observer Aborts setting

Source: Epic Games, Behavior Tree Overview .

Epic’s docs actually list a bunch of differences from a textbook BT,

  • They’re event driven. The tree doesn’t re-evaluate every frame. It waits for a blackboard change and reacts.
  • Conditions are decorators attached to a node, not leaves. The blue Has Line of Sight? box sits on top of the sequence it guards.
  • Observer Aborts let a decorator cancel a running branch the moment its condition changes. In the editor screenshot it’s set to Both, so losing sight of the player aborts the chase and also lets the chase pre-empt a lower-priority patrol.
  • Services run on a timer while their branch is active, for things like re-picking the best target.
  • Instead of full parallel nodes there’s a Simple Parallel: one main task, plus a background subtree.

But it’s tuned for one job: a character in a game world, making decisions many times a second, with a frame budget.

Where our trees differ

Our trees have the same vocabulary as Unreal but almost opposite constraints. Root, sequence, selector, decorators like retry and flip, blackboard.

Time scale. An Unreal task finishes in milliseconds to seconds. Our BuildAndVerifyUnreal leaf can take 80 model turns and several minutes of compiling. Event driven evaluation matters much less when the leaf takes minutes. Our daemon ticks the tree about every 500 ms and nearly all ticks find the same leaf still running.

Leaves think. In Unreal a task is code you wrote. Ours can be an LLM call with context keys pulled from the blackboard and an output schema. So for example AnalyzeUnrealTask has to return a JSON in a fixed format, that’s put on the blackboard for the next node, etc. It’s an action that can succeed or fail. The tree treats the model like any other action.

The tree is data. It’s just .bt.json files in our repo, versioned and diffable, etc. The GameGen app draws it live with each node’s status, so you can pause and step, see why a run is at a particular spot, etc. An Unreal tree is a binary .uasset, and you have to edit it in the editor.

Side effects are real. A commit, push, task status change, etc can’t be undone by aborting a branch. So we lean on checkpoints and resume, and on a policy layer (more on that below), instead of Unreal-style aborts.

Choosing between options is scored. That’s the biggest difference. It gets its own section.

Utility behavior trees

An Unrest Bot picks the highest of three gauges hanging from a behavior tree

A plain selector works by trying its children in order. That’s fine when one option is obviously right to try first. But that breaks down when multiple options are valid, or when the right option depends on the situation.

Unreal classic BTs don’t have a utility selector, the community has been making utility selector plugins for years, but in UE 5.5 they added utility based selection to State Tree. In State Tree the parent can choose the child with the highest utility score. Utility AI is an old idea, in games it’s like: Score each option. Pick the best.

Our trees have a utility-selector node that does the following on each decision.

  1. reads a feature vector from the blackboard,
  2. drops arms that aren’t allowed (more on that in the next section),
  3. scores each remaining arm with a scorer bound to the node,
  4. adds a small inertia bonus to the arm it picked last time, so it doesn’t flip-flop,
  5. picks the highest score and runs that child,
  6. falls back to a declared fallbackChild if the scorer errors.

It’s deterministic, no random exploration, just argmax.

It’s used by our QA agents to decide how to verify each acceptance criterion, it’s called ChooseVerificationStrategy. And has 3 arms.

  • replay a cached script that verified this criterion before,
  • prove it with a unit or static check,
  • explore the app with an agent.

The three arms differ by about five orders of magnitude in cost. The cheapest is replaying a script. The most expensive is agentic exploration which is a long model session. It looks at a record of our past attempts and the evidence each strategy produced for what it cost, then picks one. If the scorer is turned off or fails, node falls back to agentic exploration. Which costs more but never verifies less.

So tree still owns the flow, utility node only chooses how to try something, it never decides whether it passed the criterion or not.

The same trees inside the game: Awaken

Villagers on a moonlit shore, each with a small behavior tree above its head, while two Unrest Bots inspect one tree on a glass panel

Everything so far is agents that build games. Awaken is the other side, agents controlling NPCs, allies, and enemies inside a game. It uses the same tree format and the same engine.

For Unreal games made in GameGen, we have a plugin that runs .bt.json trees inside the game itself. It statically links the Rust behavior tree engine, through a C ABI, and then ticks it once per frame from an actor component. The comparison with Unreal isn’t even academic. We don’t use any of Unreal’s BT assets at all. In these games our tree replaces Unreal’s.

Also we have a different budget than the coding agent, we have to tick it every frame, the coding agent only ticks every half second. The leaves are host actions bound to the actor. The sample tree moves an actor to a target and checks if it arrived. LLM nodes need the daemon, so the embedded engine refuses to load a tree that contains one. A tree that calls an action the game hasn’t bound fails to load and names the missing call, instead of failing quietly mid-game.

Also, Awaken has no chat, you don’t talk to the characters, you watch and manage them. The second plugin draws the Awaken NPC roster and inspector inside the game frame, open with F10 or Awaken.Debug 1 in the console. Any actor running our trees will show up automatically. It shows status, current goal, loaded tree with nodes colored by status, blackboard. There’s also a separate Awaken overlay window that sits on top of a windowed game, and connects to the same agent daemon as our GameGen project.

Same debugging story as the screenshot at the top, moved into the game, when an NPC does something stupid, you see what node it was in, and what it believed when it chose.

Some of this is still ahead. Hooks to let Awaken’s agents act through the in-game SDK are planned, not shipped. Same with drawing the overlay by injecting into games without the plugin.

How this fits Unrest’s world model

People think a world model is some neural net that can dream video frames but that’s not what we’re doing. In the narrow sense it’s a table of counts, stored in a versioned JSON file. For each action it’s seen, how often each outcome happened. We predict the next outcome from that table and have a confidence. Until we have at least 5 real observations, it’s always zero. After that, it’s scaled by calibration based on held out runs. If it hasn’t seen enough, it says so and the caller falls back.

The behavior tree is just one of four symbolic parts around the world model.

  • The symbolic planner breaks a goal into subgoals and operators. It also supplies the reward: subgoals achieved, plus a bonus for the top goal, minus a small cost per tick and a penalty for failing. Nobody writes a separate reward function.
  • The utility behavior tree is the symbolic control flow. It decides what runs and when.
  • The symbolic policy engine decides what is allowed before anything chooses. A policy node in the tree writes the admissible arms, and the utility selector only scores what’s left. A learned score can rank options, but it can’t make a denied option legal.
  • The world model predicts what each action will do, from counts.

Then we have Lou, our small neural decision model, which scores the options the harness has already built and filtered. Lou can add a term to a utility arm’s score, order the planner’s methods, and stop an operator. It doesn’t predict state. It’s not the world model.

Every real run is recorded tick by tick. A trainer mines all of those runs. Then it counts the outcomes, and calibrates on held out episodes. Then it promotes a new model version only if it didn’t get worse on calibration. The world model then feeds back into the tree. Its priors can be features for a utility selector, facts a policy rule uses to refuse an operator it’s measured as unreliable, and the basis for imagined rollouts. Before acting, a simulation scorer plays each option forward through the world model and scores it with the planner’s reward. We only run the simulation scorer if the model has calibration evidence > 0.9. Otherwise it goes straight to the fallback scorer for the whole decision.

The only version of the world model that has real artifacts on disk is the count-table rung. The logistic rung exists in code. The richer rungs are design work. Predictions for our bots run in shadow mode, measured but not deciding. I’m not going to claim the world model makes the agents better until we’ve measured it against a run without it.

So, behavior trees to control AI agents?

Yes but the tree isn’t what makes it smart, it’s the dumb part, on purpose. It always runs the build, always checks validation before committing, and gives up after 3 failed builds instead of looping forever. The utility selector chooses how to verify something, but it never decides if it passed. Model gets to be smart inside a leaf and nowhere else.

That split between tree and leaf is why the same trees work for both sides of GameGen: the coding agent that builds the game, and the NPCs in Awaken that live in it. Same file format, same engine, same view of which node it was in when it did something stupid. Where the full harness runs, the policy engine says what’s allowed, the utility selector picks among what’s left, and every tick becomes data the world model learns from.

An Unrest Bot with a kite shield walks its patrol route, a small behavior tree floating above it

A stealth game guard has always known when to stop chasing and go back to patrol. Turns out a coding agent needs to know that too.