State machines are perfect for simple enemies, but as behaviours multiply the number of transitions grows quickly and the diagram turns into a cobweb. Behavior trees offer an alternative widely used in the industry: instead of describing states and the moves between them, they describe hierarchical decisions built from reusable blocks.
The basic structure
A behavior tree is a tree of nodes. At regular intervals, often several times per second, the system «ticks» it starting from the root. Each node, when run, returns one of three results:
- success: the task succeeded;
- failure: the task is not possible or did not succeed;
- running: the task needs more time, for example a movement not yet finished.
Nodes fall into two broad families: leaves, which do something or check a condition, and composite nodes, which decide the order in which to run their children.
The main composite nodes
Sequence
Runs its children from left to right. It stops and returns failure at the first child that fails; it returns success only if all succeed. It corresponds to «and then»:
Sequence
├─ Condition: player in range?
├─ Action: turn towards the player
└─ Action: attack
Selector
Runs its children from left to right until one succeeds. It returns success at the first success and failure only if all fail. It corresponds to «otherwise», and it is the natural way to express priorities:
Selector
├─ Sequence "flee if wounded"
├─ Sequence "attack if close"
├─ Sequence "chase if visible"
└─ Action: patrol
The order of the children is the priority list: the enemy first tries to flee, then to attack, then to chase, and only if none of these applies does it patrol.
Parallel
Runs several children at once, for example moving to cover while shooting. The success rule must be defined: either one child succeeding is enough, or all must succeed.
Decorators
A decorator has a single child and modifies its result or execution. Some widely used examples:
| Decorator | Effect |
|---|---|
| Invert | Turns success into failure and vice versa |
| Repeat | Runs the child a fixed number of times or forever |
| Cooldown | Prevents the child from running again before a certain time |
| Time limit | Makes the child fail if it takes too long |
| Always succeed | Returns success whatever the outcome |
A cooldown, for example, lets you say «throw a grenade, but no more than one every eight seconds» without adding variables to the attack logic.
The blackboard
Nodes need to share information: the last sighting position, the current target, the chosen cover. Instead of wiring nodes together directly, a blackboard is used: a table of keys and values accessible to the whole tree. A «find cover» action writes the position to the cover key, and a later «go to» action reads it.
This keeps nodes independent and reusable: the same «go to» action works with whatever key it is given.
The role of the «running» status
The «running» status is what sets a behavior tree apart from a simple series of conditions. An action like «walk to the point» does not finish in a single update: it returns «running» until the character arrives. On the next tick the tree resumes from there.
There is, however, an important design choice. If higher-priority branches are also re-evaluated on every tick, the enemy can interrupt its walk as soon as it sees the player. If instead execution always resumes from the running node, behaviour is more stable but less reactive. Many implementations offer both modes, with nodes that can «abort» lower-priority ones when their condition becomes true.
Behavior tree or state machine?
| Aspect | State machine | Behavior tree |
|---|---|---|
| Simple behaviours | Very clear | Slightly more verbose |
| Many behaviours | Transitions hard to manage | Just add branches |
| Priorities | Encoded in transitions | Expressed by child order |
| Reuse | States reusable with effort | Reusable subtrees |
| Debugging | Easy to see the current state | Need to visualise the path through the tree |
The two techniques are not mutually exclusive. It is common to use a state machine for broad modes, such as exploration, combat and alert, and a behavior tree inside each one.
Practical advice
- Keep leaves small: one condition or one action per node.
- Give subtrees clear names and reuse them across different enemies.
- Build or adopt a visualiser that shows in real time which nodes are active: without one, debugging becomes hard.
- Do not tick the tree every frame unless needed: a few updates per second are enough for most enemies.
In short
- A behavior tree organises decisions as a tree of reusable nodes.
- Every node returns success, failure or running.
- Sequence means «and then», selector means «otherwise» and expresses priorities.
- Decorators and the blackboard add control and data sharing.
- Behavior trees and state machines can be combined effectively.
