Picture a small robot in a maze. Food is scattered around, and so are things that hurt. The robot sees only the 5×5 square of cells around it, has never been trained, and must decide step by step where to go.
The usual modern answer is to train it, letting it fail thousands of times until a neural network learns what works. Embodied Field Intelligence (EFI), a research agent we built at FriendMachine, takes a different route. Its “mind” is a grid of interacting fields rather than a trained neural network, and its foraging version runs with no training episodes at all. It plans with almost entirely local updates, in which each cell of its internal map exchanges information only with its immediate neighbors. On our standard foraging test it scores 97% of the way from a random walker to an agent that sees the entire map and plans perfectly.
This post explains how a pile of local updates can add up to planning, and what we had to design by hand to make that possible.
Two maps in the agent’s head
The agent keeps a grid in its head that matches the world’s grid, and stores two kinds of information on it.
The first answers what is out there. For each kind of target, the agent keeps a belief at every cell about how likely it is that a target sits there. This is a Bayes filter, the standard way to update a guess as evidence arrives. Each step, beliefs spread slightly to neighboring cells, because unobserved things may have moved. Then whatever the 5×5 window shows overrides the guess for those cells. Seeing an empty cell pushes the belief there toward zero.
The second answers where to go, which is harder. Planning usually means search. An algorithm like A* explores paths outward until it finds the best one, and that requires a central process holding the whole map. EFI has no such process, so it needs a way to plan that cells can carry out by talking to their neighbors.
Planning without a planner
That way comes from a corner of control theory. A Markov decision process is the standard model of acting in a world. It has states (here, cells), moves between them, and costs and rewards. Solving one means finding the value of every state, meaning how good it is to be there.
The control theorist Emanuel Todorov identified a family of these problems, called linearly solvable, in which the agent pays a cost for steering away from a default random walk. For this family, it helps to work with a cell’s desirability, which is just its value passed through an exponential, so higher value means higher desirability. The best desirability at each cell then follows a simple rule:
A cell’s desirability is the average desirability of its neighbors, discounted by the cost of standing in that cell.
Apply the rule everywhere and repeat until the numbers stop changing. That stable state is called a fixed point, and here it is the optimal value. The calculation needs no search and no central planner, only neighbors averaging with neighbors. The agent then moves toward higher-value neighbors.
Costs and rewards fit into the same rule. Cells believed to hold food raise the value nearby, and so does a small curiosity bonus for unexplored territory. Cells believed to hold something harmful, and the agent’s own recent trail, count as costs along a path, so the planner detours around them when the detour is worth it. The trail cost keeps the agent from pacing over ground it has just covered. Walls are simply impassable.
Working in log space
In its raw form the rule fails on larger maps. Desirability shrinks exponentially with distance, so a reward thirty steps away contributes almost nothing. Far-away food effectively disappeared, and performance collapsed.
The fix is to run the same rule on the logarithm of desirability, which brings us back to value itself. The logarithm turns the exponential fade into a linear one. Value drops by a steady amount per step rather than a steady fraction, so the slope pointing toward food stays clear far from it. This is a discrete version of the Cole–Hopf transform, a classical change of variables that turns certain nonlinear equations into linear ones. With it, success on a 30×30 map rose from collapse to 98.9%, the same rate as on a 15×15 map.
The log-space planner still has a horizon. How far away a reward can be seen depends on its size and the cost of each step, and with the default settings that is about 26 cells per unit of reward. Beyond that distance a reward is invisible by design, and the limit is a number you can tune.
The animation below shows the result in a larger world than any in the tests that follow.
What it achieves
To put numbers on it, we used a 17×17 foraging world and ran 200 episodes per agent on identical seeds. Scores are normalized so that a random walk scores 0 and a full-map A* agent, which sees every target and replans each step, scores 1.
| Agent | Training episodes | Success rate | Normalized score |
|---|---|---|---|
| Random walk | 0 | 6.0% | 0.00 |
| Greedy (chases whatever it can see) | 0 | 22.5% | 0.50 |
| Tabular Q-learning | 2,000 | 0.5% | −0.02 |
| EFI | 0 | 96.5% | 0.97 |
| A* with the full map | 0 | 100.0% | 1.00 |
Tabular Q-learning, a classic trial-and-error learning method, is a weak baseline here. It learns a separate value for every distinct 5×5 view, so it cannot generalize across randomly generated worlds. This is not a comparison with modern deep reinforcement learning.
The agent can also change its mind quickly. A separate test swaps the rewards partway through an episode, so everything the agent liked becomes harmful and vice versa. Because EFI recomputes values from its current beliefs every step, a few encounters with the newly harmful targets reverse its route. It recovered in 5 steps, against 69 to 83 for the trained and greedy agents. The test was preregistered, meaning we declared it before running it, but it was small, with three seeds of three episodes each.
The planner also supports a hard safety rule. A forbidden area can be given infinite cost, and the agent then never steps into it while it has any other open move. This is a guarantee, not a probability that shrinks as you tune it.
What we designed by hand
All of this comes with a qualification about the phrase “zero training.” The agent learns nothing in advance, but we chose a great deal: the model class (beliefs per target type, a grid map, costs along paths), the price of each step, trail and hazard, the curiosity bonus, and a single dial that sets how cautious or exploratory the agent is. In effect, the work that training would do moved into the derivation. For problems in the right class, the derivation can stand in for a training run, and every internal number stays inspectable.
We also cannot credit the grid of fields itself. In a later controlled comparison on a related navigation task, an ordinary lookup table that computes the same neighbor-based value equation did no worse than the field agent, within the test’s predeclared margin, so that test found no performance advantage for the field form.
Where it falls short
These are small gridworlds with simple physics, and nothing here shows the approach works in continuous or visually complex settings. Success rates matched (98.9%) at 15×15 and 30×30, which does not establish performance at every size.
When targets move only a short distance, a memoryless greedy agent earned slightly more reward than EFI, because EFI spends effort re-exploring.
The planner is exact only for the problem class it was derived for. A world outside that class, such as one where a key must be picked up before a door opens, needs structure the agent does not yet have.
The code, experiments and recorded episode players, which show the fields and action probabilities behind each move, are in the Embodied Field Intelligence repository, released under the MIT License.