Chapter 05 of 08 · teaching experiments
The network, piece by piece
The whole policy is parameters, small enough to hold in one head, and audited three ways: a walkthrough generated from a live instance, a from-scratch reimplementation checked parameter-for-parameter against the original, and an ablation in which the data judges each design choice.
The shape of it
Three embedding tables turn the symbolic observation's object/color/state indices into vectors; the observation is symbolic (chapter 01), so embeddings replace the pixel stacks a visual task would need. Two 3×3 convolutions and a projection summarize the view; a small GRU reads the mission once per episode; embeddings of the view direction and the previous executed action complete the inputs; a fusion layer mixes everything; a second GRU carries memory across steps; a linear head emits three action logits.
Watch the real numbers flow
Below is the network itself, not a schematic. Every dot is one actual value from a trained checkpoint, replayed on a real episode: dark blue is that block's smallest value, light orange its largest. Scrub through the steps, or press play to watch one forward pass: the observation is unpacked into embeddings, convolved down to a vector, joined with the mission and context, and packed through the recurrent core into three logits.
The previous-action input deserves a sentence now: in clean runs it is nearly redundant, but it is the only channel through which an externally corrupted execution becomes visible to the policy. Its purpose exists entirely in chapter 06.
One component removed at a time
Five variants (the full model, no memory GRU, no mission input, no previous-action input, and a bag-of-words mission encoder) trained on identical demonstrations with an identical optimizer, three seeds each:
How to read this
At this training scale, clean-condition ablations are a blunt instrument: the variants land within a few points of each other and the three-seed ranges overlap heavily. Two findings deserve their own sentences. Removing the mission costs nothing here: with one distractor and a long step limit, a mission-blind policy that tours objects still ends on the right one; the endpoint tolerates detours. And the bag-of-words encoder matches the GRU at this five-word grammar. The architecture is justified by structural roles: the aliasing floor of chapter 02 for memory, the corruption channel for the previous action, the task definition for the mission, not by ablation wins smaller than seed noise. The same architecture trained under this chapter and under chapter 04 does not land in the same place twice, which is exactly why the study runs six paired replicates and reports an interval (chapter 07).
Why a GRU and not a Transformer
The dataset is a few thousand labelled steps; inference is streamed one observation at a time with constant-size state; and the model stays small enough to audit by hand, which this project treats as a feature. Nothing here claims architectural novelty; the network is the standard BabyAI treatment, dissected.
Summary
Every module of the exact network the study trains, which components carry structural weight, and a healthy distrust of small-n ablation deltas. Now we can finally break things on purpose.