Where should the next labels go?
With the same number of expert labels, is it better to see more demonstrations or get advice after a mistake? We compare both in a simulated maze.
Why this question matters
A robot learning from demonstrations may make a mistake and reach a situation its teacher never showed it. Asking for advice in those situations is a well-established idea in imitation learning. DAgger introduced a way to repeatedly collect such advice, and later robotics research has studied learning from corrections. Here we test how to spend a fixed label budget in BabyAI/MiniGrid, a small navigation simulator.
What we compare
Each of independent runs starts with one trained agent. Two copies each receive more expert labels. One learns from new expert demonstrations. The other learns from states it visits after a forced action change. Both get the same number of training updates and uses of labelled targets. We also test the original agent without further training.
How we test them
All agents face the same test scenarios. In the main comparison, we change one action using a mapping and timing not used during recovery training. Every episode counts, including one that ends before the scheduled change.
How the test scenarios were chosen
We selected scenarios from candidates using only the length of the expert’s route. This rule was fixed before testing. Counting all assigned episodes is called intention-to-treat scoring.
We compare the two trained agents within each run, then average those differences. Before testing, we chose as the smallest useful gain in success. The three test conditions are:
Results
Success under each condition
Recovery-trained agents have higher average success in all three conditions, including when no action is changed. The dots show how much results vary between training runs.
Recorded episodes
In this example, the recovery-trained agent succeeds and the agent given extra demonstrations does not. Both face the same maze, instruction, and scheduled action change.
The next example shows a recovery-trained agent that still fails.
What this result means
Within this task and training budget, recovery labels improve average success. Collecting them required more calls to the expert, because advice outside the chosen recovery window was discarded. Equal numbers of training labels therefore do not mean equal collection costs.
The experiment uses a small maze, a scripted expert, and three discrete actions. It tests one new action mapping, and every failed episode ends by reaching the step limit. We would need further experiments to know whether the same advantage holds for physical robots, human teachers, or a budget measured in teaching time.
Report and reproduction
The report gives the full methods, results, and additional analyses. The reproduction page contains the code and commands.