- Keywords: reinforcement learning, curriculum learning, sparse rewards
- Abstract: The objective in goal-based reinforcement learning is to learn a policy to reach a particular goal state within the environment. However, the underlying reward function may be too sparse for the agent to efficiently learn useful behaviors. Recent studies have demonstrated that reward sparsity can be overcome by instead learning a curriculum of simpler subtasks. In this work, we design an agent's curriculum by focusing on the aspect of goal reachability, and introduce the idea of a reachability trace, which is used as a basis to determine a sequence of intermediate subgoals to guide the agent towards its primary goal. We discuss several properties of the trace function, and in addition, validate our proposed approach empirically in a range of environments, while comparing its performance against appropriate baselines.
- One-sentence Summary: We propose the idea of reachability traces, which provides an indication of closeness to a goal state, and use this as a basis for designing a curriculum for goal-based tasks in reinforcement learning.
- Supplementary Material: zip