4.8
Combination of all DQN improvements and variants of DQN
34
5 Policy gradient methods for deep RL 36
5.1 Stochastic Policy Gradient . . . . . . . . . . . . . . . . . 37
5.2 Deterministic Policy Gradient . . . . . . . . . . . . . . . . 39
5.3 Actor-Critic Methods . . . . . . . . . . . . . . . . . . . . 40
5.4 Natural Policy Gradients . . . . . . . . . . . . . . . . . . 42
5.5 Trust Region Optimization . . . . . . . . . . . . . . . . . 43
5.6 Combining policy gradient and Q-learning . . . . . . . . . 44
6 Model-based methods for deep RL 46
6.1 Pure model-based methods . . . . . . . . . . . . . . . . . 46
6.2 Integrating model-free and model-based methods . . . . . 49
7 The concept of generalization 53
7.1 Feature selection . . . . . . . . . . . . . . . . . . . . . . . 58
7.2
Choice of the learning algorithm and function approximator
selection . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
7.3 Modifying the objective function . . . . . . . . . . . . . . 61
7.4 Hierarchical learning . . . . . . . . . . . . . . . . . . . . . 62
7.5 How to obtain the best bias-overfitting tradeoff . . . . . . 63
8 Particular challenges in the online setting 66
8.1 Exploration/Exploitation dilemma . . . . . . . . . . . . . . 66
8.2 Managing experience replay . . . . . . . . . . . . . . . . . 71
9 Benchmarking Deep RL 73
9.1 Benchmark Environments . . . . . . . . . . . . . . . . . . 73
9.2 Best practices to benchmark deep RL . . . . . . . . . . . 78
9.3 Open-source software for Deep RL . . . . . . . . . . . . . 80
10 Deep reinforcement learning beyond MDPs 81
10.1 Partial observability and the distribution of (related) MDPs 81
10.2 Transfer learning . . . . . . . . . . . . . . . . . . . . . . . 86
10.3 Learning without explicit reward function . . . . . . . . . . 89
10.4 Multi-agent systems . . . . . . . . . . . . . . . . . . . . . 91
评论0