- Title
- Steering approaches to Pareto-optimal multiobjective reinforcement learning
- Creator
- Vamplew, Peter; Issabekov, Rustam; Dazeley, Richard; Foale, Cameron; Berry, Adam; Moore, Tim; Creighton, Douglas
- Date
- 2017
- Type
- Text; Journal article
- Identifier
- http://researchonline.federation.edu.au/vital/access/HandleResolver/1959.17/159959
- Identifier
- vital:12076
- Identifier
-
https://doi.org/10.1016/j.neucom.2016.08.152
- Identifier
- ISSN:0925-2312
- Abstract
- For reinforcement learning tasks with multiple objectives, it may be advantageous to learn stochastic or non-stationary policies. This paper investigates two novel algorithms for learning non-stationary policies which produce Pareto-optimal behaviour (w-steering and Q-steering), by extending prior work based on the concept of geometric steering. Empirical results demonstrate that both new algorithms offer substantial performance improvements over stationary deterministic policies, while Q-steering significantly outperforms w-steering when the agent has no information about recurrent states within the environment. It is further demonstrated that Q-steering can be used interactively by providing a human decision-maker with a visualisation of the Pareto front and allowing them to adjust the agent’s target point during learning. To demonstrate broader applicability, the use of Q-steering in combination with function approximation is also illustrated on a task involving control of local battery storage for a residential solar power system.
- Publisher
- Elsevier
- Relation
- Neurocomputing Vol. 263, no. (2017), p. 26-38
- Rights
- Copyright © 2017 Elsevier B.V. All rights reserved.
- Rights
- This metadata is freely available under a CCO license
- Subject
- 08 Information and Computing Sciences; 09 Engineering; 17 Psychology and Cognitive Sciences; Multiobjective Reinforcement Learning; Non-Stationary Policies; Geometric Steering; Interactive Reinforcement Learning; Pareto Optimality
- Full Text
- Reviewed
- Hits: 4796
- Visitors: 4791
- Downloads: 509
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | SOURCE2 | Accepted Version | 1 MB | Adobe Acrobat PDF | View Details Download |