Softmax exploration strategies for multiobjective reinforcement learning

- Vamplew, Peter, Dazeley, Richard, Foale, Cameron

Title
Softmax exploration strategies for multiobjective reinforcement learning
Creator
Vamplew, Peter; Dazeley, Richard; Foale, Cameron
Date
2017
Type
Text; Journal article
Identifier
http://researchonline.federation.edu.au/vital/access/HandleResolver/1959.17/159910
Identifier
vital:12075
Identifier
https://doi.org/10.1016/j.neucom.2016.09.141
Identifier
ISSN:0925-2312
Abstract
Despite growing interest over recent years in applying reinforcement learning to multiobjective problems, there has been little research into the applicability and effectiveness of exploration strategies within the multiobjective context. This work considers several widely-used approaches to exploration from the single-objective reinforcement learning literature, and examines their incorporation into multiobjective Q-learning. In particular this paper proposes two novel approaches which extend the softmax operator to work with vector-valued rewards. The performance of these exploration strategies is evaluated across a set of benchmark environments. Issues arising from the multiobjective formulation of these benchmarks which impact on the performance of the exploration strategies are identified. It is shown that of the techniques considered, the combination of the novel softmax–epsilon exploration with optimistic initialisation provides the most effective trade-off between exploration and exploitation.
Publisher
Elsevier Ltd
Relation
Neurocomputing Vol. 263, no. (2017), p. 74-86
Rights
Copyright © 2017 Elsevier B.V. All rights reserved.
Rights
This metadata is freely available under a CCO license
Subject
08 Information and Computing Sciences; 09 Engineering; 17 Psychology and Cognitive Sciences; Multiobjective Reinforcement Learning; Exploration; Ε-Greedy Exploration; Optimistic Initialisation; Softmax
Full Text
Reviewed
  • Hits: 1657
  • Visitors: 2084
  • Downloads: 604
Thumbnail File Description Size Format
SOURCE2 Accepted Version 792 KB Adobe Acrobat PDF