- Title
- An evaluation methodology for interactive reinforcement learning with simulated users
- Creator
- Bignold, Adam; Cruz, Francisco; Dazeley, Richard; Vamplew, Peter; Foale, Cameron
- Date
- 2021
- Type
- Text; Journal article
- Identifier
- http://researchonline.federation.edu.au/vital/access/HandleResolver/1959.17/176314
- Identifier
- vital:15096
- Identifier
-
https://doi.org/10.3390/biomimetics6010013
- Identifier
- ISBN:2313-7673 (ISSN)
- Abstract
- Interactive reinforcement learning methods utilise an external information source to evaluate decisions and accelerate learning. Previous work has shown that human advice could significantly improve learning agents’ performance. When evaluating reinforcement learning algorithms, it is common to repeat experiments as parameters are altered or to gain a sufficient sample size. In this regard, to require human interaction every time an experiment is restarted is undesirable, particularly when the expense in doing so can be considerable. Additionally, reusing the same people for the experiment introduces bias, as they will learn the behaviour of the agent and the dynamics of the environment. This paper presents a methodology for evaluating interactive reinforcement learning agents by employing simulated users. Simulated users allow human knowledge, bias, and interaction to be simulated. The use of simulated users allows the development and testing of reinforcement learning agents, and can provide indicative results of agent performance under defined human constraints. While simulated users are no replacement for actual humans, they do offer an affordable and fast alternative for evaluative assisted agents. We introduce a method for performing a preliminary evaluation utilising simulated users to show how performance changes depending on the type of user assisting the agent. Moreover, we describe how human interaction may be simulated, and present an experiment illustrating the applicability of simulating users in evaluating agent performance when assisted by different types of trainers. Experimental results show that the use of this methodology allows for greater insight into the performance of interactive reinforcement learning agents when advised by different users. The use of simulated users with varying characteristics allows for evaluation of the impact of those characteristics on the behaviour of the learning agent. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.
- Publisher
- MDPI AG
- Relation
- Biomimetics Vol. 6, no. 1 (2021), p. 1-15
- Rights
- All metadata describing materials held in, or linked to, the repository is freely available under a CC0 licence
- Rights
- https://creativecommons.org/licenses/by/4.0/
- Rights
- Copyright © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
- Rights
- Open Access
- Subject
- 0806 Information Systems; 1503 Business and Management; Interactive reinforcement learning; Methodology for simulated users; Reinforcement learning; Reward shaping
- Full Text
- Reviewed
- Funder
- This work has been partially supported by the Australian Government Research Training Program (RTP) and the RTP Fee-Offset Scholarship through Federation University Australia.
- Hits: 3803
- Visitors: 3117
- Downloads: 141
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | SOURCE1 | Published version | 907 KB | Adobe Acrobat PDF | View Details Download |