- Title
- A conceptual framework for externally-influenced agents: an assisted reinforcement learning review
- Creator
- Bignold, Adam; Cruz, Francisco; Taylor, Matthew; Brys, Tim; Dazeley, Richard; Vamplew, Peter; Foale, Cameron
- Date
- 2023
- Type
- Text; Journal article
- Identifier
- http://researchonline.federation.edu.au/vital/access/HandleResolver/1959.17/191874
- Identifier
- vital:17892
- Identifier
-
https://doi.org/10.1007/s12652-021-03489-y
- Identifier
- ISSN:1868-5137 (ISSN)
- Abstract
- A long-term goal of reinforcement learning agents is to be able to perform tasks in complex real-world scenarios. The use of external information is one way of scaling agents to more complex problems. However, there is a general lack of collaboration or interoperability between different approaches using external information. In this work, while reviewing externally-influenced methods, we propose a conceptual framework and taxonomy for assisted reinforcement learning, aimed at fostering collaboration by classifying and comparing various methods that use external information in the learning process. The proposed taxonomy details the relationship between the external information source and the learner agent, highlighting the process of information decomposition, structure, retention, and how it can be used to influence agent learning. As well as reviewing state-of-the-art methods, we identify current streams of reinforcement learning that use external information in order to improve the agent’s performance and its decision-making process. These include heuristic reinforcement learning, interactive reinforcement learning, learning from demonstration, transfer learning, and learning from multiple sources, among others. These streams of reinforcement learning operate with the shared objective of scaffolding the learner agent. Lastly, we discuss further possibilities for future work in the field of assisted reinforcement learning systems. © 2021, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.
- Publisher
- Springer Science and Business Media Deutschland GmbH
- Relation
- Journal of Ambient Intelligence and Humanized Computing Vol. 14, no. 4 (2023), p. 3621-3644
- Rights
- All metadata describing materials held in, or linked to, the repository is freely available under a CC0 licence
- Rights
- Copyright © The Author(s)
- Rights
- Open Access
- Subject
- 46 Information and Computing Sciences; Assistance taxonomy; Assisted reinforcement learning; Externally-influenced agents
- Full Text
- Reviewed
- Funder
- This work has been partially supported by the Australian Government Research Training Program (RTP) and the RTP Fee-Offset Scholarship through Federation University Australia. Moreover, this work has taken place in part in the Intelligent Robot Learning Lab at the University of Alberta, which is supported in part by research grants from the Alberta Machine Intelligence Institute (Amii); CIFAR; a Canada CIFAR AI Chair, Amii; Compute Canada; and NSERC. This work has taken place in part in the Intelligent Robot Learning Lab at the University of Alberta, which is supported in part by research grants from the Alberta Machine Intelligence Institute (Amii); CIFAR; a Canada CIFAR AI Chair, Amii; Compute Canada; and NSERC.
- Hits: 4706
- Visitors: 3270
- Downloads: 88
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | SOURCE1 | Accepted version | 618 KB | Adobe Acrobat PDF | View Details Download |