Rule-based interactive assisted reinforcement learning
- Authors: Bignold, Adam
- Date: 2019
- Type: Text , Thesis , PhD
- Full Text:
- Description: Reinforcement Learning (RL) has seen increasing interest over the past few years, partially owing to breakthroughs in the digestion and application of external information. The use of external information results in improved learning speeds and solutions to more complex domains. This thesis, a collection of five key contributions, demonstrates that comparable performance gains to existing Interactive Reinforcement Learning methods can be achieved using less data, sourced during operation, and without prior verifcation and validation of the information's integrity. First, this thesis introduces Assisted Reinforcement Learning (ARL), a collective term referring to RL methods that utilise external information to leverage the learning process, and provides a non-exhaustive review of current ARL methods. Second, two advice delivery methods common in ARL, evaluative and informative, are compared through human trials. The comparison highlights how human engagement, accuracy of advice, agent performance, and advice utility differ between the two methods. Third, this thesis introduces simulated users as a methodology for testing and comparing ARL methods. Simulated users enable testing and comparing of ARL systems without costly and time-consuming human trials. While not a replacement for well-designed human trials, simulated users offer a cheap and robust approach to ARL design and comparison. Fourth, the concept of persistence is introduced to Interactive Reinforcement Learning. The retention and reuse of advice maximises utility and can lead to improved performance and reduced human demand. Finally, this thesis presents rule-based interactive RL, an iterative method for providing advice to an agent. Existing interactive RL methods rely on constant human supervision and evaluation, requiring a substantial commitment from the advice-giver. Rule-based advice can be provided proactively and be generalised over the state-space while remaining flexible enough to handle potentially inaccurate or irrelevant information. Ultimately, the thesis contributions are validated empirically and clearly show that rule-based advice signicantly reduces human guidance requirements while improving agent performance.
- Description: Doctor of Pholosophy
- Authors: Bignold, Adam
- Date: 2019
- Type: Text , Thesis , PhD
- Full Text:
- Description: Reinforcement Learning (RL) has seen increasing interest over the past few years, partially owing to breakthroughs in the digestion and application of external information. The use of external information results in improved learning speeds and solutions to more complex domains. This thesis, a collection of five key contributions, demonstrates that comparable performance gains to existing Interactive Reinforcement Learning methods can be achieved using less data, sourced during operation, and without prior verifcation and validation of the information's integrity. First, this thesis introduces Assisted Reinforcement Learning (ARL), a collective term referring to RL methods that utilise external information to leverage the learning process, and provides a non-exhaustive review of current ARL methods. Second, two advice delivery methods common in ARL, evaluative and informative, are compared through human trials. The comparison highlights how human engagement, accuracy of advice, agent performance, and advice utility differ between the two methods. Third, this thesis introduces simulated users as a methodology for testing and comparing ARL methods. Simulated users enable testing and comparing of ARL systems without costly and time-consuming human trials. While not a replacement for well-designed human trials, simulated users offer a cheap and robust approach to ARL design and comparison. Fourth, the concept of persistence is introduced to Interactive Reinforcement Learning. The retention and reuse of advice maximises utility and can lead to improved performance and reduced human demand. Finally, this thesis presents rule-based interactive RL, an iterative method for providing advice to an agent. Existing interactive RL methods rely on constant human supervision and evaluation, requiring a substantial commitment from the advice-giver. Rule-based advice can be provided proactively and be generalised over the state-space while remaining flexible enough to handle potentially inaccurate or irrelevant information. Ultimately, the thesis contributions are validated empirically and clearly show that rule-based advice signicantly reduces human guidance requirements while improving agent performance.
- Description: Doctor of Pholosophy
Stress resilience : meaning, measurement, and the effects of physical activity
- Authors: O’Donohue, Josephine
- Date: 2020
- Type: Text , Thesis , PhD
- Full Text:
- Description: In rodent models of human behaviour, physical activity builds stress resilience. In contrast, human studies physical activity and stress resilience have been limited, and when investigated, provided mixed results. These inconsistencies may be due to the ambiguous use of the term stress resilience in human studies, making it difficult to quantify. Therefore, I aimed to assess the effect of physical activity on stress resilience in humans, after first evaluating operational definitions of stress resilience. Initially, I evaluated the use of stress resilience across the literature to provide a definition that is consistent with the research base (Chapter 2). I subsequently defined stress resilience as an active process of adapting to unpredictable and uncontrollable stimuli. In Chapter 3, I systematically reviewed measures previously used to capture stress resilience in humans, including trait-based questionnaires, as well as affective and cardiorespiratory responses to stressful conditions. In Study 1 (Chapter 4) of this dissertation, I then assessed the convergent validity of these stress resilience measures , including the Brief Resilience Scale and the Connor-Davidson Resilience Scale, as well as various responses (i.e., affective balance, systolic blood pressure, rate pressure product, and multiple indices of heart rate variability) to three stressor conditions (i.e., no stress, auditory startles, and stressful video clips). Results of Study 1 indicated that no measure captured all elements of stress resilience, although some indices of heart rate variability were related to changes in affective balance and systolic blood pressure was related to the Brief Resilience Scale. Across two studies, I then evaluated the effect of physical activity on stress resilience using the measures from Study 1. Through a cross-sectional study (Study 2, Chapter 5), I found no difference between active participants (n = 16) and less active participants (n = 14) across stress resilience measures. Additionally, in Study 3 (Chapter 6), I found that an 8-week moderate physical activity intervention had no effect on these stress resilience measures in 14 participants from desk-based occupations. I did, however, find an increase in mood and reduction in systolic blood pressure post-intervention, compared to pre-intervention scores. I also discussed the feasibility of various recruitment techniques, intervention purity and adherence, as well as stress resilience measurement protocols for future intervention studies. Overall, this dissertation has provided a critical evaluation of the use and measurement of stress resilience, with findings indicating that physical activity may not enhance stress resilience in humans. I discuss differences in my findings, compared to rodent research, including the potential effects of environment enrichment and increased severity of measurement protocols in animal models. I also discuss limitations of stress resilience conceptualisation and provide suggestions for future research to navigate these challenges.
- Description: Doctor of Philosophy
- Authors: O’Donohue, Josephine
- Date: 2020
- Type: Text , Thesis , PhD
- Full Text:
- Description: In rodent models of human behaviour, physical activity builds stress resilience. In contrast, human studies physical activity and stress resilience have been limited, and when investigated, provided mixed results. These inconsistencies may be due to the ambiguous use of the term stress resilience in human studies, making it difficult to quantify. Therefore, I aimed to assess the effect of physical activity on stress resilience in humans, after first evaluating operational definitions of stress resilience. Initially, I evaluated the use of stress resilience across the literature to provide a definition that is consistent with the research base (Chapter 2). I subsequently defined stress resilience as an active process of adapting to unpredictable and uncontrollable stimuli. In Chapter 3, I systematically reviewed measures previously used to capture stress resilience in humans, including trait-based questionnaires, as well as affective and cardiorespiratory responses to stressful conditions. In Study 1 (Chapter 4) of this dissertation, I then assessed the convergent validity of these stress resilience measures , including the Brief Resilience Scale and the Connor-Davidson Resilience Scale, as well as various responses (i.e., affective balance, systolic blood pressure, rate pressure product, and multiple indices of heart rate variability) to three stressor conditions (i.e., no stress, auditory startles, and stressful video clips). Results of Study 1 indicated that no measure captured all elements of stress resilience, although some indices of heart rate variability were related to changes in affective balance and systolic blood pressure was related to the Brief Resilience Scale. Across two studies, I then evaluated the effect of physical activity on stress resilience using the measures from Study 1. Through a cross-sectional study (Study 2, Chapter 5), I found no difference between active participants (n = 16) and less active participants (n = 14) across stress resilience measures. Additionally, in Study 3 (Chapter 6), I found that an 8-week moderate physical activity intervention had no effect on these stress resilience measures in 14 participants from desk-based occupations. I did, however, find an increase in mood and reduction in systolic blood pressure post-intervention, compared to pre-intervention scores. I also discussed the feasibility of various recruitment techniques, intervention purity and adherence, as well as stress resilience measurement protocols for future intervention studies. Overall, this dissertation has provided a critical evaluation of the use and measurement of stress resilience, with findings indicating that physical activity may not enhance stress resilience in humans. I discuss differences in my findings, compared to rodent research, including the potential effects of environment enrichment and increased severity of measurement protocols in animal models. I also discuss limitations of stress resilience conceptualisation and provide suggestions for future research to navigate these challenges.
- Description: Doctor of Philosophy
- «
- ‹
- 1
- ›
- »