A test of a systems theory-based incident coding taxonomy for risk managers
- Authors: Goode, Natassia , Salmon, Paul , Finch, Caroline , Lenne, Michael
- Date: 2014
- Type: Text , Conference paper
- Relation: Proceedings of the 5th Applied Human Factors and Ergonomics (AHFE) International Conference p. 5098-5108
- Full Text: false
- Reviewed:
- Description: Organizations need to be able to collect reliable and accurate data on the causal factors that lead to near misses and injury causing incidents in order to design appropriate, informed, safety interventions. The aim of this study was to test the inter-rater reliability of a prototype taxonomy for classifying the causal factors involved in incidents in the outdoor education and recreation sector. The taxonomy consists of three levels, where each category level breaks the previous one down into a finer level of detail. The study involved 14 respondents, who play a key role in risk management within their organization, using the taxonomy to code 10 detailed incident reports. The incident reports were composited from reports and enquiries into actual events, and ranged in injury severity from fractures to fatalities. Participants were asked to: 1) identify the causal factors involved in each incident; and 2) identify the code/s from the taxonomy which best described those causal factors. The study demonstrated that the taxonomy can be used by risk managers to identify and code causal factors across all levels of the led outdoor activity system. However, identifying appropriate codes at the second and third level of detail was problematic.
Bridging the research-practice gap: Validity of a software tool designed to support systemic accident analysis by risk managers
- Authors: Goode, Natassia , Salmon, Paul , Taylor, Natalie , Lenné, Michael , Finch, Caroline
- Date: 2015
- Type: Text , Conference proceedings
- Relation: http://purl.org/au-research/grants/nhmrc/1058737
- Full Text: false
- Description: Despite the proposed advantages of systems accident analysis (SAA) methods for understanding incident-causation, these approaches have not been widely adopted by practitioners. This represents a significant gap between research and practice in accident analysis. The Understanding and Preventing Led Outdoor Accidents Data System (UPLOADS) provides a series of tools to address this gap. The aim of this study was to evaluate the validity of UPLOADS by comparing analyses generated by risk managers and researchers experienced in SAA. Twenty-three risk managers used UPLOADS to collect and analyse incident data from their organization over a three month period. The reports were then analyzed by two researchers experienced in SAA, and compared to those generated by participants. Participants identified half the number of factors identified by researchers, and tended to focus on only one or two factors as the causes of each incident. The potential consequences for practitioners' understanding of incident-causation and countermeasure development are discussed, as well as ways of improving the system. © Springer International Publishing Switzerland 2015.
Challenges of translating Rasmussen’s Accimap into a usable, sustainable, and useful incident reporting system: end-user attitudes following 12-month implementation
- Authors: Goode, Natassia , Shaw, Louise , Finch, Caroline , Salmon, Paul
- Date: 2021
- Type: Text , Journal article
- Relation: Cognition, Technology and Work Vol. 23, no. 1 (2021), p. 39-49
- Full Text: false
- Reviewed:
- Description: There have been several calls within the accident analysis literature to translate systems thinking-based theories and accident analysis methods into practical and usable tools for practitioners. This paper reports on the findings from a post-implementation evaluation of an incident reporting system underpinned Rasmussen’s (Saf Sci 27(2/3):183–213, 1997) risk-management framework and Accimap. An online survey was used to evaluate end-users’ use and attitudes towards the system after 12 months of implementation. While almost all respondents had used the incident reporting system to collect data, few had undertaken the required training. While the privacy and security of the system were evaluated favourably, significant problems were highlighted with usability, sustainability of the system, and utility of the data for organisations. Overall, the results highlight the significant resource constraints experienced by end-users during the implementation of the new incident reporting system. The challenges for translating a systems thinking-based approach into practice are discussed. © 2019, Springer-Verlag London Ltd., part of Springer Nature.
Closing Pandora's Box : Adapting a systems ergonomics methodology for better understanding the ecological complexity underpinning the development and prevention of running-related injury
- Authors: Hulme, Adam , Salmon, Paul , Nielsen, Rasmus , Read, Gemma , Finch, Caroline
- Date: 2017
- Type: Text , Journal article
- Relation: Theoretical Issues in Ergonomics Science Vol. 18, no. 4 (2017), p. 338-359
- Full Text: false
- Reviewed:
- Description: The popularity of running as a form of exercise continues to increase dramatically worldwide. Alongside this participation growth is the burden of running-related injury (RRI). Over the past four decades, traditional scientific research applications have primarily attempted to isolate discrete risk factors for RRI using observational study designs as commonly used in public health epidemiology. Unfortunately, only very few randomised controlled trials have evaluated the efficacy associated with a well-specified RRI prevention intervention. Even though the knowledge about risk factors as generated in observational studies is valuable for better understanding why RRI develops, it nonetheless means that there remains a major knowledge gap about how best to prevent it, especially in a way that fully addresses all causal factors. Alongside the continuing use of traditional scientific approaches, a particular systems ergonomics methodology should also be considered in light of its potential to visualise the complete distance running system. This article adapts the Systems Theoretic Accident Mapping and Processes (STAMP) model to the RRI research prevention context. The direct application of STAMP might offer new knowledge about how to prevent RRI, such as exposing questions around the feasibility of adopting novel injury prevention interventions that do not directly target runners themselves. © 2017 Informa UK Limited, trading as Taylor & Francis Group.
Developing a contributing factor classification scheme for Rasmussen's AcciMap : Reliability and validity evaluation
- Authors: Goode, Natassia , Salmon, Paul , Taylor, Natalie , Lenné, Michael , Finch, Caroline
- Date: 2017
- Type: Text , Journal article
- Relation: Applied Ergonomics Vol. 64, no. (2017), p. 14-26
- Relation: http://purl.org/au-research/grants/nhmrc/565900
- Full Text: false
- Reviewed:
- Description: One factor potentially limiting the uptake of Rasmussen's (1997) Accimap method by practitioners is the lack of a contributing factor classification scheme to guide accident analyses. This article evaluates the intra- and inter-rater reliability and criterion-referenced validity of a classification scheme developed to support the use of Accimap by led outdoor activity (LOA) practitioners. The classification scheme has two levels: the system level describes the actors, artefacts and activity context in terms of 14 codes; the descriptor level breaks the system level codes down into 107 specific contributing factors. The study involved 11 LOA practitioners using the scheme on two separate occasions to code a pre-determined list of contributing factors identified from four incident reports. Criterion-referenced validity was assessed by comparing the codes selected by LOA practitioners to those selected by the method creators. Mean intra-rater reliability scores at the system (M = 83.6%) and descriptor (M = 74%) levels were acceptable. Mean inter-rater reliability scores were not consistently acceptable for both coding attempts at the system level (MT1 = 68.8%; M T2 = 73.9%), and were poor at the descriptor level (MT1 = 58.5%; M T2 = 64.1%). Mean criterion referenced validity scores at the system level were acceptable (MT1 = 73.9%; M T2 = 75.3%). However, they were not consistently acceptable at the descriptor level (MT1 = 67.6%; M T2 = 70.8%). Overall, the results indicate that the classification scheme does not currently satisfy reliability and validity requirements, and that further work is required. The implications for the design and development of contributing factors classification schemes are discussed. © 2017 Elsevier Ltd
From control to causation : Validating a ‘complex systems model’ of running-related injury development and prevention
- Authors: Hulme, Adam , Salmon, Paul , Nielsen, Rasmus , Read, Gemma , Finch, Caroline
- Date: 2017
- Type: Text , Journal article
- Relation: Applied Ergonomics Vol. 65, no. (2017), p. 345-354
- Full Text: false
- Reviewed:
- Description: Introduction There is a need for an ecological and complex systems approach for better understanding the development and prevention of running-related injury (RRI). In a previous article, we proposed a prototype model of the Australian recreational distance running system which was based on the Systems Theoretic Accident Mapping and Processes (STAMP) method. That model included the influence of political, organisational, managerial, and sociocultural determinants alongside individual-level factors in relation to RRI development. The purpose of this study was to validate that prototype model by drawing on the expertise of both systems thinking and distance running experts. Materials and methods This study used a modified Delphi technique involving a series of online surveys (December 2016- March 2017). The initial survey was divided into four sections containing a total of seven questions pertaining to different features associated with the prototype model. Consensus in opinion about the validity of the prototype model was reached when the number of experts who agreed or disagreed with survey statement was ≥75% of the total number of respondents. Results A total of two Delphi rounds was needed to validate the prototype model. Out of a total of 51 experts who were initially contacted, 50.9% (n = 26) completed the first round of the Delphi, and 92.3% (n = 24) of those in the first round participated in the second. Most of the 24 full participants considered themselves to be a running expert (66.7%), and approximately a third indicated their expertise as a systems thinker (33.3%). After the second round, 91.7% of the experts agreed that the prototype model was a valid description of the Australian distance running system. Conclusion This is the first study to formally examine the development and prevention of RRI from an ecological and complex systems perspective. The validated model of the Australian distance running system facilitates theoretical advancement in terms of identifying practical system-wide opportunities for the implementation of sustainable RRI prevention interventions. This ‘big picture’ perspective represents the first step required when thinking about the range of contributory causal factors that affect other system elements, as well as runners' behaviours in relation to RRI risk. © 2017 Elsevier Ltd
How do I save it? Usability evaluation of a systems theory-based incident reporting software prototype by novice end users
- Authors: Grant, Eryn , Goode, Natassia , Salmon, Paul , Lenné, Michael , Scott-Parker, Bridie , Finch, Caroline
- Date: 2015
- Type: Text , Conference proceedings
- Relation: http://purl.org/au-research/grants/nhmrc/1058737
- Full Text: false
- Description: The level of usability achieved by software tools is a key factor that determines their success and indeed uptake by end users. This paper describes a study that was undertaken to evaluate the usability of a prototype incident reporting software tool. The study involved novice end users completing a series of tasks using the software tool and then completing Ravden and Johnson’s Human Computer Interaction (HCI) checklist. The findings identify aspects of the system that pose particular challenges for participants. Participants appeared to lack a clear understanding of the relationship between the information required from them, and the underpinning accident analysis method of the software tool. This is perhaps unsurprising, considering that most incident reporting systems do not include these functions. The findings indicate that the tool requires better levels of intuitiveness to assist users in complex tasks so the focus is on awareness of accident causation methods rather than task instructions. The implications for the design of incident reporting software tools are discussed. © Springer International Publishing Switzerland 2015.
Injury causation in the great outdoors: A systems analysis of led outdoor activity injury incidents
- Authors: Salmon, Paul , Goode, Natassia , Lenne, Michael , Finch, Caroline , Cassell, Erin
- Date: 2014
- Type: Text , Journal article
- Relation: Accident Analysis and Prevention Vol. 63, no. February (2014), p. 111-120
- Full Text:
- Reviewed:
- Description: Despite calls for a systems approach to assessing and preventing injurious incidents within the led outdoor activity domain, applications of systems analysis frameworks to the analysis of incident data have been sparse. This article presents an analysis of 1014 led outdoor activity injury and near miss incidents whereby a systems-based risk management framework was used to classify the contributing factors involved across six levels of the led outdoor activity 'system'. The analysis identified causal factors across all levels of the led outdoor activity system, demonstrating the framework's utility for accident analysis efforts in the led outdoor activity injury domain. In addition, issues associated with the current data collection framework that potentially limited the identification of contributing factors outside of the individuals, equipment, and environment involved were identified. In closing, the requirement for new and improved data systems to be underpinned by the systems philosophy and new models of led outdoor activity accident causation is discussed.© 2013 Elsevier Ltd. All rights reserved.
- Description: C1
Looking beyond people, equipment and environment : Is a systems theory model of accident causation required to understand injuries and near misses during outdoor activities?
- Authors: Goode, Natassia , Salmon, Paul , Lenne, Michael , Finch, Caroline
- Date: 2015
- Type: Text , Conference paper
- Relation: 6th International Conference on Applied Human Factors and Ergonomics (Ahfe 2015) and the Affiliated Conferences, Ahfe 2015; Las Vegas, USA; 26th-30th July 2015; published in Procedia Manufacturing Vol. 3, p. 1125-1131
- Relation: http://purl.org/au-research/grants/nhmrc/1058737
- Full Text:
- Reviewed:
- Description: The National Incident Database (NID) provides a standardised, industry-wide, approach to incident reporting for the outdoor sector in New Zealand (NZ). The aim of this study was to determine whether the NID contributing factor categories (i. e. people, equipment and environment) are sufficient for classifying the data that has been collected on accident causation by the NID, or whether a systems theory framework is required. A sample of injury and near miss reports (n = 228) were extracted from the NID and analysed. All contributing factors identified were classified according to Rasmussen's (1997) Risk Management Framework (RRMF), which was adapted to describe the " led outdoor activity system". In total, 58 different contributing factor categories were identified across the 228 incidents. Factors were classified across all levels of the framework, which indicates that the NID categories are inadequate. The findings also demonstrate that RRMF is appropriate for classifying the contributing factors involved in less severe injuries and near misses that do not have in-depth investigations associated with them. (C) 2015 The Authors. Published by Elsevier B.V.
Lost in translation: the validity of a systemic accident analysis method embedded in an incident reporting software tool
- Authors: Goode, Natassia , Salmon, Paul , Taylor, Natalie , Lenné, Michael , Finch, Caroline
- Date: 2016
- Type: Text , Journal article
- Relation: Theoretical Issues in Ergonomics Science Vol. 17, no. 5-6 (2016), p. 483-506
- Full Text:
- Reviewed:
- Description: ABSTRACT Despite the proposed advantages of systems accident analysis (SAA) methods for understanding incident causation, they have not been widely adopted by practitioners. The aim of this study was to evaluate the criterion-referenced validity of an SAA method embedded within an incident reporting software tool. Thirteen practitioners used the tool to collect and analyse incident data within their organisation. The incident data were then analysed by researchers experienced in using the SAA method. Overall, there were low levels of agreement between participants and researchers regarding the identification and classification of factors and relationships. The findings indicate the systems thinking principles underpinning the SAA method may have been ‘lost in translation’, in that participants often identified only one or two factors and showed a poor understanding of how to identify relationships between factors. The methodological developments required to ensure that practitioners can validly apply the SAA method are discussed.
Rasmussen's legacy in the great outdoors : A new incident reporting and learning system for led outdoor activities
- Authors: Salmon, Paul , Goode, Natassia , Taylor, Natalie , Lenne, Michael , Dallat, Clare , Finch, Caroline
- Date: 2017
- Type: Text , Journal article
- Relation: Applied Ergonomics Vol. 59, no. (2017), p. 637-648
- Relation: http://purl.org/au-research/grants/nhmrc/565900
- Full Text: false
- Reviewed:
- Description: Jens Rasmussen's seminal risk management framework and accompanying Accimap method have become highly popular in safety science circles. Despite this, widespread adoption of the model and method in practice has not yet been achieved. This paper describes a project involving the development and implementation of an incident reporting and learning system underpinned by Rasmussen's risk management framework and Accimap method. The system was developed for the led outdoor activity sector in Australia to enable reporting and analysis of injuries and near miss incidents, with the aim of supporting the development of more effective countermeasures. An analysis of the data derived from the first 3 months use of the system by 43 organisations is presented. The outputs provide an in-depth Accimap-based analysis of all incidents reported by participating organisations over the 3 month period. In closing, the importance of developing usable domain specific tools to support translation of Ergonomics theory and methods in practice is discussed.
- Description: Jens Rasmussen's seminal risk management framework and accompanying Accimap method have become highly popular in safety science circles. Despite this, widespread adoption of the model and method in practice has not yet been achieved. This paper describes a project involving the development and implementation of an incident reporting and learning system underpinned by Rasmussen's risk management framework and Accimap method. The system was developed for the led outdoor activity sector in Australia to enable reporting and analysis of injuries and near miss incidents, with the aim of supporting the development of more effective countermeasures. An analysis of the data derived from the first 3 months use of the system by 43 organisations is presented. The outputs provide an in-depth Accimap-based analysis of all incidents reported by participating organisations over the 3 month period. In closing, the importance of developing usable domain specific tools to support translation of Ergonomics theory and methods in practice is discussed. (C) 2015 Elsevier Ltd. All rights reserved.
What would you like? Identifying the required characteristics of an industry-wide incident reporting and learning system for the led outdoor activity sector
- Authors: Goode, Natassia , Finch, Caroline , Cassell, Erin , Lenne, Michael , Salmon, Paul
- Date: 2014
- Type: Text , Journal article
- Relation: Australian Journal of Outdoor Education Vol. 17, no. 2 (July 2014), p. 2-15
- Relation: http://purl.org/au-research/grants/nhmrc/565900
- Full Text: false
- Reviewed:
- Description: The aim of this study was to identify the characteristics that led outdoor activity providers agree are necessary for the development of a new industry-wide incident reporting and learning system (UPLOADS). The study involved: 1) a literature review to identify a set of characteristics that are considered to be hallmarks of successful reporting and learning systems in other safety-critical domains; and (2) the presentation of these characteristics to 25 Australian led outdoor activity providers using a two round modified-Delphi technique to obtain consensus views on their relative importance in this domain. Thirteen out of 30 characteristics were endorsed as "essential" for developing an incident reporting and learning system for the led outdoor activity sector, and a further 13 were endorsed as "required". "Essential" characteristics primarily related to operational or practical characteristics of the system, while "required" characteristics primarily related to system infrastructure, data quality and the basis for developing of countermeasures to address identified injury risks. The findings indicate that although led outdoor activity providers are primarily concerned that the demands of reporting do not adversely impact on their day to day operations, they also recognise that data collection methods and countermeasure development need to be of high quality. The paper concludes by highlighting some potential strategies for implementing the characteristics considered "essential" and "required".