Exploration of the perceptual-cognitive processes that contribute to in-game decision-making of Australian football umpires

Decision-making is fundamental to officiating in all sports. It is well established in contemporary research that decision-making skills underpin umpire expertise; however, there is little understanding of the cognitive processes that contribute to in-game decision-making. This research implemented an in-depth case study approach, using qualitative methods, to explore the in-game decision-making process of three Australian football umpires. Concurrent and retrospective verbalisation methods were used to obtain verbal reports of the cognitive processes associated with decision-making. Findings identified three salient themes related to both in-game decision-making processes (i.e. decision evaluation, player intention during game-play) and umpire performance (i.e. knowledge of game-play). These themes contributed to the development of decision-making heuristics for Australian football umpires. This study provides initial evidence of the factors that may contribute to and/or affect in-game decision-making processes; however, additional exploration is necessary to further inform training programmes aimed to develop domain-specific decision-making skills and subsequent in-game performance.

In an attempt to understand skilled referee performance, Mascarenhas, Collins and Mortimer (2005) identified five themes that informed the development of their Cornerstones Model of Refereeing Performance including: (i) knowledge and application of the law; (ii) physical fitness, positioning and mechanics; (iii) personality and game management; (iv) contextual judgement; and (v) psychological characteristics of excellence. While all themes are potentially relevant to in-game decision-making, specifically the theme knowledge and application of the law can be identified as directly relevant to in-game decision-making, and was defined as "the underpinning knowledge of the law that allows referees to accurately interpret dynamic situations and penalise accordingly" (Mascarenhas et al., 2005, p. 368). Mascarenhas et al. identified several important sub-themes under the knowledge and application of the law theme that included decision-making timing and consistency, and decision-making clarity. While the cornerstones model provides a description of the factors that may contribute to performance, the model does not indicate how these factors interact to inform the decision-making process. Therefore, to develop a greater understanding of the underlying cognitions for decision-making performance, MacMahon and McPherson (2009) suggested experimental designs include verbalisation methods, such as interviews or verbalisation of events, to better inform the decision-making process.
Despite verbalisation techniques providing an understanding of the cognitive processes that contribute to problem solving (Kuusela & Paul, 2000), this methodology has been afforded little research attention in order to understand factors that contribute to the decision-making process of sports officials. One investigation (Lane, Nevill, Ahmad, & Balmer, 2006) used retrospective verbalisation to explore the factors that five experienced soccer referees perceived to influence decision-making. Identified themes included crowd factors, decision accuracy and errors, experience, regulations, and opinions. For example, the referees strived for accurate decisions; however, in relation to decision accuracy they were accepting that human error can influence decision-making accuracy with logical reasons for inaccurate decisions being the speed of the game, or not being in the correct position to view the incident. Despite identifying some of the themes associated with decision-making, the investigation did not describe the underlying cognitive processes associated with in-game decision-making. Extending this research, Hancock and Ste-Marie (2014) used a stimulated recall technique to explore the strategies used by elite, intermediate and novice ice hockey referees when making a decision. Participants viewed footage from a head camera of a game they had refereed and were asked a series of questions relating to their decision-making strategies. Results demonstrated an expertise effect with elite referees demonstrating more refined knowledge structures. Further, strategies influencing in-game decisions such as game context, anticipating game flow and prioritising the certain decision-making situations were also identified.
While the current research exploring sports officials decision-making have identified numerous external factors that may influence the decision-making process, there has been limited exploration of the underlying processes associated with in-game decision-making. Therefore, this study aims to investigate the strategies associated with in-game decision-making of Australian football umpires. Further, the investigation attempts to describe the underlying processes associated with in-game decision-making, to potentially inform the creation of Australian football umpire specific decision-making heuristics.

Method Participants
Three male Australian football umpires, who were previously involved in training camps at an elite level and currently officiating in a regional Australian football Division 1 competition volunteered to participate in the study. Participant 1 (i.e. P1) was 30 years old and had umpired 170 senior Australian football Division 1 games. Participant 2 (i.e. P2) had umpired 350 senior Australian football games and was 32 years old. Participant 3 (i.e. P3) was 39 years old and had umpired a total of 400 senior Australian football Division 1 games. While there is a disparity in the number of games each participant had umpired, following consultation with umpire coaches with regards to current in-game decision-making performance, all participants were considered to be of the same current performance level. Ethics approval was granted by the University Human Research Ethics Committee, and participants provided approved consent prior to the commencement of the study.

Procedure
In-Game verbalisation During two competitive in-season games, which were separated by 7 days, participants were instructed to "think out loud" and provide a "running commentary" of the in-game information they were perceiving and the processes by which they made a decision. This process was believed to provide an explicit representation of the cognitive processes associated with in-game decisionmaking. In-game verbal information was recorded using an Olympus DS-5000 digital voice recorder, which was attached to the upper back of the participant using a commercially available elastic harness that is used for player tracking devices. A lapel microphone was used to capture verbal information and was attached to the shirt lapel. In addition, video footage was recorded using two digital video cameras positioned in an elevated central position on the boundary line. The first camera tracked the movement of the ball providing vision of player contests, body contact, infringements and general game-play similar to television broadcast view (i.e. the immediate vicinity of the ball). The second camera tracked the movements and actions of the participating umpire. After video recording, the video footage was coupled with the audio recording with the verbal reports transcribed verbatim.
To ensure the participants were comfortable and competent at verbalising their thought processes during the game, participants completed three in-game verbalisation familiarisation sessions, whereby they were asked to verbalise their cognitive processes. Following the familiarisation sessions the footage was reviewed and any further questions from the participants were answered.

Semi-structured interviews
To further ascertain the participant's decision-making process, individual semi-structured interviews were conducted, using the stimulated recall technique. The stimulated recall technique is an introspective procedure, whereby video recorded information showing the actions and behaviour of the participant is replayed to stimulate recall of cognitive processes (Lyle, 2003).
During the semi-structured interviews, which ranged from 26 to 39 minutes in duration, video footage from the two recorded games was shown to aid accurate recall of information (Côté, Ericsson, & Law, 2005). During the interview, which occurred at the participants first available training session following the second recorded game (i.e. three to five days after; see Figure 1), participants viewed eight short video clips (coupled with in-game verbalisation audio) and were asked to recall and verbalise their cognitive processes of the in-game incidents presented. The video clips were used as a primer to prompt decision-making processes and negate any disparity with the timing of the retrospective recall. The video clips presented situations where the participant was the controlling umpire for the passage of play and contained six incidents that resulted in a free kick, and two incidents where the participant did not award a free kick.
Following the initial recall, the video was replayed and paused at specific time points (i.e. just prior to or following the incident) and a series of open-ended questions were asked. The openended questions were focused on understanding the cognitive processes and identifying decision-making information used by the participant (e.g. can you describe the information that led you to this decision? Explain how you came to this conclusion in the passage of play rather than a different outcome?). This same sequence of questions was repeated for the remaining seven clips.
After the eight clips had been considered and discussed, participants were presented with two standardised game-play situations from a game where none of the participants were involved. The participants watched the standardised footage until they were familiar with the game-play situation and were asked to concurrently verbalise their thought processes. Akin to the earlier clips, the same series of open-ended questions that focused on cognitive processes and critical information were asked. Following the interview, verbal recordings were transcribed verbatim.

Data analysis
Data from the in-game verbal reports and semi-structured interviews were combined and analysed using content analysis, incorporating grounded theory (Glaser & Strauss, 1967). Content analysis is defined as "objective, systematic and quantitative description of the manifest content of communication" (Berelson, 1952, p. 19), whereby the process is to explore the text for themes rather than observation-based notes (Patton, 2002). The grounded theory approach requires the researcher to become immersed in the data, so meanings can be identified, with specific observations leading towards general patterns and themes (Glaser & Strauss, 1967). Therefore, the focus of the analysis will be on the manifest meaning, and not the connotative meaning (or latent content) of the verbal report (Riffe, Lacy, & Fico, 1998). To ensure the data were trustworthy, two methods were implemented. Initially, the first author concurrently listened to the audio recording and read the transcripts, which ensured no errors were included in the transcripts. Second, member checking was implemented, with all participants confirming the information within the transcripts were accurate, and any identified errors were corrected.
Following data checking, the first and second author separately read the transcripts until they were familiar with the content. A cross-case analysis of the qualitative data (concurrent in-game verbalisation and interview verbalisation data) was conducted with an integrated approach of the results presented from the three individual cases (Creswell, 2007). As the aims of the study were to identify the decision-making processes of the participants, only comments relating to decisionmaking and game management were coded and used for analysis. As a result, comments relating to physical fitness or general nongame-specific conversations were not coded or assessed in the analysis (e.g. "are you feeling tired" & "did you hear that spectator"). Therefore, coding of all the in-game transcripts identified 670 comments, with a further 460 comments coded from the semistructured interviews for analysis. Raw data (i.e. meaningful quotes) were separately coded by two authors. This process led to an initial identification of nine themes considered important to understand the processes associated with Australian football umpires' decision-making. After the raw data were analysed, the first and second author discussed and operationally defined each theme. As a result, some themes were combined within more broad groupings based on belongingness. Three themes emerged from the analysis including (i) knowledge of game-play, (ii) player intention during game-play and (iii) decision evaluation (see Table 1 for operational definitions).
To assess the trustworthiness of the data both, inter-coder and intra-coder agreement were assessed. To assess inter-coder agreement an independent assistant and the first author coded all transcripts. The percentage of agreement was then determined using Cohen's Kappa, with an inter-coder agreement of 85% indicating a high level of agreement (Riffe et al., 1998). Intra-coder agreement was established by coding the transcripts on two separate occasions, 14 days apart, as this time interval is likely to determine random differences in the coding rather than changes in behaviour or ability (Pedhazur & Pedhazur-Schmelkin, 1991). An intra-rater agreement of 89% was reached indicating a high level of agreement (Riffe et al., 1998).

Results and discussion
Analysis of both the in-game and interview data revealed two types of verbal data: internal (i.e. self-cognitions or personal thoughts that umpires verbalised during the game), and external dialogue (i.e. verbalized inter-personal communication between the umpires and the players), which will be discussed. Analysis of the verbal reports identified three salient themes including (i) knowledge of game-play, (ii) player intention during game-play and (iii) decision evaluation. Table 1. The definition and number of times each theme was discussed by the participants during the data collection phase (i.e. in-game verbalisation and semi-structured interviews).

Theme name Number Definition
Knowledge of game-play 46 Self-directed communication that demonstrated knowledge of future player actions and/or game-play. Player intention during game-play 88 The interpretation of a player's objective during game-play (e.g. body movements or actions) that guided an infringement-based decision Decision evaluation 181 The procedure that contributed to a decision outcome

Knowledge of game-play
The "internal" theme of knowledge of game-play was defined as self-directed verbal communication that demonstrated knowledge of future player actions and/or game-play. Investigators have shown skilled performers use visual information early in an action sequence to predict the next act of play (Abernethy & Russell, 1987;Larkin et al., 2011), which is an important skill in Australian football, as umpires need to perceive the action sequence to assist in positioning themselves appropriately to view the next ball contest (Larkin et al., 2011). Participant 1 (i.e. P1) demonstrated knowledge of game-play by anticipating potential infringements, "The ball is coming forward, and could be hands in the back (pause). Nah nothing, play on." In this example, P1 recognised early in the action sequence that a push in the back infringement was a likely outcome, but delayed his decision (as indicated by the pause) until contact had occurred. Anticipating but delaying judgement until contact is made is important, particularly as Australian football umpires are instructed to watch the whole incident before making a decision, because it may allow them more time to consider whether the decision is correct. In another situation where two players contested the ball on the ground, P1 stated what infringement was likely, "I'm watching for high contact on him (high tackle infringement) (pause). Fair tackle play on." Again, after perceiving the information P1 paused, again viewing the whole incident prior to making an informed decision to allow the play to continue without an infringement being called. Both of these examples demonstrate P1's ability to assess the game scenario and generate possible decision outcomes based on advance cue information. The ability to use advance cue information has been shown to be an attribute of skilled decision-making performance in athletes and gymnastic judges (Ste-Marie, 1999;Vaeyens, Lenoir, Williams, & Philippaerts, 2007). Participant 2 (i.e. P2) provided an example of knowledge of game-play through his player management skills near the conclusion of a game where one goal (six points) separated the two teams. The game became tense with players committing several infringements to either gain an advantage or prevent the opposition from scoring as P2 stated, "A few players getting edgy; as the game gets close, they start to lose the plot. I have to watch play behind the ball more." This illustrates P2's knowledge and understanding of how players become more nervous and tense (i.e. edgy) during the final stages of a close game, with the potential for players to act erratically (i.e. lose the plot). P2 recognises this change in the player's mannerisms and identifies the increased likelihood of player contact infringements. Consequently, knowledge of game context altered P2's game management style as he consciously shifted his attention to potentially illegal off the ball player confrontations. Similarly, Participant 3 (i.e. P3) monitored player behaviour by stating, I'm just pre-empting; Player 1 came through with a lifted elbow (near head height of an opposition player), if an opposition player seen that they may hit him … Its player management, less likely for biffo (i.e. the player's engaging in physical confrontation) if the players think we're onto it.
Even though the player did not commit an infringement, P3 identified that the action may have further consequences on the game (such as opposition players retaliating). Thus, both P2 and P3 illustrated a sound understanding of player behaviour and were proactive in managing these incidents as a duty of care to the players (Gabbe & Finch, 2000).
In the Cornerstones Model for Refereeing Performance, Mascarenhas et al. (2005) identified the importance of an umpire being able to "alter his or her style of refereeing to suit the particular nuances of the game" (p. 386). Based on this description, umpires should understand the way the game is played to effectively manage the game environment and alter their umpiring style based on the game context. The participants demonstrated the ability to alter their personal umpiring style based on changes within the game environment, such as game context (time and score) or player changes (personal performance changes), which exhibits an experienced level of performance (Ward & Williams, 2003).

Player intention during game-play
The theme player intention during game-play has been defined as the interpretation of a player's objective during game-play (e.g. body movements or actions) that guided an infringement-based decision. P1 used his interpretation of a player's objective to inform his decision when two players contested for a mark. In the "marking" situation, the defending player had two teammates within close vicinity of the contest who could potentially gain possession of the ball and clear it from the defensive end of the ground. Participant 1 provided his interpretation of the situation, by suggesting, This player (attacking player) is going for the ball, this bloke (defending player) is thinking I am just not fit enough so I am going to take him out of it (the contest for the ball), because I have got two team mates who are going to take over (gain possession of the ball) and try and get that ball out (of defence).
Participant 1 interpreted that the defending player's objective was to illegally impede the opposition player (e.g. "I am just not fit enough so I am going to take him out of it") by either holding or pushing the opposition player away from the contest for the ball to ensure his teammate gained possession.
Participant 3 used his interpretation of a player's intention when two players contested for a ball during a marking contest. Participant 3 described his interpretation of the marking contest, and why he penalised the player for a holding the man infringement when he stated, "The players intention was to hold up the other player and get him out of the (marking) contest (holding the man infringement) … the intention has got to be the ball for both players." In this scenario, Participant 3 interpreted the intention of the player as illegally attempting to obtain possession of the ball and used this judgement to inform the final decision.
Participant 2 provided an example of this theme when he viewed a clip of a player on the ground in possession of the ball. The player dived on the ball and dragged it underneath himself, which according to the rules, is penalised for the infringement "holding the ball" if he does not immediately knock the ball clear, or correctly dispose of the ball. P2 explains his justification for the decision by stating: His first objective was to dive on the ball and drag the ball in, and once he has done that he has lost all right to be over the ball … he has made no attempt (to dispose of the ball), so holding the ball (infringement). Participant 2 interpreted the player's actions and used this information to inform his final decision, to penalise the player for holding the ball.
The player intention during game-play theme identifies a component of an Australian football umpire's decision-making process that has not been considered within the extant literature. The interpretation of player's intentions may also be used for the officiating of other sports. For example, in soccer, the offside rule indicates that assistant referees must consider the intentions of the player when making a decision. The rule states that a player is in an offside position if they are closer to their opponent's goal line, than both the ball and second last defender, and if, in the referee's opinion, they are actively involved in the game-play (Fédération Internationale de Football Association, 2013). As the offside rule requires assistant referees to interpret whether the player intended to be actively involved in the game-play, there is potential for the subjective interpretation of the player's actions to contribute to decision-making errors. Researchers have found that incorrect offside decisions were commonly due to errors in assistant referees' positioning (Helsen, Gilis, & Weston, 2006;Oudejans et al., 2000), however no investigations have considered whether the official's interpretation of a player's intention contributes to decision-making errors.

Decision evaluation
Decision evaluation was defined as the procedure that contributed to a decision outcome. It was apparent that a key stage of P1's decision-making process was the elimination of possible decision outcomes prior to the final decision. A situation where this was evident included a holding the ball decision, in which P1 stated, With a holding the ball decision you have got to eliminate all the other free kicks first. The first thing is, is the tackle legal? Was it too high? No. Next step, was it too low? No. Eliminate all those outside pieces and then you go 'has he had prior opportunity (to dispose of the ball)?' Yes. Did he have a chance to get rid of it (the ball)? Yes, gone (holding the ball infringement).
This example illustrates the explicit cognitive steps P1 undertook when interpreting a possible holding the ball infringement. Prior to the final infringement decision, P1 eliminated all other possible infringements, such as an illegal tackle (i.e. too high, too low). Participant 1 then determined whether the player with the ball had prior opportunity or reasonable time to legally dispose of the ball before being tackled. In this instance P1 deemed the player to be tackled legally and have prior opportunity to legally dispose of the ball. Therefore, P1's final decision for this situation was a free kick for a holding the ball infringement.
Participant 2 also used an elimination process in the same way during tackle situations: "Player 1 has got the ball, he has had prior opportunity (to dispose of the ball), he was tackled legally, and he has got the ball away (legally disposed of the ball), instantaneously. So play on." This example indicates that P2 explicitly processed one aspect of the incident (i.e. prior opportunity) before processing the next (i.e. legal tackle) and that both players conform to the rules (i.e. legal tackle, legally disposes of the ball), therefore the elimination process stops and play on is indicated (no infringement).
Participant 3 also described the elimination process of a tackle situation: "If he doesn't get taken high, is it a correct tackle? Did he fall into his back? No. You got to eliminate." This example explicitly indicates that Participant 3 used a cognitive elimination process to determine the final decision in player contact situations.
Sporting officials' decision-making has been assessed (or trained) via several perceptual-cognitive video-based tools (Catteeuw et al., 2009;Larkin et al., 2011;Schweizer, Plessner, Kahlert, & Brand, 2011), however, these investigations have not reported the cognitive process when making a decision. To provide an understanding of the demands of sport-based decisionmaking on athletes, Farrow and Raab (2008) presented the Decision-Making in Sport Model. The model identifies seven key decision-making stages that an athlete sequentially moves through in order to make a decision about in-game decisions. Within the model, the stage where athletes generate and consider possible decision outcomes has been identified as a key process within the decision-making process of sport performers. The Decision-Making in Sport Model states that skilled athletes consider possible decision options and then rank these options to form the final decision. This may be an appropriate method for athletes who make movement or tactical decisions, however, this model may not sufficiently explain the decisionmaking process of an umpire, whereby based on participants in the current study, a decision outcome may be selected following an elimination process. Tversky (1972) stated that any decision is subject to a sequential elimination process, whereby each decision outcome is the result of an elimination process of multiple sub-decisions. A sub-decision is considered and if eliminated the next sub-decision is considered. This process of sub-decision elimination or selection continues until the final sub-decision is accepted. Both P1 and P2 indicated that they considered and eliminated multiple sub-decisions prior to the final decision outcome for each player contact incident. Existing sport-based decision-making models are limited in this context as they are not umpire specific, but focus on the decision-making process of athletes generally. Furthermore, the umpire specific model does not provide an indication of the specific decision-making processes (Mascarenhas et al., 2005). For these reasons, further research with a greater number of umpires may guide the development of an umpire specific decisionmaking model.
The elimination process used by the participants within this study highlighted the complexities associated with the decision-making process in Australian football. This is akin to other sports (e.g. soccer), where umpire decision-making has been identified as a complex process (Ollis, MacPherson, & Collins, 2006). Based on this qualitative analysis, and Australian football rules, two decision-making heuristics illustrating the elimination process for two different scenarios were developed. Figure 2 illustrates the elimination process for a tackle situation, and Figure 3 identifies the elimination process for a marking contest.
Comparison of the two decision-making heuristics highlights that one of the challenges associated with understanding the decision-making process in Australian football is that the elimination process is situation specific. The Decision-Making in Sport Model (Farrow & Raab, 2008) indicates that for each decision the decision-making process is consistent, however, the current results indicated that there may not be a consistent process for every in-game decision because of the variation among in-game situations. Both Figures 2 and 3 illustrate a similar elimination process for each decision, however the number of options and the potential cognitive load associated with the specific game situation (e.g. tackle, mark) is potentially different. Therefore, there must be consideration of the different decision-making processes that occur during different game situations to adequately describe the in-game decision-making process of Australian football umpires.
While this study is an important step in understanding the decision-making process of Australian football umpires, as the heuristics are based on the processes of three umpires, further investigations are needed to ensure all possible options are included in the heuristics. A research study with a greater number of umpires would also enable decision-making heuristics to be developed for all scenarios in Australian football. As indicated by Plessner and Haar (2006), there are several sub-tasks within a decision-making situation which may contribute to a correct or incorrect decision. By identifying the steps within the decision-making process for different infringement situations, it may be possible to identify potential issues within the decision-making process. Therefore, the development of further heuristics for each in-game scenario may provide a reference for identifying specific areas of decision-making mistakes and potentially inform umpire decision-making training programmes.

Conclusions
This study makes a significant contribution to the perceptual-cognitive literature as it is the first attempt to explore the cognitive processes that contribute to the decision-making processes of Australian football umpires. Decision-making skills are fundamental to the umpiring process in all sports (Helsen & Bultynck, 2004), however there has been minimal investigation of the processes that contribute to the in-game decision-making of umpires. The current exploratory study identified three themes that related to both the in-game decision-making process (i.e. decision evaluation and player intention during game-play) and in-game umpire performance (i.e. knowledge of game-play) providing initial evidence of the potential factors that may contribute to and/or affect the in-game decision-making process of Australian football umpires. These findings have led to the preliminary development of decision-making heuristics that may provide a better understanding of the decision-making process of Australian football umpires than current sport-based decision-making models (e.g. Farrow & Raab, 2008). Further pursuit of the factors that significantly impact the in-game decision-making process of Australian football umpires is needed to potentially inform future Australian football umpire decision-making training programmes.
While this study provides an understanding of the decision-making process of Australian football umpires, the findings should however, be considered in respect to methodological limitations. As the current investigation used stimulated recall and concurrent verbalisation methods to provide a description of the conscious in-game decision-making processes, cognitive processes unavailable to conscious awareness (i.e. implicit cognitions) during in-game decision-making may not have been identified during the interviews. Therefore, future investigations may use standardised decision-making situations and more sophisticated technologies, such as eye tracking (Hancock & Ste-Marie, 2013) or option generation techniques (Raab & Johnson, 2007), to provide information about subconscious visual search patterns and the generation of decision options and the associated impact on the decision-making process. This type of research will further develop the understanding of umpire decision-making performance presented here. In addition, the study is limited by the level of expertise of the participants. Due to the within game data collection methods we were unable to recruit elite level performers. While we were able to recruit participants who have been identified by elite level coaches and participated in elite level training programmes, researchers should consider exploring the decision-making processes of elite level umpires. This may provide information indicating different decision-making processing strategies which may inform umpire decision-making training programmes. Finally, the current investigation did not collect data relating to the physiological performance of the participants during the game. While previous investigations have indicated that physiological capacity does not influence decision-making performance Paradis et al., 2015), future investigations should consider whether how physiological capacity may influence the in-game decision-making processes of umpires.
From a practical perspective, based on the findings associated with umpire decision-making performance, a key component of in-game decision-making was the process of interpreting a player's objective or intent during a game action. This process, however, may contribute to inconsistent decision-making outcomes between umpires because each umpire may interpret the intention of the player differently. Therefore, umpire coaches may contemplate the introduction of novel training activities, such as video-based training, to assist the development of skills associated with the interpretation of player's intention during game actions.