- Title
- Can intelligent agents improve data quality in online questionnaires? A pilot study
- Creator
- Söderström, Arne; Shatte, Adrian; Fuller-Tyszkiewicz, Matthew
- Date
- 2021
- Type
- Text; Journal article
- Identifier
- http://researchonline.federation.edu.au/vital/access/HandleResolver/1959.17/179504
- Identifier
- vital:15633
- Identifier
-
https://doi.org/10.3758/s13428-021-01574-w
- Identifier
- ISBN:1554-351X (ISSN)
- Abstract
- We explored the utility of chatbots for improving data quality arising from collection via sonline surveys. Three-hundred Australian adults sampled via Prolific Academic were randomized across chatbot-supported or unassisted online questionnaire conditions. The questionnaire comprised validated measures, along with challenge items formulated to be confusing yet aligned with the validated targets. The chatbot condition provided optional assistance with item clarity via a virtual support agent. Chatbot use and user satisfaction were measured through session logs and user feedback. Data quality was operationalized as between-group differences in relationships among validated and challenge measures. Findings broadly supported chatbot utility for online surveys, showing that most participants with chatbot access utilized it, found it helpful, and demonstrated modestly improved data quality (vs. controls). Absence of confusion for one challenge item is believed to have contributed to an underestimated effect. Findings show that assistive chatbots can enhance data quality, will be utilized by many participants if available, and are perceived as beneficial by most users. Scope constraints for this pilot study are believed to have led to underestimated effects. Future testing with longer-form questionnaires incorporating expanded item difficulty may further understanding of chatbot utility for survey completion and data quality. © 2021, The Psychonomic Society, Inc.
- Publisher
- Springer
- Relation
- Behavior Research Methods Vol. 53, no. 5 (2021), p. 2238-2251
- Rights
- All metadata describing materials held in, or linked to, the repository is freely available under a CC0 licence
- Rights
- Copyright @ The Psychonomic Society, Inc. 2021
- Subject
- 0801 Artificial Intelligence and Image Processing; 1701 Psychology; 1702 Cognitive Sciences; Autonomous conversational agent; Chatbot; Data quality; Online survey research; Questionnaire item confusion; Response accuracy
- Reviewed
- Hits: 1553
- Visitors: 1330
- Downloads: 0
| Thumbnail | File | Description | Size | Format |
|---|