Identifying and counteracting fraudulent responses in online recruitment for health research: a scoping review.

Julie Ayre, Rajshri Roy, Rebecca Raeside, Josielli Comachio, Adam Poulsen, Adeola Bamgboje-Ayodele, Aidan Tan, Edel O'Hagan

Journal: BMJ evidence-based medicine 2025;30(3):173-182

PMID: 39715631

Free Full Text on HighWire View on PubMed

Abstract

OBJECTIVES

This study aimed to describe how health researchers identify and counteract fraudulent responses when recruiting participants online.

DESIGN

Scoping review.

ELIGIBILITY CRITERIA

Peer-reviewed studies published in English; studies that report on the online recruitment of participants for health research; and studies that specifically describe methodologies or strategies to detect and address fraudulent responses during the online recruitment of research participants.

SOURCES OF EVIDENCE

Nine databases, including Medline, Informit, AMED, CINAHL, Embase, Cochrane CENTRAL, IEEE Xplore, Scopus and Web of Science, were searched from inception to April 2024.

CHARTING METHODS

Two authors independently screened and selected each study and performed data extraction, following the Joanna Briggs Institute's methodological guidance for scoping reviews and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews guidelines. A predefined framework guided the evaluation of fraud identification and mitigation strategies within the studies included. This framework, adapted from a participatory mapping study that identified indicators of fraudulent survey responses, allowed for systematic assessment and comparison of the effectiveness of various antifraud strategies across studies.

RESULTS

23 studies were included. 18 studies (78%) reported encountering fraudulent responses. Among the studies reviewed, the proportion of participants excluded for fraudulent or suspicious responses ranged from as low as 3% to as high as 94%. Survey completion time was used in six studies (26%) to identify fraud, with completion times under 5 min flagged as suspicious. 12 studies (52%) focused on non-confirming responses, identifying implausible text patterns through specific questions, consistency checks and open-ended questions. Four studies examined temporal events, such as unusual survey completion times. Seven studies (30%) reported on geographical incongruity, using IP address verification and location screening. Incentives were reported in 17 studies (73%), with higher incentives often increasing fraudulent responses. Mitigation strategies included using in-built survey features like Completely Automated Public Turing test to tell Computers and Humans Apart (34%), manual verification (21%) and video checks (8%). Most studies recommended multiple detection methods to maintain data integrity.

CONCLUSION

There is insufficient evaluation of strategies to mitigate fraud in online health research, which hinders the ability to offer evidence-based guidance to researchers on their effectiveness. Researchers should employ a combination of strategies to counteract fraudulent responses when recruiting online to optimise data integrity.

Address: Sydney Musculoskeletal Health, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia [email protected].; Brain and Mind Centre, The University of Sydney, Sydney, New South Wales, Australia.; Biomedical Informatics and Digital Health, School of Medical Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia.; Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia.; NHMRC Clinical Trials Centre, Camperdown, New South Wales, Australia.; Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia.; Sydney Health Literacy Lab, Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia.; Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia.; Discipline of Nutrition and Dietetics, Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia.; Charles Perkins Centre, The University of Sydney, Sydney, New South Wales, Australia.; Westmead Applied Research Centre, School of Medicine, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia.

DOI: 10.1136/bmjebm-2024-113170