Overinterpretation of research findings: evidence of 'spin' in systematic reviews of diagnostic accuracy studies

Session: 

Oral session: Diagnostic test accuracy review methods (1)

Date: 

Tuesday 18 September 2018 - 11:20 to 11:40

Location: 

Carrick 2

All authors in correct order:

McGrath TA1, McInnes MD2, van Es N3, Leeflang MM4, Korevaar DA4, Bossuyt PM4
1 University of Ottawa Department of Radiology, Canada
2 University of Ottawa Department of Radiology; Clinical Epidemiology Program Ottawa Hospital Research Institute, Canada
3 Department of Vascular Medicine, Academic Medical Center, Amsterdam, The Netherlands
4 Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Centre, University of Amsterdam, The Netherlands
Presenting author and contact person

Presenting author:

Trevor McGrath

Contact person:

Trevor McGrath
Matthew McInnes
Abstract text
Background: 'Spin' has been identified as an issue that may distort readers' perception of the biomedical literature. This phenomenon has been studied and identified in both primary diagnostic accuracy research and systematic reviews of interventions, but not yet in systematic reviews of diagnostic test accuracy.

Objectives: To assess the frequency of overinterpretation ('spin') in systematic reviews of diagnostic accuracy studies.

Methods: We searched MEDLINE through PubMed from December 2015 to January 2016. We included systematic reviews of diagnostic accuracy studies in English if they reported one or more meta-analyses. We built and piloted a list of 10 items that represent actual overinterpretation in the abstract and/or full-text conclusion, and a list of nine items that represent potential overinterpretation. Two investigators independently used the items to score each included systematic review.

Results: We included 112 systematic reviews. The majority had a positive conclusion regarding the accuracy or clinical usefulness of the investigated test in the abstract (n = 83; 74%) and full-text (n = 83; 74%). Of the 112 reviews, 81 (72%) contained at least one actual form of overinterpretation in the abstract, and 77 (69%) in the full-text. This was most often a 'positive conclusion, not reflecting the reported summary accuracy estimates' in 55 (49%) abstracts and 56 (50%) full-texts. Of these 112 reviews, 107 (96%) contained a form of potential overinterpretation, most frequently 'non-recommended statistical methods for meta-analysis performed' (n = 57; 51%).

Conclusions: Most recent systematic reviews of diagnostic accuracy studies present positive conclusions and the majority contain a form of overinterpretation. This may lead to unjustified optimism about test performance and erroneous clinical decisions and recommendations.

Patient or healthcare consumer involvement: Recently, a phenomenon called 'spin' has been identified in the biomedical literature. This refers to reporting practices that mislead readers by being more optimistic about the diagnostic test under investigation than the results of the study justify. 'Spin' can bias readers' interpretation of scientific studies. For diagnostic accuracy studies, this may lead to inappropriate application of tests, which could harm patients and unnecessarily increase healthcare costs.

Attachments: 

PDF icon Table 1 - Spin.pdf

Relevance to patients and consumers: 

Recently, a phenomenon called “spin,” has been identified in the biomedical literature. This refers to reporting practices that mislead readers by being more optimistic about the drug, intervention, or diagnostic test under investigation than the results of the study justify. Such studies are often accompanied by optimistic claims about the accuracy or clinical usefulness of the investigated drug, intervention, or test. "Spin" can bias readers' interpretation of scientific studies. For diagnostic accuracy studies, this may lead to inappropriate application of tests, which could harm patients and unnecessarily increase healthcare costs.