Are retracted studies affecting our reviews?

ID: 

114

Session: 

Poster session 1

Date: 

Sunday 16 September 2018 - 12:30 to 14:00

All authors in correct order:

Pardo Pardo J1, Harbin S2, Welch V3
1 Cochrane Musculoskeletal, Canada
2 Cochrane Back and Neck, Canada
3 Campbell Collaboration, Canada
Presenting author and contact person

Presenting author:

Jordi Pardo Pardo

Contact person:

Jordi Pardo Pardo
Abstract text
Background:
Although most of the scientific literature is sound, there are cases of fraud in the scientific arena. Retraction is the mechanism used by publishers to indicate that a study is problematic.

Objectives:
To assess how Campbell and Cochrane Reviews have dealt with retracted studies, and to calculate the prevalence of retracted studies in the Cochrane Database of Systematic Reviews.

Methods:
We searched the Cochrane Library through the Wiley interface, looking for the word "retraction" in the full text of the review. We searched full text of Campbell reviews by using the site restricted search in Google. To calculate the prevalence of retractions, we used the function “included in review” and “in Medline” of the Cochrane Register of Studies on-line, retrieve the unique identifiers for each study and crossed them with "Retracted Publication" heading on Publication Type.

Results:
For Cochrane Reviews, 37 reviews have identified retracted studies, and including a total of 63 retracted studies. Thirty-four reviews excluded the retracted studies, noting this in the table of excluded studies. Of the 37 reviews, 70% had only one study retracted. The range of retractions went from one to nine. Two reviews attempted sensitivity analysis to assess the impact the retracted studies would have had, if included. Of the three reviews that included retracted studies, one noted the retraction and announced that the study would be withdrawn in a future update. The other two considered that the reasons for retraction of the studies did not invalidate the data, and decided to include the study.
For Campbell reviews, no reviews had mentioned retracted studies.
For the prevalence of retracted studies, a preliminary search limited to one Cochrane Group shows it affects a minority of all included studies: only 0.0025% (4/1625) are retracted studies.

Conclusions:
Retracted studies are an infrequent problem, but they appear often enough to potentially influence the results of some Cochrane Reviews. A simple exclusion of retracted studies might lead to publication bias. Careful consideration of the exclusion of the studies is needed, and to evaluate the impact that any decision could have in the results of the review in a sensitivity analysis.

Patient or healthcare consumer involvement:
No patients were involved in developing this project.

Relevance to patients and consumers: 

This study looks at how Cochrane reviews address retraction of studies. Studies get retracted because of scientific malpractice or mischief. Having Cochrane review based on potentially fraudulent data could bias the results and misinform the readers about the true value of the intervention under study. On the other hand, depending on the malpractice, not including the results of a retracted study might lead to publication bias. Patients, as readers of the reviews, deserve to have the most unbiased view of all the available evidence.