Are the search and data extraction procedures similar between Cochrane Reviews and systematic reviews from other high impact factor journals?

ID: 

279

Session: 

Poster session 2

Date: 

Monday 17 September 2018 - 12:30 to 14:00

All authors in correct order:

Porfírio G1, Pacheco RL1, Parra MT1, Latorraca CDOC1, Martimbianco ALC1, Pachito D1, Riera R1
1 Cochrane Brazil, Brazil
Presenting author and contact person

Presenting author:

Rafael Pacheco

Contact person:

Rafael Pacheco
Abstract text
Background: Screening and retrieving information from primary studies is the basis of a robust and reliable systematic review (SR). The unbiased selection of included studies is one of the main objectives of the research synthesis. Since its origins, the methodology of how to screen and extract data was broadly studied.
Objectives: To compare the search data extraction procedures between Cochrane and high impact factor journals systematic reviews.
Methods: Cross-sectional study comparing a random sample of full Cochrane Reviews (published in 2015) versus SRs published in the six higher impact factor (IF) general journals (IF - JCR > 11,862, 2015). We extracted and compared data from the searching strategy and data extraction performed by the SRs and discussed the findings.
Results: We included 134 SRs (67 Cochrane and 67 non-Cochrane). Cochrane Reviews considered more databases for retrieving studies than non-Cochrane (mean, SD: 6.58, 4.01 versus 4.12, 4.01; P < 0.0002). There was a significant difference favoring the Cochrane Reviews regarding the variables: 1) providing at least one complete search strategy (risk ratio (RR) 1.44; confidence interval (CI) 95% 1.22 to 1.72); 2) no language restriction (RR 2.44; CI 95% 1.77 to 3.36); 3) no year of publication restriction (RR 2.13; CI 95% 1.50 to 3.01); 4) searching grey literature (RR 1.48; CI 95% 1.21 to 1.80); 5) searching clinical trials protocols register databases (RR 2.28; CI 95% 1.65 to 3.16); 6) study selection performed by more than one researcher independently (RR 1.20; CI 95% 1.07 to 1.35); 7) data extraction performed by more than one research independently (RR 1.31; CI 95% 1.14 to 1.50); and 8) the process used for solving disagreement (RR 1.36; CI 95% 1.18 to 1.58). There was no difference regarding the report of a PRISMA flow for selecting studies (RR 1.07; CI 95% 0.97 to 1.18).
Conclusions: Searching methodology was significantly more rigorous along Cochrane Reviews than in those from general high-impact factor journals. The questions that remain and merit investigation are: why has the Cochrane Library not achieved a higher IF so far? And what must Cochrane authors and members do to achieve this goal?
Patient or healthcare consumer involvement: Improving search and data extraction methods is crucial to produce reliable evidence that could be used for patients and healthcare consumers.

Relevance to patients and consumers: 

Improving search and data extraction methods is crucial to produce reliable evidence that could be used for patients and healthcare consumers for decision making.