How often is a Cochrane Review the best available evidence? Evidence map using Epistemonikos

Session: 

Oral session: Priority setting (1)

Date: 

Sunday 16 September 2018 - 11:30 to 11:50

Location: 

Carrick 2

All authors in correct order:

Rada G1, Bravo G2, Ortiz L2, Zavala C2, Pérez-Bracchiglione J3, Morel M2, Meza N3, Madrid E3, Vásquez J1, Pérez M1
1 Epistemonikos foundation, Chile
2 Centro Evidencia UC, Pontificia Universidad Católica de Chile, Chile
3 CIESAL, Universidad de Valparaíso, Chile
Presenting author and contact person

Presenting author:

Gabriel Rada

Contact person:

Gabriel Rada
Gonzalo A Bravo-Soto
Abstract text
Background: Cochrane Reviews (CR) are acknowledged to be rigorous, and their average quality is better than non-Cochrane reviews (NCR). However, quality comes at a cost of currency, and the inability to conduct as many reviews as needed to answer all the possible questions.

Objectives: To explore how often CRs are available for clinical questions; if they outperform other NCRs; and if they include all the relevant evidence.

Methods: We developed a map of all the possible diagnostic or interventions questions (i.e. participants, interventions, comparators, outcomes [PICO] questions) in three conditions (acute appendicitis, common cold and Parkinson’s disease) through the evaluation of existing literature (systematic reviews [SR] and guidelines) and discussion with clinicians and consumers. We conducted searches in Epistemonikos database, screened potentially eligible SRs for each question and created a matrix of evidence (a table comparing SRs and their included studies).
We measured the number of available SRs for each question; the number of questions answered by a CR or non-Cochrane review (NCR); and the number of times a CR was the newer or more complete.

Results: We have collected 853 questions, screened 2328 records and identified 1144 relevant SRs (questions/records/included: acute appendicitis 484/163/217; common cold 58/484/85; Parkinson’s disease 311/1681/842).
Preliminary results: the average number of SRs per question is nine (range 0 to 84); we identified at least one SR in 94% of the questions, but only 35% were answered by a CR. For the questions for which a CR was available, in 52% the CR was the more complete, and in 30% it was the more recent.

Conclusions: Most questions can be answered with SRs, but coverage of CRs is far from ideal. When there are both CRs and NCRs, CRs are slightly more complete, but the currency of NCRs is better. Our results show that for a given real-life question there is rarely Cochrane evidence to trust. Major changes are needed in Cochrane's production model in order to reverse this problem.

Patient or healthcare consumer involvement: Patients have been involved in the generation of the question maps, including making sure all of the questions relevant for them have been addressed.

Relevance to patients and consumers: 

Cochrane reviews are key to make evidence-informed decisions. However, there is no Cochrane review for many questions. On the other hand, Cochrane reviews might have limitations or be out of date. So, our work maps all of the treatment and diagnosis questions in three conditions, and showed the following: most questions could be answered with systematic reviews, there were too many questions not covered by Cochrane reviews, and when there were both Cochrane and non-Cochrane reviews, Cochrane were slightly more complete, but less current than non-Cochrane.