NICE and Cochrane – how do evidence synthesis methods and interpretation compare?

Session: 

Oral session: Understanding and using evidence (2)

Date: 

Monday 17 September 2018 - 14:00 to 14:20

Location: 

Carrick 3

All authors in correct order:

Chong L1, Ftouh S2, Cox S3, O Mahony R4, Samarasekera E2, Ashe J2, Bellorini J3, Kitterick P5, Ferguson M5, Schilder A6
1 Ateimed Consulting Ltd, United Kingdom
2 National Clinical Guideline Centre, United Kingdom
3 Cochrane ENT Group, United Kingdom
4 NICE, UK
5 Nottingham Biomedical Research Centre, United Kingdom
6 Cochrane ENT, United Kingdom
Presenting author and contact person

Presenting author:

Lee-Yee Chong

Contact person:

Lee-Yee Chong
Saoussen Ftouh
Abstract text
Background:
NICE and Cochrane produce systematic reviews for interventions based on Cochrane and GRADE recommended methods. Both organisations also prioritise areas that impact on patient care and strive to use patient-important outcomes. Therefore, there is an opportunity to work together. Insight into the approach and methods used by NICE and Cochrane will facilitate such collaboration.

Objectives:
To identify similarities and differences in evidence synthesis and interpretation by NICE versus Cochrane.

Methods:
We compared the methods prescribed by the NICE Guideline Manual to those of Cochrane as outlined in the Cochrane Handbook and MECIR guidelines for systematic reviews of interventions.

Results:
Both organisations have nearly identical review methods. We identified minor differences in the following areas:

1) types of evidence/studies searched;
2) process of abstract screening and data extraction;
3) GRADE and evidence interpretation.

Cochrane requires searches for non-English publications and grey literature; this is not always required for NICE guidelines. To achieve quality control Cochrane requires abstract screening and data extraction by two independent review authors, whereas NICE uses other methods. Whilst Cochrane uses the GRADE criteria for systematic reviews, NICE uses these criteria for clinical guidelines. This has implications for the rating of two GRADE domains: imprecision and indirectness. NICE's reviews and ratings are focused on the NHS, whereas Cochrane Reviews have an international focus and therefore leave some value judgements, such as important thresholds of benefit and harm, to the users of the reviews.

Conclusions:
NICE and Cochrane share nearly identical methods for conducting systematic reviews. If current Cochrane standards such as MECIR are met, other factors such as currency of the evidence and choice of outcomes and comparisons are likely to be more important factors that affect whether Cochrane Reviews are fully used in NICE guidelines. Collaboration between Cochrane and NICE thus avoids duplication of work, saves resources and will benefit patients.

Relevance to patients and consumers: 

This abstract shows that NICE and Cochrane uses essentially the same methods to synthesise evidence. This provides patients and consumers with an insight into how clinical evidence is synthesized using systematic review methods by Cochrane and NICE. These are then incorporated into the NICE clinical guidelines development process to make clinical recommendations which directly affect patient care. They will also gain an appreciation on the important but subtle differences between evidence interpretation (what the research says) versus evidence application (what clinicians/patients should do) based on these discussions.