Methodological challenges in performing network meta-analyses using data from Cochrane intervention reviews: are data across reviews reliable and consistent?

Session: 

Oral session: Network meta-analysis (1)

Date: 

Sunday 16 September 2018 - 12:10 to 12:20

Location: 

Carrick 3

All authors in correct order:

Imamura M1, Wallace S2, Brazzelli M1, Hudson J1, MacLennan G3, Javanbakht M4, Moloney E5, Vale L6, Craig D5
1 Health Services Research Unit, University of Aberdeen, UK
2 Cochrane Incontinence Group and Evidence Synthesis Group, Institute of Health & Society, Institute of Health & Society, Newcastle University, UK
3 Centre for Healthcare Randomised Trials (CHaRT), Health Services Research Unit, University of Aberdeen, UK
4 Health Economics Group, Institute of Health & Society, Newcastle University, UK
5 Health Economics Group and Evidence Synthesis Group, Institute of Health & Society, Newcastle University, UK
6 Health Economics Group and Cochrane Incontinence Group, Institute of Health & Society, Newcastle University, UK
Presenting author and contact person

Presenting author:

Mari Imamura

Contact person:

Mari Imamura
Abstract text
Background: When there are several interventions for a single condition, several Cochrane intervention reviews (CIR) may cover the plethora of pairwise comparisons. Nevertheless, consumers and policy-makers face challenges in judging which intervention is best. Network meta-analyses (NMA) allow comparison of multiple interventions. A NMA using data from CIRs may reduce research waste and avoid duplication. However, it can also present some unique methodological challenges.
Objectives: To describe the challenges faced in compiling data from CIRs for a NMA.
Methods: We used a NMA of randomised controlled trials (RCT) of nine distinct surgical interventions for women with stress urinary incontinence (SUI). Primary outcomes were cure and improvement of SUI. Data came from eight CIRs from a single Cochrane Review Group (CRG). The CIRs provided relevant studies, outcome data from individual RCTs, study characteristics and findings of risk of bias (RoB) assessment. Where the same trial was used in two or more CIRs, we compared data extraction. When data were inconsistent or insufficient, we assessed primary trial reports. We updated literature searches to a common date and reviewed newly identified studies.
Results: We included 147 RCTs from eight CIRs and 28 new studies. A challenge for the NMA was caused by poor reporting in the original trial reports (e.g. number of participants in each group). This led to CIR authors making assumptions in order to use data, but limited reporting in the CIRs meant that extracted data from the CIR could not be replicated or verified. Additionally, methods also varied between CIRs; for example, outcomes were grouped into categories using different criteria. Study characteristics were also inconsistently reported across reviews as were descriptions of adverse events. The different publication dates of the CIRs, meant different versions of the Cochrane RoB tool were used.
Conclusions: Considerable additional work was required to make data from the CIRs useable. The NMA also served as a methods audit and highlighted the need to ensure consistency across reviews addressing substantially the same participants, intervention, comparator and outcomes (PICO). CIRs and CRGs need to ensure assumptions for handling missing or ambiguous data are consistent and clearly described, and CRGs need to ensure consistency of methods.

Relevance to patients and consumers: 

Several Cochrane reviews address the effects of surgery for the treatment of stress urinary incontinence (SUI) in women. Each review typically assesses one operation versus alternatives. It is challenging for consumers to answer the question of which operation is most effective and safe. A network meta-analysis (NMA) comparing multiple interventions simultaneously can help. Taking data for a NMA from existing Cochrane reviews may be efficient but presents unique methodological challenges. This study explores these challenges using the experience from an evidence synthesis of surgical interventions for the treatment of SUI.