Mapping methods to deal with commonly encountered scenarios in overviews of reviews

Session: 

Oral session: Overviews of reviews and network meta-analysis (2)

Date: 

Monday 17 September 2018 - 14:10 to 14:30

Location: 

Fintry Auditorium

All authors in correct order:

Lunny C1, Brennan S2, Steve M2, McKenzie JE3
1 Cochrane Australia, Monash University, Australia
2 Cochrane Australia, Australia
3 School of Public Health and Preventative Medicine, Monash University, Australia
Presenting author and contact person

Presenting author:

Carole Lunny

Contact person:

Carole Lunny
Joanne McKenzie
Abstract text
Background: Overviews of reviews present unique methodological challenges for which authors need to plan. Through a systematic review, we have previously developed a comprehensive framework to catalogue method options available to overview authors. Here, we extend this work by linking the methods to commonly encountered scenarios in overviews.

Objectives: To provide guidance on methods for dealing with common scenarios in overviews.

Methods: We identified methodological challenges through systematic review of cross-sectional studies, guidance and commentaries on methods used in overviews. We used thematic synthesis to specify common scenarios for which authors need to plan, then mapped methods from our framework to each scenario.

Results: We included 42 studies for analysis. We identified seven common scenarios: overlapping data; discrepant data; missing or varying information; incomplete coverage of the overview question; out-of-date reviews; concerns about bias in included reviews; and discordant results or conclusions. We identified multiple methods for addressing each scenario. For example, a commonly cited approach for dealing with reviews with overlapping primary studies is to specify eligibility criteria (or decision rules) to select one review. However, multiple methods exist for addressing overlap at later steps of the overview. During synthesis, for example, authors can:
1) use decision rules to select one (or a subset) of meta-analyses with overlapping studies;
2) use statistical approaches to deal with overlap;
3) ignore overlap; or
4) acknowledge overlap as a limitation.
Alternatively, overlap may be addressed when assessing certainty of the evidence. Any of these approaches can be combined with methods to quantify and present overlap visually.

Conclusions: The methods literature often suggests a single approach for addressing common scenarios in overviews. However, our research identified multiple options that can be used at different steps, some of which can be combined. Our findings may usefully inform the methods to be used when planning an overview.

Patient or healthcare consumer involvement: None

Relevance to patients and consumers: 

Overviews of reviews have been identified as an important review format for making evidence accessible to healthcare consumers. However, careful planning of methods is required to ensure that overviews present the most valid, useful, and complete information for decision-making. This research aims to help overview authors make well-informed decisions when planning an overview.