Deciding when to include non-randomized studies of interventions in a Cochrane Review of an intervention

Workshop category: 

  • Inclusion of non-randomized designs
Date and Location

Date: 

Sunday 16 September 2018 - 16:00 to 17:30

Location: 

Lennox 1
Contact persons and facilitators

Contact person:

Barnaby Reeves

Facilitators:

Reeves B1, Shea B2
1 University of Bristol, United Kingdom
2 Ottawa Hospital Research Institute, University of Ottawa, Canada

Acknowledgements:

Tugwell P1, Wells G2
1 Musculoskeletal review group, University of Ottawa, Canada
2 Non-randomized studies methods group, University of Ottawa, Canada
Target audience

Target audience: 

Review authors, consumers, editorial staff and methodologists, who know the current Handbook requirements for a protocol for a Cochrane systematic review of the effects of an intervention.

Level of difficulty: 

Intermediate
Type of workshop

Type of workshop : 

Discussion
Abstract

Abstract:

Background:
Cochrane Reviews require a protocol prespecifying the methods. When non-randomized studies of interventions (NRSIs) are to be included, protocols need to specify additional details. Authors are encouraged to consider whether NRSIs should be included, and to justify the decision in the protocol [1]. Alternatively, this decision could be made during the conduct of the review. The first approach requires a decision to be made with incomplete information, but allows the protocol to focus on eligible study designs. The second approach requires all protocols to specify the methods to be used for both RCTs and NRSIs and criteria for the decision.
Objectives:
- Present the two options.
- Describe the additional information needed in a protocol that includes NRSIs.
- Elicit opinions about the pros and cons of the two options.
Description:
The following questions are relevant to a decision about including NRSIs:
- how many participants are available in RCTs?
- do the PICO (participants, intervention, comparator, outcomes) questions evaluated by RCTs address the review question directly? [2]
- Do the PICO questions evaluated by NRSIs address the review question directly? [2]
- Are the NRSIs, or a subset of them, reasonably free of bias?
Review authors can answer these questions either by a rapid scoping exercise (approach 1) or by appraising studies identified from searches during the review (approach 2).
Protocols for reviews that intend to include NRSIs often fail to specify adequately: the types of eligible NRSI (considering their intrinsic ability to estimate a causal treatment effect) [3,4]; search strategy; confounding domains; methods to assess the risk of bias. Other review features are often not well specified, irrespective of the included study designs: elements of the review question; outcome domains; strategy for synthesising data across time points and outcome measures. These inadequacies can mean that decisions about methods are made after the review has started, or not at all, and hinder interpretation of review findings.
Given the need to justify inclusion of NRSIs and the protocol details required, participants will discuss the pros and cons of the two approaches. The facilitators will present the rationale for additional details in protocols and seek participants’ opinions about the priority of including this information.
Attachments

Attachments: 

PDF icon Reeves_workshop_Edinburgh2018_references.pdf

Relevance to patients and consumers: 

Non-randomized studies of interventions (NRSIs) may be required to address a review question fully for the benefit of consumers. However, NRSIs are more complex (e.g. more diverse, likely to be flawed), make reviews harder to do (e.g. need a team with more diverse expertise and methods to manage the diversity between NRSI) and require protocols that specify relevant methods. At a time when Cochrane is focusing on the timeliness of review publication, the decision about whether to include NRSI in reviews is critical. This workshop will explore the pros and cons of scoping before drafting a protocol so that the protocol is fit-for-purpose.