Assessing the validity of an ultra-rapid review strategy: four case studies from oncology and public health

Session: 

Oral session: Innovative solutions to challenges of evidence production (4)

Date: 

Tuesday 18 September 2018 - 14:50 to 15:00

Location: 

Carrick 1

All authors in correct order:

Affengruber L1, Wagner G1, Waffenschmidt S2, Nussbaumer-Streit B1, Thaler K3, Lhachimi S4, Patel S5, Gartlehner G6
1 Department of Evidence-based Medicine and Clinical Epidemiology, Danube University Krems, Krems & Cochrane Austria, Danube University Krems, Krems, Austria
2 Information Management Unit, Institute for Quality and Efficiency in Health Care, Cologne, Germany
3 Medical Department I, Hanusch Krankenhaus der Wiener Gebietskrankenkasse, Vienna, Austria
4 Research Group Evidence-Based Public Health, Leibniz Institute for Epidemiology and Prevention Research (BIPS), Bremen & Health Sciences Bremen, Institute for Public Health and Nursing, University of Bremen, Bremen, Germany
5 RTI-University of North Carolina Evidence-based Practice Center, RTI International, North Carolina & Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, North Carolina, USA
6 Department of Evidence-based Medicine and Clinical Epidemiology, Danube University Krems, Krems; Cochrane Austria, Danube University Krems, Krems & RTI International, North Carolina, Austria & USA
Presenting author and contact person

Presenting author:

Lisa Affengruber

Contact person:

Lisa Affengruber
Abstract text
Background: Rapid reviews have become a pragmatic alternative to systematic reviews (SR) because they meet time-sensitive needs of decision-makers. By simplifying methodological steps, results from rapid evidence syntheses are available within a few weeks or months. Ultra-rapid reviews (URR) employ multiple methodological shortcuts and are conducted in a very limited timeframe (a few days). Because of these shortcuts, URR have the potential to be less reliable than SR.

Objectives: The primary objective of this study was to assess whether an URR approach leads to different overall conclusions about treatment effects compared to a SR approach. This study also sought to explore whether practicality and validity of the URR strategy vary between clinical and public health topics.

Methods: We used a non-random (convenience) sample of four ongoing Cochrane Reviews as reference standards against which we compared the validity of the URR strategy with strongly abbreviated literature searches and single screening of abstracts and full texts. Two reviews addressed oncological topics and two were on public health topics.

For each topic we conducted a simple-structured Boolean search in combination with the first 20 similar articles in PubMed. The starter sets for the similar articles function in PubMed were highly relevant, published studies provided by the Cochrane review authors. Literature searches were conducted centrally by an information specialist. For each of the four topics, three reviewers screened literature independently.

We will survey Cochrane review authors to solicit whether the detected bodies of evidence from the URR would lead to different conclusions than the respective body of evidence detected by the Cochrane review. We will assess whether effect estimates differ between URR and SR.

Results: Final results will be available by the time of the Cochrane Colloquium.

Conclusions: Findings of this project will provide information about the practicality and validity of using an URR approach.

Patient or healthcare consumer involvement: Usually rapid reviews engage patients and healthcare providers in the review process to focus on patient relevant health outcomes. The synthesized evidence is then presented simply and concisely for informed decision-making.

Relevance to patients and consumers: 

Clinicians and health care decision makers often need quick answers for urgent questions. Producing reliable evidence syntheses in a very short time frame with an ultra-rapid review approach enables timely evidence-based decision making in healthcare which ultimately benefits the patients.