Assessment of the completeness of reporting of methods in overviews of systematic reviews

ID: 

268

Session: 

Poster session 2

Date: 

Monday 17 September 2018 - 12:30 to 14:00

All authors in correct order:

Lunny C1, McDonald S2, Brennan S2, McKenzie JE3
1 Cochrane Australia, Monash University, Australia
2 Cochrane Australia, Australia
3 School of Public Health and Preventative Medicine, Monash University, Australia
Presenting author and contact person

Presenting author:

Carole Lunny

Contact person:

Carole Lunny
Joanne McKenzie
Abstract text
Background:
Overviews of reviews should be fully and transparently reported to allow readers to assess the adequacy of the methods used and whether there are any threats to the validity of the overview findings.

Objectives:
To assess the completeness of reporting of methods in overviews.

Methods:
We used a published framework of methods for conducting an overview as the basis for assessing the adequacy of the reporting of the methods. A random sample of 50 overviews was identified through a search of MEDLINE and independently screened by two authors. One author assessed the completeness of reporting for all overviews, while a second author independently assessed 37 overviews.

Results:
A clinical rationale for undertaking the overview was reported in 29/50 and 26/50 defined the overviews' PICO elements (population, interventions, comparisons, outcomes). A full search strategy was reported in 18/50 overviews. Almost all overviews (49/50) searched bibliographic databases and 44/50 searched systematic review databases (e.g. Cochrane, Epistemonikos). Nearly half of the overviews (24/50) reported a process for selecting systematic reviews and for extracting data (23/50). Methods for assessing the risk of bias (RoB)/quality of the systematic reviews were reported in 38/50 overviews but were infrequently incorporated into the results and conclusions (14/38). Tools to assess RoB/quality of the systematic reviews included AMSTAR (23/50), OQAQ (9/50), R-AMSTAR (2/50) and ROBIS (1/50). Thirteen of 50 overviews reported how methods for RoB/quality of primary studies were assessed. No overviews reported how different tools to assess primary studies' RoB/quality were dealt with across systematic reviews. Few overviews (10/50) sufficiently detailed the methods used for narrative summary and one-quarter (12/50) reported conducting "narrative synthesis" without any further description of the method. However, fewer (3/50) reported methods of quantitative synthesis. Of the 17/50 overviews that reported assessing the certainty of the evidence (CoE), about half used GRADE (8/17), 5/17 used the Jadad tool and 2/17 used other tools to assess CoE.

Conclusions:
Results from this study provide a benchmark of the completeness of reporting of overview methods and indicate that improvement is needed. Development of a specific reporting guideline for overviews in combination with strategies to facilitate the uptake of the guideline are required.

Patient or healthcare consumer involvement:
None.

Relevance to patients and consumers: 

Overviews of reviews have been identified as an important review format for making evidence accessible to healthcare consumers. However, careful planning of methods is required to ensure that overviews present the most valid, useful, and complete information for decision-making. This research informs the need for, and content of, a reporting guideline for overviews, and will be useful for those faced with peer review of overviews who may require a checklist to guide their evaluations.