Prevalence and types of 'Risk of bias' assessments across systematic reviews in sports and exercise medicine

Session: 

Oral session: Investigating bias (3)

Date: 

Tuesday 18 September 2018 - 11:40 to 11:50

Location: 

Carrick 1

All authors in correct order:

Büttner F1, Winters M2, Ardern C3, Delahunt E1
1 UCD School of Public Health, Physiotherapy & Sports Science, University College Dublin, Dublin, Ireland
2 Research Unit for General Practice in Aalborg, Department of Clinical Medicine, Aalborg University, Aalborg, Denmark
3 Division of Physiotherapy, Linköping University, Linköping, Sweden
Presenting author and contact person

Presenting author:

Fionn Cléirigh Büttner

Contact person:

Fionn Cléirigh Büttner
Abstract text
Background: Combining biased evidence propagates biased systematic review findings in the absence of an appropriately conducted 'Risk of bias' (RoB) assessment. Systematic reviews including research studies at high or unclear RoB yield inflated effect estimates compared to reviews possessing studies at low RoB. The type of tool selected and evaluation method used to undertake a RoB assessment are essential for a true interpretation of review findings.

Objectives: To determine RoB assessment prevalence, tool types, and evaluation methods across systematic reviews in a leading international sports and exercise medicine journal.

Methods: A systematic search for review articles in the British Journal of Sports Medicine archive identified and included systematic reviews published from 1 January 2016 to 10 May 2017 (Figure 1). Independently, two authors screened review titles and abstracts and, following agreement in this process, abstracted descriptive information related to each systematic review manually, using a predetermined data extraction template.

The primary variables of interest included:
1) prevalence of RoB assessments used across the included systematic reviews;
2) type of RoB tool used;
3) type of RoB assessment performed (e.g. domain-level, outcome-level, etc.).

Results: We included 66 systematic reviews, 65 (98.5%) of which undertook a critical appraisal of included studies. Only 11 (16.9%) implemented a RoB tool that evaluated the internal validity of included studies. To assess RoB, the remaining systematic reviews implemented tools that evaluated methodological quality (47.7%), reporting (1.5%), a combination of internal validity, external validity, methodological quality, and reporting (18.5%), or 'other' (15.4%). Twelve (18.5%) reviews performed an outcome-level RoB assessment. Only three (4.6%) systematic reviews undertook an outcome-level RoB assessment in the context of bias-specific domains.

Conclusions: RoB assessments are commonly used across systematic reviews in a leading international sports and exercise medicine journal. However, suboptimal tools and assessment methods, which evaluate domains extraneous to internal validity, are frequently implemented to perform RoB assessments. This substantially limits the extent to which readers can interpret the effect of bias on systematic review findings.

Attachments: 

PDF icon büttner_fionn_cochrane_2018_abstract_figure.pdf

Relevance to patients and consumers: 

This research initiative, intended as an educational review in the British Journal of Sports Medicine, will educate journal readers about the importance of appropriate risk of bias assessment conduct and interpretation. An educational review that enhances clinicians' interpretation of bias affecting the credibility of review findings is pertinent for the identification of true effect estimates in systematic reviews, rather than potentially biased, inflated findings. Improved identification of bias-limiting conclusions within systematic reviews will empower clinicians to select best-evidence patient outcomes to optimise patient management.