Experience in using ROBINS-I for the assessment of controlled before-after studies

ID: 

181

Session: 

Poster session 1

Date: 

Sunday 16 September 2018 - 12:30 to 14:00

All authors in correct order:

Moore T1, MacAleenan A2, Kesten J1, López-López J2, Ijaz S1, Richards A1, Grey S3, Savović J1, Audrey S2
1 National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care West (NIHR CLAHRC West), University of Bristol, Population Health Science Institute, UK
2 University of Bristol, Population Health Sciences Institute, UK
3 University of the West of England, UK
Presenting author and contact person

Presenting author:

Theresa Moore

Contact person:

Theresa Moore
Abstract text
Background:
ROBINS-I assesses risk of bias (ROB) in non-randomised controlled studies of interventions. The tool has primarily been designed for use with cohort-type studies.
Objectives:
We report on our experience in using ROBINS-I to assess RoB in controlled before-after (CBA) studies
Methods:
We performed a systematic review of how changes to the built environment affects mental health, which included 11 CBA studies. We assessed RoB in duplicate using ROBINS-I (Table 1). Difficulties in applying ROBINS-I to CBA studies were documented.
Results:
Specifying the target trial: As changes to the built environment: a) affect groups of people, we designated the target trial use cluster randomization; and b) occur at one time point we specified an intention to treat effect. Confounding: Current questions do not address issues of confounding in CBA studies, namely: differences between the intervention and control groups in trends in outcome, or baseline mediators/moderators, before the intervention starts; nor the possibility that methods to choose intervention and control groups might lead to regression to the mean; nor the threat of co-occurring events affecting one group. We implemented a rule that studies with only one control and one intervention site and one pre- and one post- measurement were at critical risk of bias, as the effect of intervention could not be distinguished from differences between sites. Missing data: Signalling questions were appropriate for four studies where the target trial mimicked the CBA in surveying the same participants both before and after the intervention (‘cohort’ type follow-up). But seven studies recruited people in two cross-sectional samples from intervention and control areas pre and post. they do not suffer from attrition (unless a cluster drops out) and we rated them as ‘low’ RoB. Additional specific challenges encountered together with our solutions will be presented.
Conclusions:
ROBINS-I can be used for CBA studies. All seven domains are relevant, although it seems unlikely that CBA studies will be at risk of bias in some domains. ‘Translating’ the signalling questions for use with CBA studies was problematic. A version of ROBINS-I specifically for CBA studies could be helpful in reducing time and increasing reliability of using ROBINS-I.
Patient or healthcare consumer involvement:
None

Attachments: 

PDF icon Table 1.pdf

Relevance to patients and consumers: 

This work is relevant to patients and consumers in that it examines estimating bias in primary research for systematic reviews. The original idea for the systematic review, upon which this work draws, was instigated, in part by a 'Health Impact Team' (HIT) the Supporting Healthy Inclusive Neighbourhood Environments (SHINE) HIT. This group includes two authors from this work http://www.bristolhealthpartners.org.uk/health-integration-teams/supporting-healthy-inclusive-neighbourhood-environments-hit There was no consumer involvement in assessing bias for controlled before and after studies using ROBINS-I.