ROBINS-I assesses risk of bias (ROB) in non-randomised controlled studies of interventions. The tool has primarily been designed for use with cohort-type studies.
We report on our experience in using ROBINS-I to assess RoB in controlled before-after (CBA) studies
We performed a systematic review of how changes to the built environment affects mental health, which included 11 CBA studies. We assessed RoB in duplicate using ROBINS-I (Table 1). Difficulties in applying ROBINS-I to CBA studies were documented.
Specifying the target trial: As changes to the built environment: a) affect groups of people, we designated the target trial use cluster randomization; and b) occur at one time point we specified an intention to treat effect. Confounding: Current questions do not address issues of confounding in CBA studies, namely: differences between the intervention and control groups in trends in outcome, or baseline mediators/moderators, before the intervention starts; nor the possibility that methods to choose intervention and control groups might lead to regression to the mean; nor the threat of co-occurring events affecting one group. We implemented a rule that studies with only one control and one intervention site and one pre- and one post- measurement were at critical risk of bias, as the effect of intervention could not be distinguished from differences between sites. Missing data: Signalling questions were appropriate for four studies where the target trial mimicked the CBA in surveying the same participants both before and after the intervention (‘cohort’ type follow-up). But seven studies recruited people in two cross-sectional samples from intervention and control areas pre and post. they do not suffer from attrition (unless a cluster drops out) and we rated them as ‘low’ RoB. Additional specific challenges encountered together with our solutions will be presented.
ROBINS-I can be used for CBA studies. All seven domains are relevant, although it seems unlikely that CBA studies will be at risk of bias in some domains. ‘Translating’ the signalling questions for use with CBA studies was problematic. A version of ROBINS-I specifically for CBA studies could be helpful in reducing time and increasing reliability of using ROBINS-I.
Patient or healthcare consumer involvement: