An online implementation of the ROBINS-I (risk of bias in non-randomized studies – of interventions) tool

Session: 

Oral session: Investigating bias (3)

Date: 

Tuesday 18 September 2018 - 11:00 to 11:20

Location: 

Carrick 1

All authors in correct order:

McGuinness LA1, Hutton D1, Sterne JAC1, Reeves BC2, Savovic J1, Higgins JPT1
1 Bristol Medical School, University of Bristol, UK
2 Bristol Medical School, University of Bristol, United Kingdom
Presenting author and contact person

Presenting author:

Luke McGuinness

Contact person:

Luke McGuinness
Abstract text
Background: Non-randomized studies of interventions (NRSI) can provide useful data on the effects of healthcare interventions but are likely to be subject to confounding and other biases, some of which do not apply to trials. The ROBINS-I (risk of bias in non-randomized studies – of interventions) tool provides a comprehensive assessment of risk of bias in NRSI, facilitating their inclusion in robust systematic reviews.
Objective: To describe the features of an online implementation of the ROBINS-I tool.
Methods: The online implementation contains all the elements of ROBINS-I (target trial specification, preliminary consideration of confounders/co-interventions, signalling questions informing 'Risk of bias' judgements within seven bias domains, derivation of an overall 'Risk of bias' judgement) and incorporates significant additional functionality. The implementation was built using ASP.net and Microsoft SQL Server.
Key features: Information necessary to perform the assessment, including recent updates to ROBINS-I signalling questions, question-specific guidance and relevant study documents (publications/protocols/etc.), is presented in a user-friendly layout. Additional features include:
- different levels of user access depending on prespecified roles (e.g. only review owners can create, edit and assign assessments to other users);
- signalling questions asked only when relevant (conditional on previous answers);
- algorithms for domain-level risk of bias judgements based on answers to signalling questions, with the potential for manual override;
- comparison function, allowing rapid reconciliation of assessments made by different raters;
- online help.
Conclusions: Online implementation of ROBINS-I will aid both novice and experienced reviewers by making the tool easier and quicker to use and by increasing the accuracy of their assessments. This work will pave the way for similar implementations of other tools, including the Risk of Bias 2.0 tool for trials and the forthcoming ROBINS-E (risk of bias in non-randomized studies – of exposures).
Patient/consumer involvement: Risk of bias assessment requires advanced epidemiological knowledge, so involving patients in its development is challenging. However, we involved the target ‘consumers’ for this online implementation, review authors, during development to ensure the tool was fit for use.

Relevance to patients and consumers: 

Online implementation of the ROBINS-I (Risk Of Bias In Non-randomized Studies – of Interventions) assessment tool will promote the production of robust systematic reviews that incorporate evidence from non-randomized studies of interventions (NRSI). Reviews often need to include NRSI to maximise their relevance to patients and the public, for example by incorporating information about rare outcomes or serious harms that are key to informed patient choice. The tool also assesses bias domains separately, providing patients and consumers with a transparent description of the reasons why specific studies should or should not be trusted.