Comparison of the Academy of Nutrition and Dietetics updated Quality Criteria Checklist and Cochrane's ROB 2.0 as risk of bias tools

ID: 

188

Session: 

Poster session 1

Date: 

Sunday 16 September 2018 - 12:30 to 14:00

All authors in correct order:

Handu D1
1 Academy of Nutrition and Dietetics, USA
Presenting author and contact person

Presenting author:

Deepa Handu

Contact person:

Deepa Handu
Abstract text
Background:
An essential step in systematic reviews is the assessment of the quality of included studies. The Academy of Nutrition and Dietetics recently updated their Quality Criteria Checklist (QCC) to assess risk of bias in randomized controlled trials. During this time, Cochrane also updated their 'Risk of bias' tool and developed the ROB 2.0.

Objective:
To compare the Academy of Nutrition and Dietetics updated QCC and Cochrane's ROB 2.0 as risk of bias assessment tools for randomized controlled trials within a systematic review.

Methods:
We included 10 RCTs in this comparison study. Two review authors independently reviewed the 10 articles using the updated QCC and ROB 2.0 tool. The updated QCC contains six validity questions (with signalling questions for each main question) focusing on selection bias, detection bias, performance bias, attrition bias and reporting bias. The ROB 2.0 contains five validity questions (with signalling questions for each main question). We calculated inter-observer agreement for each risk of bias domain for each tool. First, authors mapped the individual questions for both the tools to Cochrane's risk of bias domains (selection, attrition, detection, performance, other bias). Next, we analyzed comparisons between the ratings for the two tools for each domain.

Results:
Mapping of each question for the two tools indicated that both the updated QCC and ROB 2.0 questions were very well distributed across all the risk of bias domains. Results indicated that inter-observer agreement was higher for QCC compared to ROB 2.0 for all risk of bias domains. Performance bias and detection bias domains had the lowest agreement (performance bias: 60% for QCC, 40% for ROB 2.0; detection bias: 50% for QCC, 40% for ROB 2.0). We are still in the process of calculating overall inter-observer reliability status for both the tools and investigating which signalling questions contribute to more discrepancy/disagreement between observers.

Conclusion:
The updated QCC has higher inter-observer agreement compared to ROB 2.0. However, both tools are easy to use and time-efficient.

Patient or healthcare consumer involvement:
Not involved in the first phase of comparing tools; however, a patient advocate is involved in each guideline project.

Relevance to patients and consumers: 

This proposed abstract indirectly will have an impact on patient-focused health outcomes. As systematic reviews are a backbone of good clinical practice guidelines, having valid tools to assess risk of bias in studies included in systematic reviews is important. Assessing the relevance of populations, interventions, and outcomes measured, as well as fidelity of implementation of interventions helps make sure that patient interest is taken into consideration. The new tool proposed in this abstract takes into consideration patients/healthcare consumer interest.