Systematic reviews can be a reliable source for information-based decision-making. They should retrieve all relevant primary studies addressing a research question, use robust methods for appraising the quality of included studies, and apply appropriate and transparent methods to synthesise these. The Risk of Bias in Systematic Reviews (ROBIS) tool was designed to identify the risk of bias in the review process (sometimes called 'metabias') in order to guide users, guideline developers and other stakeholders in assessing the reliability of systematic review conclusions. This tool has previously only been available in English; there is a growing need to assess the methods of systematic reviews in Spanish.
To assess the validity and usability of the Spanish version of the ROBIS tool.
We performed two independent translations of the ROBIS tool into Spanish. We established a group of experts in evidence-based medicine (EBM) who reviewed the two versions and drafted a preliminary version assessing face validity. We then pilot tested the tool with six systematic reviews using both the original tool and the translated tool, in order to identify problems in its effective implementation, which we corrected.
We plan to apply the original ROBIS tool using a group of methodologists and trainees in EBM who will independently assess systematic reviews in pairs using both the original tool and the translated tool during EBM meetings, including the Cochrane Iberoamerican Meeting in Cuba (May 2018). We plan to evaluate the agreement between the assessments of each domain comparing the original tool and the translated tool. Additionally, we will record the time taken to apply it, and we will collect feedback from the participants regarding its usability.
Results and conclusions:
We have developed a Spanish version of the ROBIS tool that has received positive feedback during our initial pilot testing. We expect to refine this tool throughout this year and we will present the results of this experience at the Colloquium. We believe that this refined version will help the formal assessment of metabias in systematic reviews in Spanish and the development of overviews.
Patient or healthcare consumer involvement:
Our team is formed of methodologists and EBM experts of which five, including the main author, are practising clinicians.