When methodologies conflict: developing tools to compare and select appropriate methodologies for conducting, and appraising evidence from systematic reviews

ID: 

119

Session: 

Poster session 1

Date: 

Sunday 16 September 2018 - 12:30 to 14:00

All authors in correct order:

Kuusi T1, Knittle K1
1 University of Helsinki, Finland
Presenting author and contact person

Presenting author:

Touko Kuusi

Contact person:

Touko Kuusi
Abstract text
Background:
Multiple guidelines, recommendations, and meta-scientific evidence apply at various phases of systematic reviews and meta-analyses: from planning and registering, to analysing and reporting data, and even to appraising evidence. These frameworks are not unanimous in their recommendations or approaches, resulting in a confusing situation for researchers and practitioners who rely on systematic reviews as a gold standard of evidence. One topical example of this is the discussion around the effectiveness of antidepressant drugs for acute depression. Formal tools are therefore needed to help interested parties contrast and select the ideal methodological approaches to conducting and interpreting systematic reviews.

Objectives:
The first aim of this study is to develop a graphical guide for systematic review authors to choose which guidelines and approaches to use from a plethora of (sometimes disagreeing) options. The second aim is to develop a tool for contrasting methodological arguments behind disagreeing interpretations of the quality of evidence in systematic reviews and meta-analyses, in order to inform conclusions about effectiveness and subsequent decisions about implementation.

Methods:
A literature search identified guidelines, critical appraisal checklists, bias correcting methods, and other scientific literature related to methodology in systematic reviews and meta-analyses. We develop methods to display this information logically to potential end users, including review authors, students, guideline makers and a wider audience.

Results:
We present a graphical guide of available review methodologies and guidelines, and a critical appraisal contrasting tool, which consists of the best arguments and counterarguments arising from each approach to evidence appraisal.

Contributions:
The tools in this study can help those conducting systematic reviews to select the appropriate methodological guidelines, and those interpreting the results of systematic reviews to understand better whether the available evidence supports implementation of an intervention, or not.

Patient or healthcare consumer involvement:
We asked for feedback from members of the general audience to maximize understandability.

Relevance to patients and consumers: 

Many healthcare consumers have difficulty understanding how experts can come to different conclusions from the exact same systematic review evidence, depending on the methodological approach used. Take an example the discussion around the effectiveness of antidepressant drugs for acute depression. This work hopes to facilitate better understanding and communication about these conflicts by developing tools to compare and select appropriate methodologies for conducting, and appraising evidence from systematic reviews.