Supporting systematic review authors to report and explore the applicability and transferability of their findings

Session: 

Oral session: Understanding and using evidence (2)

Date: 

Monday 17 September 2018 - 14:50 to 15:10

Location: 

Carrick 3

All authors in correct order:

Kneale D1, Rojas-García A2, Thomas J1
1 EPPI-Centre, UCL Institute of Education, UCL, UK
2 Dept of Applied Health Research, UCL, UK
Presenting author and contact person

Presenting author:

Dylan Kneale

Contact person:

Dylan Kneale
Abstract text
Background:
Effective evidence use in public health decision-making is impeded by evidence that does not meet decision-maker and public health practitioners' needs. This includes issues in the generalisability of evidence both in terms of the feasibility of running interventions (applicability) and whether the same direction and magnitude of effect sizes would be observed (transferability).

Objectives:
This presentation will report on work undertaken to:

1) design more effective ways in which systematic review authors can consider the generalisability of their findings; and
2) explore the transferability of the overall effect, using an example from a review of asthma self-management in schools.

Methods:
The first objective is addressed through developing structured summaries of generalisability within review software (EPPI-Reviewer) based on recommended reporting practices and existing generalisability tools. The second is addressed through developing and testing new methods of using contextual data to assess the transferability of meta-analytic evidence. In this presentation, using an example from a review of school-based asthma interventions, we undertake a form of enhanced subgroup analysis to identify clusters of dis/similar studies.

Results:
We outline some of the ways in which systematic review authors can better report the generalisability of studies and evidence through EPPI-Reviewer. Preliminary work on enhancing the transferability of meta-analysis involved the re-analysis of findings from a systematic review of school-based asthma interventions, and exploring the differential impact if the intervention was run in London schools. Analysis of school absences suggests that sites more similar to London schools in terms of the ethnic profile, baseline level of school absence and gender split of children recorded less improvement in levels of absence than those that were dissimilar. We will discuss the potential reasons for this discrepancy and how this information can be valuable to decision-makers. In addition, we will explore potential drawbacks of the method, as well as future avenues for research.

Patient or healthcare consumer involvement:
EPPI-Reviewer users will be involved in creating the new capacity. In addition, the steps undertaken are based on previous work with public health practitioners.

Relevance to patients and consumers: 

Systematic reviews are considered one of the most robust sources of evidence, and yet research suggests that their use is relatively infrequent in public health decision-making. One primary reason is that systematic reviews represent a global summary of evidence, although public health decision-makers and practitioners need to know if evidence is relevant to their local area. This is in terms of whether an intervention is feasible, and whether the same effect is likely to be observed. Improving the uptake of review evidence in decision-making could also improve public health outcomes.