On the aggregation of historical prognostic scores for causal inference

Session: 

Oral session: Diagnostic test accuracy review and prognostic methods (2)

Date: 

Tuesday 18 September 2018 - 14:40 to 14:50

Location: 

All authors in correct order:

Nguyen T1, Collins G2, Loymans R3, Moons K4, Debray T4
1 University of Montpellier, France
2 Oxford University, UK
3 Academic Medical Center Amsterdam, The Netherlands
4 Julius Center for Health Sciences and Primary Care, The Netherlands
Presenting author and contact person

Presenting author:

Thomas Debray

Contact person:

Abstract text
Background:
Comparative effectiveness research in non-randomized studies is often prone to various sources of confounding. Recently, prognostic score analysis has been proposed to address this issue, which aims to achieve prognostic balance across the different treatment groups. Although it is common to use the non-randomized data at hand to develop the necessary prognostic scores, this strategy is problematic when sample sizes are relatively small. It has previously been demonstrated that prognostic scores from historical cohorts may actually outperform internally developed prognostic scores for causal inference, and that their accuracy can further be improved through evidence synthesis.

Objectives:
To present new meta-analysis methods for causal inference in non-randomized data sources. Hereto, we consider the aggregation of multiple prognostic scores derived from historical cohorts.

Methods:
We extend existing methodology for causal inference and meta-analysis of prediction models, and propose new methods to derive comparative treatment effects from non-randomized studies. We conducted an extensive simulation study based on a real clinical dataset comparing different treatment strategies for asthma control. We aggregated previously identified prognostic scores for predicting exacerbations of asthma, and used the resulting model to estimate the average treatment effect in the overall (ATE) and in the treated (ATT) population of various simulated datasets. We compared various implementation strategies by assessing the bias and mean squared error of the estimated ATE and ATT, and the ratio of the estimated standard errors to the empirical standard deviations.

Conclusions:
Initial simulation study results suggest that aggregation of historical prognostic scores may substantially improve the estimation of comparative treatment effects in non-randomized data sources.

Patient or healthcare consumer involvement:
A clinician was involved in the provision of relevant patient-level data, and the interpretation of comparative treatment effect estimates.

Relevance to patients and consumers: 

We propose new statistical methods to improve comparative effectiveness research and the identification of clinically relevant subgroups. This will help to better identify whom to treat, when to treat and how to treat. Ultimately, this will contribute towards more patient-focused healthcare. Our research was supported by statisticians, as well as epidemiologists and clinical experts.