A comparison of methods for meta-analysis of a small number of studies with binary outcomes

ID: 

341

Session: 

Poster session 3

Date: 

Tuesday 18 September 2018 - 12:30 to 14:00

All authors in correct order:

Mathes T1, Kuss O2
1 Institute for Research in Operative Medicine, Witten/Herdecke University, Germany
2 Institute for Biometrics and Epidemiology, German Diabetes Center, Leibniz Institute for Diabetes Research at Heinrich Heine University, Düsseldorf, Germany
Presenting author and contact person

Presenting author:

Tim Mathes

Contact person:

Tim Mathes
Abstract text
Background: Meta-analyses often only include a few studies. Estimating between-study heterogeneity is difficult in these cases. An inaccurate estimation of heterogeneity can result in biased effect estimates and confidence intervals (CIs) that are too narrow. Research has shown that this is particularly true when using the DerSimonian and Laird (DLRE) heterogeneity variance estimator.

Methods: To compare different methods for meta-analysis of a small number of studies with binary outcomes.

Results: We compared the DLRE method with other meta-analytic methods, including the modified Hartung-Knapp (mHK) method, the Paule-Mandel (PM) method and the beta-binominal (BB) model considering odds ratios. For the comparison of the methods for meta-analysis of few studies (≤ 5), we performed a simulation study that used true parameters from meta-analyses that had actually been performed in Cochrane Reviews to mirror meta-analyses observed in practice. For each scenario we simulated 10,000 meta-analyses. We performed various sensitivity analyses to assess the robustness of the results (effect sizes, baseline probabilities, between-study heterogeneity, 10 to 50 studies in meta-analysis). We estimated median bias, 95% empirical coverage, power and robustness to assess the performance of the methods.

Conclusions: All methods outperformed the standard DLRE method. The BB model performed best, considering the balance between correct empirical coverage and power for meta-analysis of few studies. If one is willing to accept a slightly higher type-I error rate than the nominal level for a higher power, the PM methods is an alternative.

Patient or healthcare consumer involvement: Not applicable.

Relevance to patients and consumers: 

We evaluated which statistical methods for pooling results from different studies (meta-analysis) give valid results. Different meta-analysis methods can yield different results and thus result in different conclusions on the effectiveness of an intervention. This means that using the “wrong” method can lead to “wrong” conclusions. Therefore, choosing the “right” meta-analytic method is important for “right” decision making. Our work can help finding the "right" method and thus improve policy and medical decision making. No patient/consumer partners helped me reach this statement.