Background: Studies included in systematic reviews are not always identical. This is often due to clinical or methodological differences that may cause variability or heterogeneity in the treatment effect. Inconsistency in treatment effect undermines the generalisability of the findings and assumptions about consistency play an important role in approaches to analysis, interpretation and assessing the quality of the evidence. There is limited guidance on how to determine potential sources of heterogeneity.
Objectives: The objective of this study was to document how often certain study characteristics are a source of statistical heterogeneity in the HIV literature.
Methods: We conducted a methodological review of all HIV systematic reviews in the Cochrane Database of Systematic Reviews. We included systematic reviews of interventions to improve HIV-related outcomes from the Cochrane Database of Systematic Reviews (Issue 12, December 2017). We extracted data in duplicate on the number of included studies, pre-specified sources of heterogeneity, whether a meta-analyses was conducted, sources of heterogeneity investigated and the presence of a subgroup effect (defined as a significant test for subgroup effect or non-overlapping confidence intervals).
Results: We retrieved a total of 116 records published between 2000 and 2017. These systematic reviews included a median of six studies (min 0; max 44) and 67 (57.8%) included a meta-analysis. Among the planned sources of heterogeneity, age (24; 20.7%), gender (17; 14.7%), CD4 count (14; 12.1%), disease severity (12; 10.3%) and dose (10; 8.6%) were the most frequent. The most frequently investigated sources of heterogeneity were gender (5; 4.3%), disease severity (4; 3.4%), dose (3; 2.6%) and CD4 count (3; 2.6%). Only three reviews among those with meta-analyses (4.5%) detected a genuine subgroup effect for duration of follow-up, maternal gravidity and dose.
Conclusions: Planned subgroup analyses are often not conducted and when they are they do not yield genuine subgroup effects in the HIV literature. More efficient approaches to investigating subgroup effects are warranted.
Patient or healthcare consumer involvement: Investigations into how interventions affect subpopulations do not often produce actionable results in the HIV literature.