Background: In a quantitative synthesis of studies via meta-analysis, it is likely that some studies provide a markedly different intervention effect or have a large impact on effect estimates and/or heterogeneity. There are several methods to identify outlying studies in a meta-analysis. Most of them rely on backward methods that evaluate the impact that a study has on the meta-analysis results once it has been removed. It has been proven that such methods have a high likelihood of false positives and false negative results. One robust outlying diagnostic method developed initially for regression analysis that has been applied to meta-analysis is the forward search (FS) algorithm. Little work has been done in the field of network meta-analysis (NMA), which allows for multiple treatment comparisons and a joint synthesis of direct and indirect evidence. In the NMA setting, extreme study effects may be contribute to statistical inconsistency; that is the disagreement between direct and indirect evidence.

Objectives: To extend the FS algorithm to NMA and create a relevant R library.

Methods: FS algorithm starts by fitting the NMA model to a small subset of studies and then, gradually adding studies according to their closeness to the hypothesized model until all studies are included. Outlying and influential cases are detected through a series of forward plots of statistics related to model parameters and fit. We extended each stage of the FS algorithm to NMA and we developed a routine in R. We applied the suggested outlier detection method to real and artificial, contaminated by outliers, networks to explore their performance.

Results: Application in real networks indicated that studies responsible for heterogeneity and inconsistency influenced the results.

Conclusions: The FS algorithm is very useful in detecting outlying studies as well as studies responsible for a large heterogeneity and inconsistency. FS algorithm is a promising diagnostic tool for dealing with heterogeneity and inconsistency. Outliers can be excluded in a sensitivity analysis to explore robustness of results.

Patient or healthcare consumer involvement: None.