Background:
Network meta-analysis (NMA) combines evidence on multiple treatments from several studies to provide internally consistent treatment effect estimates and is frequently used to inform clinical guideline recommendations. Evidence is typically assessed for risk of bias using subjective tools and checklists; however, these provide no information on the effects of potential bias on decisions based on the results of the NMA.

Objectives:
We demonstrate a new method for quantifying the effects of bias adjustment on treatment decisions based on a NMA, applied to a series of examples from published NICE guidelines.

Methods:
We propose a new method that provides quantitative assessment of the effects of potential bias adjustments, by deriving bias-adjustment thresholds that describe the smallest changes to the data that would result in a change of treatment decision. In other words, the treatment decision is unaffected by biases within the threshold limits. Bias adjustments can be considered for individual study estimates or for overall treatment contrasts. We extend our method to treatment decisions based on net benefit resulting from a probabilistic cost-effectiveness analysis.

Results:
In most cases the treatment recommendation was robust to plausible levels of bias in all but a small proportion of contrasts or studies. In larger, well-connected networks with large numbers of trials, recommendations were robust against almost any plausible bias adjustments. Sensitivity to bias adjustments for net benefit decisions resulting from cost-effectiveness analysis was also considered, showing similar results.

Conclusions:
Threshold analysis provides insight into the effects of bias adjustment on treatment decisions. Applying the method to treatment contrasts confers considerable flexibility, since practical applications are often based on complex models with multiple types of data input. We can have more confidence in treatment recommendations where bias-adjustment thresholds are large, and focus attention on the quality of decision-sensitive trials and contrasts. The need for laborious critical appraisal of all included trials may be reduced, whilst providing patients and healthcare consumers with better informed recommendations.