Background: Network meta-analysis (NMA) is a statistical technique used to compare treatments when there is a lack of direct (head-to-head) evidence or to assess multiple treatments simultaneously in a meta-analysis. The approach is rapidly being considered an important addition to a traditional systematic review of interventions. NMAs are visualised as a complex network of interconnected treatments, treatment arms and the studies contributing to each comparison. However, building networks can be time-consuming and labour-intensive, particularly when a large number of treatments have to be considered. A number of software tools have been developed, which facilitate network building and can help automate the process. However, little is known about their usefulness.

Objectives: To compare and evaluate a selection of specialised tools that support developing networks as part of a NMA.

Methods: We conducted a multi-criteria decision analysis, taking the form of a feature analysis, to evaluate the tools. We developed an initial evaluation framework comprising a set of required features, weightings and scoring instruments to assess the tools. We mainly assessed tools based on their level of automation, ease of adding and removing studies/interventions to and from the network, ability to label nodes and edges, and overall usability. We also took into consideration factors concerning the ease of installation and setup of the tools.

Results: We compared and evaluated four tools using the framework: namely, NodeXL, yED, Gephi and R (the 'netmeta' package). Each of the candidates presented some strengths and some weaknesses. NodeXL had the highest overall score and Gephi had the lowest overall score. NodeXL scored well on automated network development features and less well on ease of introduction and setup. The majority of features in Gephi were considered only partially supported leading to a lower score.

Conclusions: The results of this study provide new insight into specialised tools that support network development as part of a systematic review and NMA. The findings have informed a refined version of the feature analysis framework, and an expansion of the Systematic Review Toolbox (a Cochrane recommended catalogue of tools that provide support for systematic reviews) to classify network development tools.