Software tools to support network development in network meta-analysis: a feature analysis




Poster session 2


Monday 17 September 2018 - 12:30 to 14:00

All authors in correct order:

Marshall C1, Wilson K1, McCool R1
1 York Health Economics Consortium, UK
Presenting author and contact person

Presenting author:

Chris Marshall

Contact person:

Abstract text
Background: Network meta-analysis (NMA) is a statistical technique used to compare treatments when there is a lack of direct (head-to-head) evidence or to assess multiple treatments simultaneously in a meta-analysis. The approach is rapidly being considered an important addition to a traditional systematic review of interventions. NMAs are visualised as a complex network of interconnected treatments, treatment arms and the studies contributing to each comparison. However, building networks can be time-consuming and labour-intensive, particularly when a large number of treatments have to be considered. A number of software tools have been developed, which facilitate network building and can help automate the process. However, little is known about their usefulness.

Objectives: To compare and evaluate a selection of specialised tools that support developing networks as part of a NMA.

Methods: We conducted a multi-criteria decision analysis, taking the form of a feature analysis, to evaluate the tools. We developed an initial evaluation framework comprising a set of required features, weightings and scoring instruments to assess the tools. We mainly assessed tools based on their level of automation, ease of adding and removing studies/interventions to and from the network, ability to label nodes and edges, and overall usability. We also took into consideration factors concerning the ease of installation and setup of the tools.

Results: We compared and evaluated four tools using the framework: namely, NodeXL, yED, Gephi and R (the 'netmeta' package). Each of the candidates presented some strengths and some weaknesses. NodeXL had the highest overall score and Gephi had the lowest overall score. NodeXL scored well on automated network development features and less well on ease of introduction and setup. The majority of features in Gephi were considered only partially supported leading to a lower score.

Conclusions: The results of this study provide new insight into specialised tools that support network development as part of a systematic review and NMA. The findings have informed a refined version of the feature analysis framework, and an expansion of the Systematic Review Toolbox (a Cochrane recommended catalogue of tools that provide support for systematic reviews) to classify network development tools.

Relevance to patients and consumers: 

Systematic reviews help shape key guidance and policy across available patient healthcare. The addition of a network meta-analysis is rapidly being considered an important extension to a traditional systematic review. However, systematic reviews are a time consuming endeavor, especially those which include an NMA. In particular, visualising the results as a network of interconnected treatments, arms and studies can be quite challenging. Specialised automated tools that can be used to develop complex networks might be beneficial to reviewers and help improve the efficiency of the overall systematic review and NMA process.