Advantages and challenges of including randomised and non-randomised evidence in a systematic review: example of advanced or metastatic renal cell carcinoma

ID: 

149

Session: 

Poster session 1

Date: 

Sunday 16 September 2018 - 12:30 to 14:00

All authors in correct order:

Edwards S1, Karner C1, Kew K1, Wakefield V1
1 BMJ, UK
Presenting author and contact person

Presenting author:

Charlotta Karner

Contact person:

Charlotta Karner
Abstract text
Background: Several therapies have recently become available for people with previously treated advanced or metastatic renal cell carcinoma (amRCC), but comparative efficacy cannot be assessed for all treatments via network meta-analysis (NMA) of only randomised controlled trials (RCTs). Incorporating non-randomised studies (NRS) could allow all treatments to be included in a single network, increase the statistical precision of effect estimates and enhance the applicability of the results.
Objectives: Use the Cochrane 'Risk of bias' tool and ROBINS-I to assess the quality of RCTs and NRS, respectively, and compare clinical effectiveness results of NMA with and without NRS of treatments for pretreated amRCCs.
Methods: We searched MEDLINE, Embase and the Cochrane Library up to January 2018. Two reviewers abstracted data and critically appraised studies by outcome. We assessed RCTs and NRS with the Cochrane 'Risk of bias' tool (seven domains) and ROBINS-I (seven domains), respectively. Fixed-effect NMAs were conducted for overall survival (OS) and progression-free survival (PFS) estimated as hazard ratios (HRs). Data of low risk of bias for all domains formed the primary analyses, with studies at high or serious risk of bias for at least one domain included in sensitivity analyses. NRS at critical risk of bias were excluded.
Results: Four RCTs and eight non-RCTs were included in the review. Primary analyses of OS and PFS were based on RCT data. Sensitivity analyses incorporating NRSs allowed two additional treatments to be included in the network. ROBINS-I required more time to conduct and reach consensus between reviewers, initial agreement was low compared with the Cochrane 'Risk of bias' tool, and the overall judgement concealed appreciable differences between outcomes and studies. Results of the sensitivity analyses were consistent with the primary RCT analyses.
Conclusion: Review authors should consider the resources and skills required to incorporate NRSs, and the potential benefit of doing so. Study protocols should define and justify the domains on which RCTs and NRS will be included in primary and sensitivity analyses.
Patient involvement: This research was initiated by the National Institute for Health and Care Excellence (NICE), which included patient representation from national patient groups in the preparation of the scope of the project.

Relevance to patients and consumers: 

Patients cannot make an informed decision about their treatment options if the benefits and potential harms of ALL relevant treatments have not been compared. Including non-randomised studies (NRS) in systematic reviews of randomised controlled trials (RCTs) could allow more treatments to be compared and thereby make the results of the review more useful.