Generalizability of evidence from randomized controlled trials published in the leading medical journals

ID: 

377

Session: 

Poster session 3

Date: 

Tuesday 18 September 2018 - 12:30 to 14:00

All authors in correct order:

Malmivaara A1
1 National Institute for Health and Welfare, Finland
Presenting author and contact person

Presenting author:

Antti Malmivaara

Contact person:

Antti Malmivaara
Abstract text
Background:
Generalizability of evidence from randomized controlled trials (RCTs) is critical for clinical decisions and for the appraisal of current evidence in systematic reviews. The international classification of functioning (ICF) framework proposes a bio-psycho-social context when describing patients' conditions.

Objectives:
To assess the generalizability of evidence from RCTs published in the four leading medical journals, considering the comprehensiveness of patient description.

Methods:
All RCTs in the BMJ, JAMA, the Lancet and the New England Journal of Medicine (NEJM) from 1 January to 30 September 2017 were searched by hand and data were extracted. The papers were assessed using the CONSORT statement criteria and the criteria for observational effectiveness studies, the 'benchmarking controlled trials' (BCTs) framework.

Results:
Of the 161 RCTs, four were published in the BMJ, 50 in JAMA, 71 in the Lancet and 36 in the NEJM. Seventy-two RCTs were conducted in the US or Europe and 70 were multinational. Sixty-two per cent studied pharmacological interventions, 21% other conservative interventions, 7% surgical interventions, 3% rehabilitation interventions and the rest combinations of these. The patient's path before randomization was described in 13.7% of papers, reasons for exclusions were reported in 62.1% and healthcare settings were described in 7.5% of trials. Demographic and disorder-specific data were provided in 97.5% of papers, but any measure of functioning (disorder-specific or generic disability, or health-related quality of life) in 48.4%, at least two co-morbidities in 34.8%, any behavioral factor (alcohol consumption, smoking, physical activity or obesity) in 50.9%, any environmental factor (work and living conditions, marital status) in 10.6%, and any factor related to potential inequity (education, socioeconomic status, deprivation, ethnicity) in 50.3% of papers. There were major differences between journals in the completeness of reporting.

Conclusions:
Most of the RCTs were conducted in industrialized countries and on pharmacological or conservative therapy. Deficiencies in the description of patient selection, study setting and patient characteristics are common. These deficiencies hamper the generalizability of evidence for clinical decision-making and for systematic reviews. The use of the CONSORT statement and the BCT framework may widen the description of patients in RCTs and increase our ability to generalize evidence.

Relevance to patients and consumers: 

Effectiveness estimates may be associated with several patient characteristics, not just with the diagnosis and severity of indication. Patients’ disabilities, comorbidities, behavioral, environmental, and inequity related factors may modify the treatment response, and the evidence may not be generalizable to all patient groups with different backgrounds. This presentation will bring forward the need to address patients more comprehensively in randomised controlled trials.