Background: The US Veterans Health Administration (VHA) promotes innovation to provide veterans with the best healthcare. Thus, policy-makers are increasingly looking to evidence synthesis programs for decision support on innovative treatments with early evidence bases. Decision-makers make the point that while they need independent, unbiased information, traditional evidence syntheses take too long and address whether evidence is definitive, rather than whether an innovation is promising. In response, systematic reviews need to assess whether an early evidence base is exceptional in ways that may predict success, a concept we called ‘rigorous speculation’.
Objective: To conceptualize a framework for ‘rigorous speculation’ in evidence synthesis, an approach to improving health system decision support about innovative treatments with early evidence-bases.
Methods: In a June 2017 VA Evidence-based Synthesis Program (ESP) strategic planning meeting, discussions with health system decision makers led to identification of the ‘rigorous speculation’ concept. Based on the ESP Coordinating Center’s (ESP CC) experience piloting ‘rigorous speculation’ in rapid evidence reviews, we provide operational guidance on how to apply this approach.
Results: We identified the following processes for improving evaluation of early evidence-bases: 1) use of a new language that moves away from boilerplate language ('low strength evidence', etc); 2) improved description of case series’ ideal characteristics (e.g. unbiased patient selection, stability of treatment and patient characteristics pre-intervention, well-defined intervention, objective or blinded assessment); 3) identification of relevant historical evidence patterns for use in predicting success; 4) increasing linkages to relevant existing dataset resources; and 5) increased consideration of local context (e.g. access, feasibility, cost, patient preferences, etc.).
Conclusion: We propose this framework to increase consistency and transparency of evidence reviewers’ use of ‘rigorous speculation’, thus expanding their role as knowledge brokers for health system decision making. We will refine this framework based on further expert consultation and literature review.
Patient or healthcare consumer involvement: This framework considers patient/healthcare consumer preferences as a key local contextual factor.