Introduction

New ways are needed to generate evidence in addition to conducting randomised controlled trials (RCTs).1^, 2 This is particularly so as medicine struggles to understand how to apply an evidence base developed for single conditions to an ageing population with multimorbidity.3 Studies using Real-World Evidence (RWE) are a potential next step (Box 1).

Patients, providers and payers all need evidence beyond that generated in clinical trials:

• Patients want to know to what extent health interventions work in the real world for patients like them;

• Providers of health care want to make more informed recommendations for patients;

• Commissioners and payers need to know how interventions work in their populations in the real world;

• Society needs to be informed of any long-term unexpected complications of therapies.

RWE is enabled by the ubiquity of health care information technology systems and the resultant increase in health care data that is born digital, as well as by advances in analytics. The move towards greater use of RWE is also driven by the general shift in health care from a transactional, fee-forservice model to a more value-based system with payments based on outcomes.4

In summary, RWE is a form of evidence generated by applying a research question or testing a hypothesis using real-world data (RWD), and applying a method that generally includes advanced analytics to generate evidence (Figure 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1. RWE is generated by applying a question or testing a hypothesis using RWD and applying analytical techniques set out in the study design

Real-World Data (RWD)

RWD is either primary – collected specifically for the RWE study – or secondary use of data collected primarily for another purpose (Box 2). All these data have strengths and limitations5 that need to be set out in full.

Computerised medical record (CMR) systems have been a key enabler of RWE investigations, particularly in health systems where individuals have a unique ID and can be followed over time. 5^,6 However, access to RWD is not always straightforward – and laws and policies to protect privacy can be a greater limitation on the ability to carry out RWE research than conducting analyses.

Real-world analytics

Advanced analytic methodologies are used to mine RWD and provide evidence for populations and subgroups. Whereas trials and conventional evidence inform how populations might respond to treatment, RWE looks to provide evidence for subgroups and eventually for individuals.

Limitations of RWE

The primary limitation of RWE is that as it is a new and emerging approach, there is no agreed upon guidance for RWE methodologies and evaluation. There is uncertainty about sample size, what represents a valid and reliable method, how to compare care pathways, especially in people with complex health needs, the best method for economic evaluation, and how to reduce bias. A recent debate around possible harms of screening for lung cancer reflects this uncertainty.7

Guidance from the Enhancing the Quality and Transparency of health Research network (http://www.equator-network.org/), notably Consolidated Standards of Reporting Trials, informs whether an RCT is of quality. However, despite all attempts to reduce the risk of bias, there are concerns that funding source may bias trial results,8^,9 though there is not a consensus about this.10^,11 The International Society for Pharmacoeconomics and Outcomes Research has to some extent sought to address this gap,12 though this does not address the study of RWE per se.

The RWD used in RWE studies are messy and often incomplete; sophisticated statistical methods must be used to rigorously gain accurate RWE insights. Further, apparently easy to define conditions can actually be more complex – diabetes provides a good example of this.13 Pitfalls in case selection are all too easy and can only be detected if coding lists used in analysis are published.14 Case finding if it is to be reliable between different brands of computer system – let alone international – needs to be much more ontological15^,16 (i.e. transparent models that link clinical concept to coding lists used to identify cases, outcomes, confounders, etc.).17

RWE is complementary to RCT evidence

Whilst the RCT remains the gold standard for testing hypotheses about treatment, other forms of evidence are needed, as RCTs fail to answer questions of benefit for those excluded from trials. RCTs will always be needed and form an important part of the licencing of medications; it is important that they are demonstrated to be effective in the core target group who will receive them. However, a number of factors suggest that whilst RCTs will always be required, a first response of ‘Let’s do a trial’ to test a hypothesis, particularly about the effectiveness of treatments in clinical practice, should be accompanied by a careful evaluation as to whether an RCT is the best option. Downsides of RCTs (Figure 2) include high costs, the time taken from hypothesis to creating evidence and uncertainty about the relevance of trial results to an ageing population with multimorbidity. RCTs in the therapeutic world are short – mainly because that is all that regulators require. As a result, rare complications of therapies may not be picked up in a trial that lasts 6–12 months. This is a very important function of RWE or post-marketing studies, i.e. the detection of unusual but significant drug-related issues –either on target or off target. It is it not just the size of the numerator that is important but the possibility of tracking large numbers of patients over long periods that provides such useful information. Whilst RWE will most often not prove causality, it does enable a hypothesis to be raised and explored further. The potential advantages of using RWE are that costs are potentially much lower and retrospective RWE investigations can take place much faster, and the same may be true of some prospective investigations. Whilst RWD are inherently more messy, their advantage is that RWE studies will include people with multimorbidity, on usual prescribed doses, and standard patterns of adherence. Also, adverse effects of medication may not always be detected in the initial trials.18^,19

• Download figure
• Open in new tab
• Download powerpoint

Figure 2. The case for increasing the use of RWE

Call for the establishment of RWE centres

Notwithstanding these limitations, studies using RWE are probably here to stay; the urgent need is for methods to assess the quality of these studies. Unless we develop robust ways of working, RWE might be dismissed as just another attempt to create new ways to conduct comparative effectiveness research.20 Research funders should be sponsoring the development of RWE centres to develop methodologies and conduct RWE studies. Prospective RWE studies could focus on stratified medicine.21 These studies should look to understand which groups of patients therapeutic interventions fit with trial evidence and for which groups they are more or less effective in real-world use. Retrospective studies can rapidly answer questions about adherence, thresholds at which treatments are implemented, unanticipated safety issues, and other important questions. They should inform health services about the cost-effectiveness of different care pathway options; currently, they are often dealt with piecemeal.

Summary

RWE studies are needed to properly assess benefit/risk of treatments when used in standard clinical practice. CMR can facilitate concurrent safety monitoring and data collection without direct patient contact and enable large randomised study populations to be identified for pragmatic RCTs. Informaticians or investigators with a detailed understanding of the strengths and weaknesses of routine data – how to infer meaning and to avoid pitfalls – should lead such study teams. Ensuring privacy and ethical use of patient data is also essential for the success of these projects. Analysis of health data needs to take place within a secure environment. There needs to be a high level of trust alongside professionalism in protecting privacy.22^,23

New et al. sum up the current situation and the need for RWE: RCTs are the gold standard but additional information is needed about the risk/benefit profile of new treatments in real-world practice, and especially evidence providing relative ‘values’ on novel medications, in cash-limited health services.24 There are enormous opportunities for the informatics community if these predictions about the future research direction are correct. The future research landscape will be one in which RCTs are required for regulation and for approval of therapies, devices, and other interventions. RCTs are needed primarily to demonstrate comparative efficacy and safety in a controlled environment. RWE studies will become the norm for demonstrating effectiveness and cost-effectiveness, particularly in the context of an ageing population with multimorbidity.

Conflicts of interest

Simon de Lusignan is the lead academic and Neil Munro is a professor supporting the RWE centre based at University of Surrey funded by Eli Lilly and Company (http://www.surrey.ac.uk/fhms/research/researchgroups/real_world_evidence_centre_fhms.htm).