Review••

Scoping review exploring the impact of digital systems on processes and outcomes in the care management of acute kidney injury and progress towards establishing learning healthcare systems

Abstract

Objectives Digital systems have long been used to improve the quality and safety of care when managing acute kidney injury (AKI). The availability of digitised clinical data can also turn organisations and their networks into learning healthcare systems (LHSs) if used across all levels of health and care. This review explores the impact of digital systems i.e. on patients with AKI care, to gauge progress towards establishing LHSs and to identify existing gaps in the research.

Methods Embase, PubMed, MEDLINE, Cochrane, Scopus and Web of Science databases were searched. Studies of real-time or near real-time digital AKI management systems which reported process and outcome measures were included.

Results Thematic analysis of 43 studies showed that most interventions used real-time serum creatinine levels to trigger responses to enable risk prediction, early recognition of AKI or harm prevention by individual clinicians (micro level) or specialist teams (meso level). Interventions at system (macro level) were rare. There was limited evidence of change in outcomes.

Discussion While the benefits of real-time digital clinical data at micro level for AKI management have been evident for some time, their application at meso and macro levels is emergent therefore limiting progress towards establishing LHSs. Lack of progress is due to digital maturity, system design, human factors and policy levers.

Conclusion Future approaches need to harness the potential of interoperability, data analytical advances and include multiple stakeholder perspectives to develop effective digital LHSs in order to gain benefits across the system.

Introduction

The National Health Service (NHS) was in the midst of a rapid phase of digital transformation before the COVID-19 pandemic, which has patently further forced the pace of change.1 The increasing availability of digitised clinical data has the potential to turn individual organisations and their networks into learning healthcare systems (LHSs), systems that use information collected routinely as part of the care process to identify trends and variations and drive learning and quality improvement.2 When this clinical information becomes near to or real-time, it opens up the prospect not only of more detailed retrospective review of care but also the possibility of making more frequent and subtle adjustments across the system, to ensure quality is maintained as care proceeds.

The potential for real-time clinical information to enable rapid adaptive responses to improve outcomes is clearly recognised at an individual patient level. Over the last 20 years digitised Early Warning Scores have been introduced onto many hospital wards to reduce response time to deteriorating patients with mixed results.3 4 However for a LHS to be fully realised these data need to drive agile adaptation across different levels of the organisation and potentially the wider local health and social care system, facilitating changes that increase the chances of good outcomes for populations of patients while at the same time reducing risks of iatrogenic harm. Broadening ‘recognition and response’ mechanisms from those focused on rapidly identifying and managing acute changes in individuals to real-time matching of acute illness burden to staff numbers and skill set on wards or converting hospital beds to higher care levels based on changes in demand is the next step towards building a LHS.5 Limited progress in this direction has been reported, occurring mainly within individual organisations or healthcare systems rather than across the wider health and care system.6

Recent patient safety initiatives have prioritised detection and prevention of sudden deterioration, through focus on areas such as acute kidney injury (AKI) management. AKI is a common complication found among acutely ill patients and has been associated with longer hospital stays, increased morbidity and mortality.7 It can be a complication of an illness such as sepsis or a result of drugs or treatments the patient receives, especially where kidney function is already compromised by comorbid illness.8 There are no curative treatments but much can be done to limit kidney damage through institution of simple early interventions. This, in turn, avoids more complex interventions such as dialysis or renal replacement at a point where the kidneys can no longer be salvaged.

Diagnosis depends on a rising blood creatinine level or falling urine output. Laboratory values for creatinine can be easily digitised and the availability of electronic healthcare records (EHRs) have enabled the real-time/near real-time reporting of values to clinicians. The NHS has recently introduced a standardised electronic reporting system for creatinine in an effort to decrease response times to treatment.9 For EHRs that support clinical decision support systems, computer physician order entry and electronic prescribing, alerts related to rising creatinine can be notified to the patient’s clinical team via the EHR providing real-time advice on an appropriate course of action and treatment choices.10 Alternatively, such systems can send an alert to a pharmacist or renal rapid response team (RRT) to prompt action.11 12 As well as promoting earlier diagnosis, some digital systems are predictive, identifying patients at risk and allowing closer monitoring or tailoring of treatment to avoid the condition developing.13 Others play a part in harm-reduction by highlighting the potential dangers of certain drugs or doses to kidney function.

Given that digitisation of creatinine levels and real-time digital recognition and response systems for management of AKI have been available for over a decade, we used the literature to explore the extent to which such systems have impacted on patient care processes and outcomes across all levels of health and care systems (patient, organisation and population levels), to gauge progress towards the goal of establishing LHSs and to identify where current gaps in the research exist.

Methods

Scoping review

An initial scan of the literature on the use of real-time data for AKI management indicated a large variety of study approaches of varying methodology and rigour. A scoping review approach was selected to synthesise a metanarrative and identify themes based on the broad body of research in this field without exclusion based on study methods or formal assessment of study quality. A protocol based on the recommended items in the Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for scoping review was developed but not published (online supplemental material 1).14

Search strategy

Databases (Embase, PubMed, MEDLINE, Cochrane, Scopus and Web of Science) were searched for papers published from inception to 31 January 2020 using free-text keywords related to our review questions (online supplemental material 2). Additional articles were identified through citation searches of relevant articles and reviews (figure 1). As one of the main objectives was to identify gaps in research, only published research articles were included as a source of evidence.

Figure 1
Request permissionrequest permission icon
Figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analysis flowchart of literature search. AKI, acute kidney injury.

Study selection

We included original research or case reports in the English language, conducted using any study design, in any setting, for any cohort of patients. We only included publications that reported process and/or outcome measures resulting from a real-time or near real-time healthcare professional response to data in the management of AKI, for example, interventions in medicines management in response to renal insufficiency. During the search review articles and non-English studies were excluded. During abstract and full-text screening, narrative reports, articles that did not report a real-time response to data and changes in process or outcomes, audits, qualitative studies, population health studies and publications which focused on model or alert development were excluded.

Data extraction

Our review objective was addressed through the following questions that formed a basis for thematic data extraction (table 1). Each article was mapped against concepts linked to the review questions.

Table 1
|
Review questions mapped to themes used to analyse the studies

Results

We identified 2050 unique articles (figure 1). Following title and abstract screening using pre-specified criteria, 120 full-text articles were reviewed, resulting in 43 studies (online supplemental material 3) of interventions using real-time clinical information on AKI to drive service change and reported changes in either process or outcome measures (tables 2 and 3). The included studies were published between 1994 to 2020, with only seven publications before 2010.15–21 The majority of studies were from the USA and the UK, with 11 from other countries.16 21–30 Most studies were conducted in hospitals with two in primary care,31 32 and one involving community pharmacy services.24 There were eight randomised controlled trials (RCTs).13 22 31 33–37 The other studies used a range of observational designs, with the majority being uncontrolled before and after studies.

Table 2
|
Thematic analysis of studies classifying the afferent arm, efferent arm, timing, targeted group, study type and level of digital maturity
Table 3
|
Thematic analysis of studies highlighting the process measures and outcome measures used, and findings reported

Micro level

Twenty-six studies featured an intervention at the micro (individual patient) level. In 12 studies the main purpose of the intervention was harm prevention,15 19–21 23 24 28 30 35 38–41 in 12 studies it was earlier diagnosis16 22 26 27 32–34 42–46 and in 2 studies it was risk prediction.13 47 Harm prevention interventions involved alerts to clinicians of the need to change nephrotoxic drugs (non-prescription, dose altering or drug suspension) based on a patient’s renal function. The main purpose of early diagnosis interventions was to alert individual clinicians of a patient’s deteriorating renal function to trigger an early review and appropriate intervention. Risk prediction interventions used algorithms to identify high risk individuals and institute individual management plans to prevent the development of AKI.

Interventions at this level were based on real-time data apart from four studies, which used near real-time data.23 26 42 46 Three quarters of these interventions used interruptive alerts,15 23 32 40 45–47 and in a third the alert was outside the clinicians’ workflow.21 23 27 32–34 42 46 All early diagnosis alerts, apart from one (urine output27), were activated by changes in serum creatinine (SCr) levels. This was similar for harm prevention, with a minority of interventions using nephrotoxic drug exposure instead.20 41 All the risk prediction interventions used algorithms to trigger alerts.13 47

In almost half the interventions where it was specified, the alert was targeted at a physician,16 20 21 26–28 30 35 39 41–43 with a member of the multidisciplinary team (MDT) being the next most common target.13 15 19 23 33 34 40 46 47 The digital maturity of the interventions clustered at level 2 (standalone databases not fully integrated into the EHR)20 21 24 27 28 30 32–34 39 41 43 and level 4,13 16 19 22 35 38 44 45 47 two were at level 142 46 and four at level 3.15 23 26 40

Meso level

Fourteen interventions were found at meso (management) level. Two-thirds were harm prevention,11 17 18 25 29 36 48 49 the others enabled earlier diagnosis.12 50–53 Harm prevention interventions usually involved pharmacist surveillance of nephrotoxic medication across groups of patients at ward, specialty unit or hospital level. Such surveillance led to patient intervention when kidney function was deteriorating and was often accompanied by feedback and education for clinical teams. Meso-level interventions aimed at early diagnosis were generally part of an approach to reducing the incidence and severity of AKI across a number of wards or the whole organisation. These interventions used the digital data in a variety of ways including to alert hospital-wide renal RRTs, to review patient management plans within ward-based safety huddles or to audit the timely implementation of AKI bundles (elements of protocolised AKI management plans). All but one of the interventions at meso level used changes in levels of SCr to trigger an alert,11 with two-thirds based on near real-time activation,11 17 18 25 29 48 49 52 and half being interruptive.12 29 36 48 50 51 53 In five studies the alerts were presented within the clinical workflow.25 29 49–51 The most popular recipient of the alerts was a pharmacist for harm prevention interventions and a member of the MDT for early diagnosis interventions. The digital maturity of interventions was low with the majority at level 2 and only three at level 3 or above.36 50 53

Macro level

Just three studies had interventions that were designed to work at the macro (whole system) level.31 37 54 Two focused on earlier diagnosis,31 54 and one on harm prevention.37 Two studies were based in the ambulatory care setting, one used alerts to notify primary care physicians of patients with AKI who needed review and the other identified contraindicated medication prescription in patients with compromised renal function. The third study described an organisation-wide quality improvement programme that included staff education, development of a care bundle and a renal RRT. All used changes in SCr level to trigger a response, all were interruptive, two-thirds were real-time and targeted at physicians. These studies involved digital systems that spanned more than one organisation across the care system and therefore considered to have high digital maturity.

Measures and outcomes

Study measures provide an implicit indication of the intervention goals. At the micro level, process measures for harm prevention interventions included adjustment of individual patient medication dose, completion of a medication review and the time to medication adjustments or changes in monitoring regimes. Similar process measures were seen for early diagnosis and risk prediction interventions, focussing on changes in the recognition and recording of AKI, institution of appropriate individual patient management and the timing of such actions or the timing between recognition of deterioration and escalation to higher acuity or specialist levels of care.

Process measures at the meso level were similar to those seen for micro harm prevention interventions, with the addition of measures reflecting the degree of acceptance of pharmacist recommendations by physicians. Meso-level interventions that focused on early diagnosis used process measures such as time to AKI recognition, the percentage of changes made across the care pathways of interest, number of activations of renal RRTs and the time between team activation and patient intervention. AKI detection rate and clinician engagement with renal RRTs were process measures for early diagnosis interventions at the macro level. For harm prevention interventions, the proportion of inappropriately prescribed nephrotoxic drugs was measured.

Outcome measures were similar across all system levels and included AKI rates, AKI severity, rates of recovery, progression, initiation of renal replacement therapy, admissions to higher acuity or specialist care, length of stay and mortality. For harm prevention interventions this was supplemented with proportions of adverse events.

The impact of the interventions was mixed. Among micro-level interventions over half of early diagnosis interventions showed positive changes in outcomes.26 27 34 44–46 Only one study was a RCT34 and this showed a reduced length of stay. A quarter of harm prevention studies at this level found improvements in outcomes,15 38 40 none of which were RCTs. Both of the risk identification studies had a positive impact on outcomes. At the meso level there were no high-quality studies. One-fifth of harm prevention11 18 and two-fifths of early diagnosis50 51 interventions had the desired impact. At the macro level, one RCT found a reduction in mortality following an ambulatory care intervention to increase the recognition of AKI.31 Across harm prevention interventions at all levels there was evidence of a positive change in the most common process measures (reduced prescription of nephrotoxic medication and more appropriate dosing) in 42% of studies.15 17 19 21 23–25 35 36 47 49 Fewer earlier diagnosis intervention studies (29%) showed positive findings for the most common process measures (time to recognition and response to AKI and institution of more elements of appropriate management).12 26 27 32 34 50

Discussion

Given the longstanding availability of AKI digital information we used this condition to examine how digital clinical systems were maturing towards LHSs. Our findings show that while such systems have had a positive effect for over 30 years at micro levels, their application at macro levels is emergent. Most interventions used SCr levels to trigger alerts or algorithms in real or near real-time to enable risk prediction, early recognition of AKI or harm prevention by individual clinicians or specialist teams such as pharmacists and renal RRTs. Evaluations using process measures indicate apparent gains in harm reduction through avoidance of nephrotoxic medications or doses, or earlier prediction of the risk of deterioration. Evidence for improved outcomes is limited, with change more often seen in proximal outcomes such as length of stay in the lower quality studies and a few studies reporting reduction in mortality.31 40 44 45 47 54 Much remains to be understood about the longevity and sustainability of the interventions, but there are signals that this may be feasible within integrated health systems.54

The limited evidence on interventions and positive outcomes at the meso and macro level may be explained by several factors. Many digital systems have evolved from clinician interest in better management of individual patients and recognition that the ‘right’ data needs to be presented in an appropriate format, in a timely manner at the appropriate point in the workflow. Thus, the majority of reported interventions were targeted at individual clinicians or specialist teams, using changes in SCr as the trigger. Expansion of the use of real-time digital clinical information to improve quality of care at meso and macro levels will also require the increasing digital maturity of systems. With the transition from standalone to integrated EHRs within and across health systems more data will be available not just to clinicians at the point of care, but also the wider MDT as well as organisation and system managers.

However, data and digital systems alone are insufficient for changing or influencing behaviours. Recognising and considering the role of human factors in EHR design, adoption and utilisation is important to ensure maximum benefit of digitally enabled real-time data at relatively neglected meso and macro levels. Furthermore, challenges of generating actionable data include considerations of how the data are conveyed to enable a real-time response from the most appropriate persons. In the evidence reviewed, many systems relied on interruptive alerts or alerts that were outside the clinicians’ workflow. Other reviews have highlighted that success of alerts and accompanying clinical decision support systems to change user behaviours is dependent on workflow integration, level of intrusiveness and presence of multiple competing alerts, with alert fatigue cited as the most frequent reason for ineffectiveness.55 56

Successful transition from data utilisation to data driven healthcare has implications for technical factors (system design), individual practices (behavioural impact) and resources (individuals, infrastructure), and requires a supportive, adaptive policy environment.57 Advances in technical factors through EHR systems within organisations are becoming established but need to progress towards integration and interoperability across organisations and with other systems, such as management databases for staffing. A range of disciplines need to be involved in further developments, including clinicians, human factors experts, behavioural scientists, technology experts and data scientists. Developing the analytics capability and digital literacy of clinical and administrative staff is fundamental for successful LHSs, to develop mechanisms to monitor the impact of the use of information and to enable continuous tailoring (to different contexts and staff compositions), especially in light of changing contexts and the need to respond to user feedback.

The recent experience of the COVID-19 pandemic illustrates that under these unusual conditions adaptive and enabling policies, with the rapid development, deployment and innovative use of digital systems can enable continuity of healthcare delivery across acute and primary care sectors. Other examples of data-driven enabling policies at macro level such as the UK value-based commissioning,58 or ‘getting it right first time’ programmes,59 demonstrate the feasibility of using routinely collected clinical data at system level to determine care outcomes or to better understand the causes of their variation, signalling what might be possible within an effective digital LHS.

The NHS and healthcare systems more widely are quite complex, and therefore from a research perspective, evidence is needed from studies that go beyond immediate care settings expanding measurement to indicators of system dependent health outcomes such as hospital avoidance, reduced length of stay and access to healthcare services. Well chosen patient-centred process and outcome indicators from across the system will provide feedback in real-time to steer individual patient care, as well as provide information that may be available later for reflective and responsive learning at population level, from small groups of patients up to larger populations. This requires a different real-time focus on the same data, promoting reactive behaviour at the micro level while also providing insight into variations that may be addressed at meso and macro levels through adaptive changes in service delivery and resource (re)distribution, as seen during the recent pandemic responses and policy changes.60–62

Strengths and limitations

Our scoping literature review format combining clearly defined key concepts and a systematic approach enabled exploration and synthesis of a complex and heterogeneous area and the capture of most relevant and appropriate articles. However, there may be examples of the use and impact of real-time data at meso and macro level not published in academic literature as developments at these levels are relatively immature. Moreover, we may have misclassified some intervention across micro, meso and macro levels as the interventions were not always well described. It was also not our intention to formally assess the quality of included papers given that we were as interested in which dimensions of intervention process or outcomes were chosen for measurement as we were in the impact of the intervention. In the majority of cases, drawing conclusions about the latter was difficult given the limitations of study designs used.

Conclusions

Digital transformation, use of data in real-time and LHSs are cornerstones for achieving the triple aim to improve population health, quality of care and cost control.63–65 Wider approaches are now required to build on the initial impact seen at individual patient level in order to gain benefits across the system, particularly in service delivery and resource distribution. This will require a coordinated effort across developments in technical, human factor and policy arenas with adequate resourcing. The lessons learnt from deployment of digital systems to enable the coordination of resources across primary and secondary care during the COVID-19 pandemic should act as a powerful catalyst.

Overview

Article metrics
Altmetricopen-url
Dimensionsopen-url