Original Research•

Changes in medication administration error rates associated with the introduction of electronic medication systems in hospitals: a multisite controlled before and after study

Abstract

Background Electronic medication systems (EMS) have been highly effective in reducing prescribing errors, but little research has investigated their effects on medication administration errors (MAEs).

Objective To assess changes in MAE rates and types associated with EMS implementation.

Methods This was a controlled before and after study (three intervention and three control wards) at two adult teaching hospitals. Intervention wards used an EMS with no bar-coding. Independent, trained observers shadowed nurses and recorded medications administered and compliance with 10 safety procedures. Observational data were compared against medication charts to identify errors (eg, wrong dose). Potential error severity was classified on a 5-point scale, with those scoring ≥3 identified as serious. Changes in MAE rates preintervention and postintervention by study group, accounting for differences at baseline, were calculated.

Results 7451 administrations were observed (4176 pre-EMS and 3275 post-EMS). At baseline, 30.2% of administrations contained ≥1 MAE, with wrong intravenous rate, timing, volume and dose the most frequent. Post-EMS, MAEs decreased on intervention wards relative to control wards by 4.2 errors per 100 administrations (95% CI 0.2 to 8.3; p=0.04). Wrong timing errors alone decreased by 3.4 per 100 administrations (95% CI 0.01 to 6.7; p<0.05). EMS use was associated with an absolute decline in potentially serious MAEs by 2.4% (95% CI 0.8 to 3.9; p=0.003), a 56% reduction in the proportion of potentially serious MAEs. At baseline, 74.1% of administrations were non-compliant with ≥1 of 10 procedures and this rate did not significantly improve post-EMS.

Conclusions Implementation of EMS was associated with a modest, but significant, reduction in overall MAE rate, but halved the proportion of MAEs rated as potentially serious.

Summary

What is already known?

  • Medication errors are one of the leading causes of preventable patient harm in hospitals, and in response the WHO has nominated medication safety as the current 5-year global patient safety challenge.

  • Electronic medication systems (EMS) have been demonstrated to effectively reduce prescribing error rates, yet evidence of these systems improving medication administration error (MAE) rates or reducing associated harm is very limited.

  • No previous multisite or controlled studies evaluating the effects of EMS on MAE rates have been published.

What does this paper add?

  • We conducted a controlled before and after study at two major adult teaching hospitals to measure the effects of EMS on MAE rates.

  • Overall, 30.2% of all administrations observed had one or more clinical error and 3.3% of errors were rated as potentially serious and likely to cause permanent harm.

  • Implementation of EMS was associated with a significant but modest 14% overall reduction in MAE rate, with the greatest reduction observed in ‘wrong timing’ errors.

  • However, the intervention wards experienced a 56% reduction in medication administrations with a potentially serious MAE, suggesting that EMS are effective in reducing safety risks during medication administration.

Introduction

In 2017 the WHO announced medication safety as the next global patient safety challenge, reflecting continuing concerns regarding the high rates of medication errors and their impact on health outcomes and costs.1 Much attention has focused on prescribing error rates. In comparison, assessments of medication administration errors (MAEs) are substantially less frequent. A major reason for this disparity in evidence is the difficulty in measuring MAEs. Unlike prescribing errors, which can be assessed using retrospective chart reviews, accurate detection of MAEs requires direct observational studies, which are methodologically more difficult and resource-intensive to conduct.2 MAE rates calculated from incident report and chart reviews vastly underestimate the true occurrence of errors.3–6 Available evidence from systematic reviews of direct observational studies shows considerable variation in MAE rates between studies, with pooled estimates between 5% and 30% of dose administrations with errors.2 7–10

Evidence of effective interventions to reduce MAEs is also scant. A systematic review and meta-analysis of observational studies assessing the effectiveness of various training and technology-related interventions to reduce MAE rates found no significant effects.11 Electronic medication systems (EMS), also known as computerised provider order entry systems (which allow clinicians to record the prescription and administration of medications electronically), have made significant contributions to reducing the rates of both procedural medication errors (eg, illegible or incomplete medication orders) and clinical prescribing errors (eg, wrong drug errors). However little research has investigated whether the electronic medication administration records in these systems, used by nurses for the preparation and administration of medications, are associated with reduced MAE rates. A small number of before and after studies have been undertaken,12 13 but none has applied a controlled study design and all have been in single sites.

The aim of this study was to conduct a controlled before and after study to investigate changes in MAE rates following the implementation of EMS at two large adult teaching hospitals.

Methods

Study design and intervention

A controlled before and after study was conducted at two major metropolitan adult teaching hospitals in Sydney, Australia. Hospital A is a 400-bed hospital and hospital B a 326-bed hospital. Each hospital was in the process of implementing a commercial EMS which included electronic prescribing and medication administration functionality. At hospital A data were collected from four general medical/surgical wards (two acute aged care, one renal/vascular/dermatology and one acute respiratory ward; figure 1). The two acute aged care wards (intervention wards) implemented the EMS (Cerner), and the remaining two wards acted as control wards where paper medication charts continued to be used. At hospital B, an EMS (iSoft Medchart) was implemented on one ward (acute orthopaedic) and a second ward (acute neurological) acted as a control ward.

Figure 1
Request permissionrequest permission icon
Figure 1

Study design. MAE, medication administration error.

Both EMS had limited decision support and medication administration was not supported with bar-code scanning. Nurses used the EMS to identify the medications to be administered to patients, with medications due for administration highlighted on screen.

Data were collected on all six wards in both the pre-EMS and post-EMS implementation periods. A minimum of 10 weeks postsystem implementation elapsed before the postdata collection occurred (figure 1).

Observation procedures

Nurses were recruited through information sessions and invited to participate by researchers on the study wards. In total 180 of nurses (constituting >90% of available participants) agreed to participate in the study and provided signed informed consent. Trained observers arrived on the study wards each day at the key medication administration times between the hours of 06:00 and 22:00 and randomly selected nurses to observe based on a list of consented nurses on that ward.

Observers shadowed nurses as they prepared and administered medications on the ward. Detailed information, including drug name, dose and route given, was recorded in real time using a handheld computer with specialised software (the Precise Observation System for the Safe Use of Medicines (POSSUM)).14 Adherence to relevant medication administration safety procedures and guidelines, such as reading the medication label, checking patient identification and using aseptic technique, was also recorded (see table 1 for the 10 procedures observed).

Table 1
|
Clinical and procedural medication administration error categories

Observers recorded medications without knowledge of what medications were documented on patients’ medication charts. If observers noticed an error which had the potential to cause serious patient harm (eg, a tenfold dose of a medication), they were instructed to follow a ‘serious error’ protocol devised for the study which outlined the steps required to intervene.

Observers were nurses independent from the study hospitals and underwent extensive training using scenarios and field testing. Inter-rater reliability scores were calculated during practice sessions by having two observers collect data from the same medication administration events and then comparing them for agreement. Data collection commenced once κ scores15 were >0.8. Data collected during observations were later compared against patients’ medication charts to identify MAEs.

Error classification

Errors were classified as either clinical errors (13 types) or procedural errors (10 types) (see table 1). Clinical errors were defined as medication administrations which deviated from the prescriber’s medication order documented in the patient’s chart; the manufacturer’s preparation/administration instructions; or relevant hospital medication administration policies, for example the Australian Injectable Drugs Handbook.16

Clinical errors included wrong timing errors, which were defined as a medication that was administered >60 min before or after the prescribed time, or >30 min before or after a meal time if the medication was ordered to be administered with or in advance of food. Procedural errors were medication administrations where the preparation, administration or documentation of the drug did not comply with the law, State Department of Health or hospital policy or guideline.

The potential severity of errors was rated using a 5-point severity scale based on the New South Wales Health Department state-wide hospital incident monitoring system (see table 2). A nurse and pharmacist rated the severity, and disagreements were reviewed by a clinical pharmacologist to gain consensus. Those MAEs rated as 3 or above were defined as potentially serious MAEs.

Table 2
|
Clinical error severity rating scale

Statistical analysis

The error rate for each clinical error category was calculated by dividing the number of errors by the total number of administrations where that error category was applicable. The overall MAE rate was presented for each study period (pre and post) and group (control and intervention wards). The difference in error rate changes post-EMS in the intervention group relative to control groups was defined as the differences of the error rates over time between control and intervention groups, expressed as:

Display Formula

The 95% CIs were obtained using the normal approximation method. The overall procedural error rate and rate change for each procedural error category were calculated using the same definition and procedure as for clinical error rates. The proportion of administrations with at least one clinical error and proportion of administrations with potentially serious errors, that is, severity score ≥3, were presented by study period and group. The z-tests for proportions were used to compare the changes over time between study groups, with the level of significance set at p<0.05. Data analysis was conducted using SAS V.9.3.

Results

A total of 7451 medication administrations were observed during the study, of which 4176 were during the baseline/preintervention period (2463 on control wards, 1713 on intervention wards) and 3275 were during the postintervention period (1528 on control wards, 1747 on intervention wards).

Clinical and procedural MAE rates at baseline

Table 3 reports the clinical MAE rates at baseline by error category and ward. At baseline, 30.2% of administrations across the six wards had one or more clinical MAEs. The four most frequent clinical error types were wrong intravenous rate (46.2 errors per 100 administrations), wrong timing (19.6 per 100 administrations), wrong solvent volume for injectable medications (12.0 per 100 administrations) and wrong dose errors (3.0 per 100 administrations) (table 3).

Table 3
|
Clinical medication administration errors at baseline

Table 4 reports the procedural error rates and shows that the lowest level of compliance at baseline was failing to correctly check infusion pumps and patient identification. Overall, 74.1% of administrations were observed to have one or more procedural error at baseline.

Table 4
|
Procedural medication administration errors at baseline

Changes in MAE post-EMS implementation

We measured the pre–post changes in MAEs on the control and intervention wards and then compared the magnitude of changes to determine whether the intervention wards experienced a greater change in MAE rates relative to any changes observed on the control wards.

As shown in table 5, on the intervention wards we found there was a significant reduction in the percentage of administrations with a clinical error relative to the control wards (4.2%; 95% CI 0.2% to 8.3%, p=0.04). The greatest decline was in wrong timing errors, which decreased by 3.4 errors per 100 administrations (95% CI 0.01 to 6.7) on the intervention wards relative to the control wards.

Table 5
|
Clinical medication administration errors rates post implementation of the EMS and change in error rates between baseline and post-EMS

Changes in procedural MAE rates

There was no significant change in the overall rate of procedural errors between the intervention and control groups over the study period (administrations with a procedural error decreased by 1.5% (95% CI −5.2% to 2.2%, p=0.4). Some individual categories of procedural error rates on the intervention wards did change significantly relative to the control wards post-EMS. Compliance with reading medication labels improved by 3.3% (95% CI 2.0 to 4.7) and checking patients’ pulse or blood pressure prior to digoxin administration improved by 40.3% (95% CI 5.1 to 75.4) in the intervention group relative to the control group. However, there were also procedural error rates which increased on the intervention wards post-EMS. These included an increase in the rate of temporary storage of medications prior to administration (18.5% reduced compliance, 95% CI −22.1 to −14.80), reduced aseptic technique (−11.7%, 95% CI −11.5 to −8.2), two nurses witnessing administration of S4 and S8 drugs (−22.8%, 95% CI −39.6 to −6.1), checking patient identification (−5.9%, 95% CI −10.2 to −1.5) and two nurses checking the preparation for injectable S4 and S8 drugs (−5.0, 95% CI −9.7 to −0.3).

Changes in potentially serious MAE

At baseline, 3.3% (95% CI 2.7% to 3.8%) of all administrations had MAEs with a potential severity score of ≥3 (table 3). The intervention wards experienced a reduction in administrations with serious MAEs from 4.2% (95% CI 3.3% to 5.2%) pre-EMS to 1.8% (95% CI 1.2% to 2.5%) post-EMS. The control wards experienced no significant change in the proportion of serious MAEs (2.6%, 95% CI 2.0% to 3.3%, pre-EMS; 2.6%, 95% CI 1.8% to 3.4%, post-EMS). Thus, the size of reduction in serious MAEs attributable to the EMS was an absolute reduction in serious MAEs of 2.4% (95% CI 0.8% to 3.9%; p=0.003). This equated to a 56% overall relative reduction in serious MAEs on the intervention wards.

Discussion

This study confirms that MAEs are a frequent occurrence in hospitals, with a baseline clinical MAE rate of 30.2%. This rate is at the higher end of the range of those previously reported, with a systematic review of 52 direct observational studies finding a median error rate of 19.6% (IQR 8.6%–28.3%) of medication administrations with at least one error, including wrong timing errors.9 Procedural errors were particularly frequent, with 74.1% of administrations incurring one or more procedural error.

The introduction of EMS in our two study hospitals reduced the overall occurrence of clinical MAEs by 4.24 errors per 100 administrations, a reduction of 14%. There are few comparative study results. Vicente Oliveros et al 12 conducted a before and after EMS study which involved two pharmacists directly observing medication administrations and identifying and classifying errors in real time. They reported a significant 23.1% decline in the overall proportion of medication administrations with errors post-EMS from 48.0% to 36.9% (p<0.05). The greatest reduction was observed in omitted dose errors, which halved. A direct observational study of 428 ‘opportunities for administration errors’ on a geriatric ward in an English hospital found no significant change in MAE rates following EMS implementation (4.2%, 95% CI 2.3% to 6.1% pre-EMS vs 3.4%, 95% CI 1.9% to 5.0% post-EMS) but excluded wrong timing errors.13 In that study the observers usually viewed patients’ medication charts prior to observing the administration process, which may have introduced a source of bias. A further English study by Franklin et al 17 using similar methods on one general surgery ward over 56 drug rounds reported a significant 4.2% reduction in MAEs (from 8.6% pre-EMS to 4.4% post-EMS; p=0.0003). In contrast to our study, Franklin et al 17 found no change in mean severity score for MAEs post-EMS. Without control wards it is difficult to assess the extent to which results from previous studies can be attributed to the EMS. As our study demonstrates, without the data from our control wards we would have substantially overestimated the changes in MAE rates on the intervention wards attributable to EMS use.

The overall decrease in MAEs we identified post-EMS was largely driven by a decline in wrong timing errors, which fell by 3.4 errors per 100 administrations, approximately a 17% reduction from baseline rates. Wrong timing errors were the second most frequent clinical error at baseline, consistent with previous reports. Keers et al 9 reported in their systematic review that approximately 80% of MAE studies reporting wrong timing errors identified them as one of the top 3 most common error types. The definition of wrong timing applied in these studies varied, but administration within ±60 min of scheduled administration was the most often applied definition. As our results indicate, EMS systems may be particularly helpful in guarding against wrong timing errors and dose omissions due to the capacity of systems to highlight and flag scheduled doses.18–21 We were unable to study dose omissions, as not all medication administrations on the wards were observed and thus it was impossible to be certain that a dose was missed.

The effectiveness of EMS in reducing prescribing error rates in our study hospitals has been published previously.22 The introduction of EMS had a greater effect on reducing prescribing error rates than on MAE rates, with 57.5%–66.1% reductions in prescribing errors on the intervention wards. The magnitude of the effect on overall MAEs we found was lower, at around 14%. However, despite the modest overall reduction in MAEs, there was a substantial reduction, of 56%, in the proportion of MAEs that were rated as potentially serious (from 4.2 to 1.8 potentially serious MAEs per 100 administrations) on the intervention wards post-EMS. In a direct observational study of 2314 medication administrations in two clinical units in a Spanish hospital which had an EMS in place along with automated dispensing cabinets, an MAE rate of 22 per 100 administrations was reported. In total, 2.7% of those errors required some form of additional monitoring or resulted in temporary harm to patients.23

Our study provides valuable prevalence data on specific categories of MAEs. As other studies have reported,9 24 we found that intravenous medications had high rates of clinical errors at baseline and post-EMS, with the most frequent clinical error category being wrong intravenous infusion rate (46.2% of all intravenous administrations at baseline) and wrong volume errors which occurred at a rate of 12% of intravenous administrations. Intravenous medication administrations are a known high-risk event for patient safety, having previously been estimated to be five times more likely to have an error in their administration than non-intravenous medications.2

Nurses failing to follow medication administration procedural policies and guidelines was common. At baseline, only one in four medication administrations complied with all relevant procedures studied (74.1% non-compliant). Nurses failed to have infusion pump rates checked by a second nurse 63% of the time. Kim and Bates25 reported non-adherence rates of up to 95.5% for certain medication administration guidelines. Assuming that the guidelines correctly identify processes at high risk of error, the low compliance rate could be contributing to the high frequency of MAEs, although for many of the accepted safety procedures there is insufficient evidence to determine that they are effective in reducing MAEs in practice (eg, double-checking).26 Certain guidelines are frequently not followed, which suggests that there are learnt workplace behaviours or organisational culture factors that could be addressed.27 For example, a study by Drach-Zahavy et al 28 linked the learning culture on wards to MAE rates. The EMS could potentially reduce demands on nurses associated with some safety procedures. For example, aiding dose calculations could reduce cognitive demands in the double-checking process.

Following EMS implementation, we found no change in the overall procedural error rate. We examined a broad range of procedures, some of which involved direct interaction with the EMS (eg, recording and signing for administrations; correctly checking patient identification) while others (eg, use of aseptic technique and temporary storage of medications) did not. For several medication administration procedures (eg, temporary storage of medications, using aseptic technique, double-checking administration of ‘dangerous drugs’ and checking patient identification prior to administration), compliance substantially decreased on the intervention wards compared with the control wards. These results may reflect an increased burden of the EMS on nurses’ time and workflow efficiency, with nurses skipping steps to make up for time spent using the system.29 Another possibility is that it is an artefact of being observed—staff may initially have been more vigilant in complying with all procedural guidelines, but once the intervention was introduced they may have assumed the observers were more interested in the EMS than the whole MAE process, or were unable to maintain the same level of vigilance due to increased cognitive demands placed on them while using the new system. However, our study is unable to shine a light on the reasons for non-compliance with the range of medication administration procedures captured. Understanding the way EMS are used, how, for example, system interfaces and functionality facilitated or inhibited work, and why specific procedures are followed or not is best investigated using qualitative methods.30 As others have reported, the introduction of medication technologies influences the ways in which work is performed in both expected and unexpected ways.31 32 Whatever the reasons, our results indicate that introduction of an EMS was not associated with increased compliance with many core medication safety procedures.

However, one substantial area of procedural improvement on the intervention wards was the measurement of patients’ blood pressure and pulse prior to administration of digoxin. This result related to the addition of targeted decision support in the EMS which alerted nurses to this requirement prior to administration. On the intervention wards there was a 40.3% increase in compliance with this procedure post-EMS, suggesting that judicious use of alerts and other decision support tools can be effective.

This is the first controlled, multisite, before and after study to quantify the impact of EMS on MAE rates, but it has some limitations. The intervention wards were selected by hospital management for operational reasons and could not be randomised. Only a sample of medication administrations were observed during the study periods, and therefore dose omissions could not be included. As dose omissions have been reported in several studies as a common MAE type,8 19 21 33 34 and also an error type that can be reduced by the introduction of an EMS,19 we may have underestimated both the true frequency of MAEs at baseline and also the impact of the intervention.

In summary, we found that the introduction of EMS at two hospitals was associated with a modest, but significant, reduction in overall MAE rate, and the proportion of MAEs rated as potentially serious halved. While the effects of the EMS on MAE reduction were less than the reductions previously reported for prescribing errors, it is important to recognise that the introduction of these systems is a foundation for moving towards complete closed-loop medication systems which incorporate further technologies, such as bar-coding, and can bring additional benefits in error reduction.17 35 Thus, in assessing these systems the summative value of the different components is an important consideration.

Overview

Article metrics
Altmetricopen-url
Dimensionsopen-url