Article Text
PDF
Abstract
Objectives Computerised provider order entry (CPOE) systems have been implemented around the world as a solution to reduce ordering and transcription errors. However, previous literature documented many challenges to attain this goal, especially in paediatric settings. The objectives of this study were to (1) analyse the impact of a paediatric CPOE system on medication safety and (2) suggest potential error prevention strategies.
Methods A pre-post observational study was conducted at the pilot ward (n=60 beds) of a paediatric academic health centre through mixed methods. The implementation project and medication management workflows were described through active participation to the project management team, observation, discussions and analysis of related documents. Furthermore, using incident reports, the nature of each error and error rate was compared between the preperiod and postperiod.
Results The global error rate was lower, but non-statistically significant, in the post implementation phase, which was mostly driven by a significant reduction in errors during order acknowledgement, transmission and transcription. Few errors occurred at the prescription step, and most errors occurred during medication administration. Furthermore, some errors could have been prevented using a CPOE in the pre-implementation period, and the CPOE led to few technology-related errors.
Discussion and conclusion This study identified both intended and unintended effects of CPOE adoption through the entire medication management workflow. This study revealed the importance of simplifying the acknowledgement, transmission and transcribing steps through the implementation of a CPOE to reduce medication errors. Improving the usability of the electronic medication administration record could help further improve medication safety.
- information technology
- clinical pharmacy information systems
- health information systems
- pharmacy research
- safety management
Data availability statement
No data are available.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- information technology
- clinical pharmacy information systems
- health information systems
- pharmacy research
- safety management
WHAT IS ALREADY KNOWN ON THIS TOPIC
Many challenges to the safe implementation of electronic medication prescribing have been documented, especially in paediatric settings. Reducing medication errors is difficult, and new errors may arise with the introduction of the technology.
WHAT THIS STUDY ADDS
This study highlights the importance of simplifying the workflows of the whole medication management process with the technology. Specifically, acknowledgement of the order (by nurses), validation for dispensation (at the pharmacy) and medication administration (by nurses) are crucial in improving safety of medication use with technology.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Future work should focus on the whole medication management process and analyse the usability of key features prior and during implementation.
Introduction
The medication management process is a complex process that includes prescribing, transmission, preparation and administration of medication. Several problems can arise at various stages of this process, such as transcription errors, drug interactions or administration problems. Medication errors are prevalent worldwide, with WHO launching the global Medication Without Harm initiative in 2017 to cut medication errors in half in 5 years.1 In 2019, Québec’s Ministry of Health and Social Services’ annual accident/incident report identified that 26% of accidents/incidents in all the province’s healthcare facilities (n=130 520) are drug related, including errors due to illegible prescriptions, undetected allergies and wrong prescription weight. Similarly, the CHU Sainte-Justine (CHUSJ), an academic mother-and-child health centre with over 400 beds, identified that 27% of accidents/incidents (n=1346) are drug related.2
Computerised provider order entry (CPOE) is one of the promising solutions to improve the quality of use and safety of prescribed medication.3 4 In Québec, the CHUSJ was the first healthcare institution to implement CPOE that enabled both prescription of medications and non-medication orders. However, the challenges of implementing CPOE can lead to some detrimental consequences, for example, by generating errors due to the system configuration,5 and might not improve medication safety.6 Therefore, the impact of CPOE on medication safety depends greatly on the clinical setting and the CPOE system configuration.5
Additionally, paediatric patients are particularly vulnerable to medication errors due to the off-label use of numerous drugs, paediatric-specific drug–drug interactions, as well as their wide variation in age and weight, which can lead to 10-fold dosing errors (ie, underdosing or overdosing error by a factor of 10).7–9 Indeed, Tolley et al identified the lack of dosing support as the most crucial factor that contributed to CPOE-related errors in paediatrics.10 Previous studies in paediatric settings have also highlighted the importance of minimising disruptive alerts and modifying directly the ordering workflow to avoid error-prone steps (eg, implementing rules to avoid 10-fold errors directly in the CPOE).11 12 Therefore, designing and implementing an effective and satisfactory system tailored to the paediatric population’s needs and local clinical environment is critical to ensuring medication safety.13–16
This project leverages the clinical adoption meta-model framework,17 in which the dimensions of availability, usage and outcomes continuously evolve based on one another, to evaluate the impact of the implementation of CPOE on medication safety in a paediatric pilot unit. More specifically, this study sought to measure and contextualise the impacts of the CPOE adoption by (1) describing the CPOE implementation project and the medication ordering workflows before and after CPOE implementation (as an indicator of availability and usage), (2) describing the rate and types of clinical errors during various stages of the medication management process (as an indicator of safety outcomes) and (3) identifying potential health information technology (HIT)-related prevention strategies based on error reports (as a continuous improvement strategy).
Methods
Study design and site
An observational pre-post study was conducted in the 60-bed general paediatric medicine unit, the largest unit at the CHUSJ, in October 2019. The CHUSJ was selected for this study because it was the first paediatric hospital to implement this newly developed CPOE. Furthermore, the general paediatric unit was chosen for the evaluation of the CPOE, notably because it was the pilot unit within the hospital and received a variety of orders as a general paediatric medicine unit. The unit comprises four medical teams, each led by one attending physician and composed of medical residents and students, and two clinical pharmacists, and four nursing stations.
All orders were handwritten into the patient’s paper record before the implementation of the CPOE system. The hospital uses a pharmacy information system (PIS), an electronic medication administration record (eMAR) since 2017, a clinical data repository, as well as laboratory and radiology ancillary information system. Clinical and nursing notes are documented in paper records.
Data source and analysis
To evaluate the system’s availability and usage, non-participant observation sessions (13 hours over 3 days in May 2019),18 19 active participation to the project implementation team and content analysis of related documents (eg, internal presentations, training documents, discussions with stakeholders) were conducted to develop a better understanding of local usage practices. Free text observation notes and discussions on the medication management workflow were first documented in a table where each row corresponds to the observed user and each column corresponds to a step of the workflow and synthesised into a table describing the difference and similarities of the workflow before and after the CPOE implementation. A timeline of the implementation project was iteratively elaborated with stakeholders during the study period.
To evaluate outcomes, all medication-related incident or accident reports in the paediatric unit at the CHUSJ from 20 October 2018 to 21 October 2020 (ie, 1 year before and after the implementation of the CPOE on 21 October 2019) were extracted and analysed. These safety reports are manually collected by clinical staff on a routine basis as mandatory reporting to the Health Ministry if the error has directly affected the patient and required some monitoring or treatment (grade D or higher).2 Data extracted from the reports were (1) description of the event, (2) type of event, (3) consequences observed for the person affected, (4) measures taken to avoid or limit the consequences, (5) declarant’s proposed prevention strategy and (6) declarant’s assessment of the severity. Additionally, drug categories, drug routes, type of events and type of proposed prevention strategy were thematically constructed based on categories used in similar studies.20–24 Lastly, two pharmacists working in HIT identified additional technology-related prevention strategies by analysing the error’s descriptions, which were compared with the declarants’ proposed prevention strategies. This study was reported using the Statement on Reporting of Evaluation Studies in Health Informatics guidelines.25
Results
Implementation project
The implemented CPOE system was PANDAWebRx,26 a web application developed by CGSI@SOLUTIONS-TI in collaboration with the hospital project team (one clinical informatics manager and two full-time project managers). A previous usability analysis conducted a month before the implementation revealed the need to optimise the clinical decision support system (CDSS) to identify inappropriate dosing instructions for paediatric patients,27 which is also known to be a recommended practice in the Safety Assurance Factors for EHR Resilience(SAFER) Guides.28 The implementation process lasted more than a year (online supplemental figure 1). Order sets were developed by a team of clinicians on the paediatric unit to standardise the ordering process and ensure that they were adapted to the clinical workflows. Pilot testing occurred at each workstation during the summer of 2019 for periods of 48 hours. In the month prior to the Go-live, all physicians, nurses and other providers on the unit were required to complete online training modules (30–90 min), followed by in-classroom order scenario testing (60 min).
Supplemental material
The Go-live occurred on 21 October 2019. On-site and phone line support was provided 24/7 for 4 months. Although the entire unit switched to electronic prescriptions, some rare paper prescriptions were still written in the first months following the Go-live. These paper prescriptions occurred when the prescriber, most often from another specialty, would want to prescribe using a specific formulary and had difficulty doing so with the electronic format. As of August 2021, the CPOE has been implemented in other hospital units.
Description of the CPOE system
Previous versions of PANDAWebRx were used to prescribe drugs in adult settings. The new version was adapted for paediatric prescribing (eg, paediatric order sets). However, the system did not include paediatric dose range checking or other paediatric specific alerts since the development of a paediatric CDSS represented a separate feature that was not developed during the first phase of the CPOE implementation. However, the CPOE system integrated a CDSS (RxVigilance by Vigilance Santé) for allergies and drug–drug interactions alerts, which is deactivated by default and is interfaced directly with the hospital’s PIS and eMAR. Figure 1 shows the screenshots of the CPOE and eMAR modules. The CPOE was designed by following many of the recommendations in the SAFER Guides to prevent unwanted consequences, as detailed in the online supplemental table 1.
Supplemental material
PANDAWebRx (Aa) CPOE view and (Bb) eMAR view. In the CPOE view, the provider can choose to group orders by type of orders (medication, nutrition, surveillance, imaging, etc.) or by protocol. Critical information for prescribing (weight, allergies, kidney function) is displayed on the top panel. CPOE, computerised provider order entry; eMAR, electronic medication administration record.
Description of the medication management process
Based on a content analysis of the non-participant observation notes, project documents and discussions, we identified that the medication management process was significantly revised for the CPOE implementation (online supplemental table 2). Significant changes occurred for the ordering and acknowledgement, transmission and transcribing steps. Nurses were no longer tasked with scanning and transmitting the prescription to the pharmacy department. Furthermore, nurses acknowledged new prescriptions electronically and would import the CPOE data to the eMAR without manual transcription. Similarly, in the pharmacy department, pharmacy technicians would import the order information instead of manually transcribing the orders. After importing the CPOE data into the PIS, the pharmacy technicians would complete the order with other required information (eg, dispensed drug product, pharmacy comments).
Medication errors
A total of 133 and 109 medication-related accidents and incidents were reported during the pre-implementation and post implementation periods, respectively. Medication-related errors in the paediatric unit represented 31% (133/429) and 23% (109/466) of all types of medical incidents/accidents in pre-implementation and post implementation periods, respectively. There were no statistically significant differences when comparing the proportions for the drug categories, drug routes, time of incident/accident or severity, except for a difference in reports related to drugs with buccal administration and errors with a severity level of A (online supplemental table 3). The majority of the reports were adverse events that did not lead to any patient consequence (72% and 73% of the reports in preperiods and postperiods, respectively). A similar analysis conducted to evaluate the effect of the COVID-19 pandemic by comparing reports from the first 6 months of the pandemic (March to October 2020) with the same period a year before revealed no differences for all variables.
Most of the medication errors occurred during the nurse administering step (step 4) (table 1). The rate ratios for each step were not significant, except for the order acknowledgement, transmission and transcribing steps (step 2; rate ratio: 4.4, 95% CI 1.1 to 32.5), which represented the second most common type of errors in the pre-implementation period. Overall, there is a slight, but not significant, reduction in the total number of medication errors between the two periods (rate ratio: 1.2, 95% CI 0.8 to 1.7). Medication errors during the ordering step (step 1) did not decrease after the CPOE implementation (rate ratio: 0.9, 95% CI 0.2 to 3.7).
Number and rate of medication errors at each step of the medication management process before and after the CPOE implementation
Examples of medication errors during both periods are presented in table 2 and grouped by the stage of the medication management process. There were notable differences between the types of errors occurring before and after the CPOE implementation, primarily at the ordering (step 1) and acknowledgement, transmission and transcribing (step 2) steps (table 1), which correlates with the considerable changes at these steps in the medication management process (online supplemental table 2).
Examples of errors identified at each step in the inpatient medication management process before and after the CPOE implementation
At the ordering stage, the CPOE standardised mandatory steps (eg, required countersignature for orders from external consultants) (example 1) and formalised steps that were previously not electronically documented (example 5). Furthermore, although the CPOE was designed to enable all possible order scenarios, some orders were prescribed on paper (example 4), which delayed the order. At the acknowledgement, transmission and transcribing steps, the electronic transmission of orders ensured that there were no transmission errors (example 9), and that the order information was correctly entered in the other systems (examples 10 and 11). Types of errors that were common to both periods occurred most frequently at the nurse administering stage, specifically during the preparation of the order. These include the preparation of orders with the wrong dose or wrong frequency, as well as documentation errors (examples 21, 22 and 25).
Lastly, although some examples were only reported during one of the two periods, these errors might not be related to the use (or lack of) of the CPOE. Instead, some examples could be due to other factors, such as technical factors (eg, eMAR usability: examples 19 and 20), human factors (eg, staff’s knowledge and skill: example 27) or organisational factors (eg, structure, culture, processes: examples 13, 16 and 17).
Recommendations for preventing medication errors
A total of 232 and 199 recommendations were extracted from the medication error reports and categorised based on the proposed taxonomy by Franklin et al21 (table 3). The most frequent types of recommendation during both periods were vigilance (eg, always make sure to check the patient’s eMAR carefully), counselling (eg, met with the nurse to review event) and education/training of healthcare workers (eg, redo training on diabetes, event discussed during 5 min staff huddles), which are person-based approaches. There were few system-based approaches (eg, speaking with the pharmacy department to ensure that medication orders with irregular frequency are prepared accordingly).
Event review approaches and specific categories with frequency counts and percentages before and after the CPOE implementation
On the total of 133 and 109 drug errors reported in the pre-implementation and post implementation periods, respectively, two pharmacists reviewed all reports and identified potential HIT-related prevention strategies for 85 (64%) and 64 (59%) of the pre-implementation and post implementation error reports. The remaining error reports were mainly associated with human and organisational factors and no technology-related prevention strategies were identified (table 4).
Potential HIT-related prevention strategy identified for all medication errors before and after the CPOE implementation
Discussion
Impact of the CPOE system on medication errors
Although e-prescribing systems have been previously evaluated in Canada, notably in outpatient settings,29 we believe, to the best of our knowledge, that this is the first study that evaluates the impact of an inpatient CPOE on medication safety in Canada. In 2021, less than 20% of the specialist physicians in the country used a system that could send orders electronically.30 By comparison, in 2016, 95.6% of the hospitals in the USA, where the majority of the CPOE evaluation studies were conducted, have adopted a CPOE with CDSS.31 Therefore, this study was conducted in a unique context in which the CHUSJ was an early adopter of a new, locally developed commercial CPOE with limited decision support. In addition, very few pre-post studies have been conducted to evaluate the impact of CPOEs in paediatric settings.32 Thus, by conducting a thematic analysis of the safety reports, we were able to target specific types of medication errors and identify potential prevention strategies adapted to the needs of this population.
The mixed-methods evaluation of the impacts enabled the identification of the main challenges related to the implementation of the CPOE based on the error frequencies, while contextualising the error rates with an analysis of the changes to the local practices. For instance, although the ordering process was significantly reviewed with the implementation of the CPOE, there was no difference in the rate of errors, which was also observed in previous studies32 and could be due to the small number of errors at this step. However, the types of error differed between the two periods, as shown in the examples in table 2. Further system improvements could focus on preventing configuration-related errors (eg, drop-down selection errors) and preventing wrong patient weights and dosages from being entered, as suggested in a previous publication on the system’s usability.27
Furthermore, 19 out of 133 error reports in the pre-implementation period could have been prevented by adopting a CPOE, whereas 4 out of 109 error reports in post implementation could have been prevented by improving the CPOE’s configuration. This finding suggests that the CPOE managed to prevent medication errors and led to very few technology-induced detrimental effects. The low number of technology-generated errors could be related to the application of techniques used in successful HIT implementations, such as the involvement of prescribers in the system design and the training of colleagues, the modification of the CPOE in response to feedback and the direct observation of prescriber workflow.33
Improving risk assessment
The medication error rate in this study, which was 0.4 errors per 100 orders (242 per 56 189 orders) overall, was lower than reported in previous publications.32 However, a systematic review revealed that the prevalence of medication errors among paediatric inpatients is highly variable.32 The lower error rate could be attributed to the hospital’s safety culture,34 and by how medication errors were defined and captured. The combination of multiple data collection methods at different point of medication management process (eg, ordering, administration) can be helpful to assess the prevalence of medication errors fully. However, this approach is resource and time intensive. Developing a more proactive and data-driven system could provide a more accurate risk assessment35 and inform future system optimisation. Furthermore, the use of an event reporting system integrated with the other systems could also improve the quality of the data found in these reports (eg, less missing data, typing errors).
Improving error reporting recommendations
Declarants for most reports proposed only person-based recommendations, which are generally less effective in the long term.21 The declarants proposed very few system-based approaches, which could help further reduce the rate of errors.36 When reviewing the events description through an HIT lens, the most frequent potential prevention strategy was improving the eMAR’s usability to ensure timely administration of medications. The Institute of Safe Medication Practices guidelines recommend ‘changing the appearance of a medication entry for delayed doses in eMARs, setting different time limits for the removal of scheduled medications from automated dispensing cabinets, highlighting time-critical scheduled medications on eMARs, differentiating between first doses and subsequent scheduled doses, displaying alerts to show doses that will soon be overdue or that have been omitted’.37 As of September 2021, the hospital has been working on the implementation of a feature for the nursing dashboard that would highlight delayed medication administration. Previous studies have also suggested that other potential prevention strategies, such as bar code medication administration, CDSS and intravenous interoperability systems, could also reduce medication errors.38–40
Limitations
This study has a few limitations. First, this study was based on data from error reporting, which is mandatory only when there are patient consequences. Therefore, although stable through the study period, the prevalence of medication errors is likely higher than reported in this study. Indeed, the low error rate could be attributed to on-unit pharmacists reviewing the medication regimen during ordering, thus preventing errors from reaching the patient. Additionally, this study was conducted in only one pilot unit. However, the general paediatric unit represents the largest unit in the hospital, and the results from this study will serve to inform the implementation process in other units. Furthermore, it is possible that there were fewer error reports in the first month following the CPOE implementation due to the constant support from the implementation team.
Conclusion
This study highlights the importance of improving the usability of the eMAR to further reduce the number of medication errors. Error reports could be improved by developing a proactive event reporting system and system-based recommendations.
Data availability statement
No data are available.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by CHU Sainte-Justine Research Ethics Committee (REB number: 2018-1915). Participants gave informed consent to participate in the study before taking part.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @Manqing_L, @maxthibault
Contributors MQL, MT, PJ and AM conceptualised the study and developed the methodology. MQL, MT and M-PM collected the data. MQL, MT and TS conducted the formal analysis. MQL wrote the original draft and acted as guarantor. MT, PJ, M-PM and AM reviewed and edited the draft. PJ, DL and AM acquired the funding and supervised the project.
Funding This work was supported by a grant from MEDTEQ’s Fonds de soutien à l'innovation en santé et en services sociaux (FSISSS; grant number: 2018-2-04) and was conducted as part of a graduate degree research project, which was supported by a scholarship from the Canadian Institutes of Health Research (CIHR).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.