Original Research•

Clinician preferences for computerised clinical decision support for medications in primary care: a focus group study

Abstract

Background To improve user-centred design efforts and efficiency; there is a need to disseminate information on modern day clinician preferences for technologies such as computerised clinical decision support (CDS).

Objective To describe clinician perceptions regarding beneficial features of CDS for chronic medications in primary care.

Methods This study included focus groups and clinicians individually describing their ideal CDS. Three focus groups were conducted including prescribing clinicians from a variety of disciplines. Outcome measures included identification of favourable features and unintended consequences of CDS for chronic medication management in primary care. We transcribed recordings, performed thematic qualitative analysis and generated counts when possible.

Results There were 21 participants who identified four categories of beneficial CDS features during the group discussion: non-interruptive alerts, clinically relevant and customisable support, presentation of pertinent clinical information and optimises workflow. Non-interruptive alerts were broadly defined as passive alerts that a user chooses to review, whereas interruptive were active or disruptive alerts that interrupted workflow and one is forced to review before completing a task. The CDS features identified in the individual descriptions were consistent with the focus group discussion, with the exception of non-interruptive alerts. In the individual descriptions, 12 clinicians preferred interruptive CDS compared with seven clinicians describing non-interruptive CDS.

Conclusion Clinicians identified CDS for chronic medications beneficial when they are clinically relevant and customisable, present pertinent clinical information (eg, labs, vitals) and improve their workflow. Although clinicians preferred passive, non-interruptive alerts, most acknowledged that these may not be widely seen and may be less effective. These features align with literature describing best practices in CDS design and emphasise those features clinicians prioritise, which should be considered when designing CDS for medication management in primary care. These findings highlight the disparity between the current state of CDS design and clinician-stated design features associated with beneficial CDS.

Summary

What is already known?

  • There are best practices in clinical decision support (CDS) design that increase likelihood of CDS to be successful.

  • End user input in CDS design increases the likelihood of CDS to be successful.

What does this paper add?

  • Clinicians identified beneficial design features of CDS for chronic medications that align with best practices in CDS design, including CDS that are clinically relevant and customisable, present pertinent clinical information and optimise workflow.

  • Clinicians prefer CDS that are not interruptive to their workflow, but may recognise interruptive CDS are more effective when designed well.

Introduction

Computerised clinical decision support systems (CDS) are intended to assist clinicians in clinical decision-making and thereby improve quality of healthcare provided.1 In light of the growing body of medical information and the difficulty of clinicians to truly stay up to date, CDS offers a powerful means to intelligently support clinicians in their clinical decision-making. Implementation of CDS across a variety of settings has led to improvements in patient care processes, healthcare costs, use of preventative medicine and adherence to standards of medical practice.2–4 Although CDS are widely used and have led to some positive outcomes, there are also numerous examples of CDS leading to no or negative changes in outcomes.5–8 One key reason for the mixed results is poor clinician adoption.

Clinician engagement or adoption is crucial to the effectiveness of CDS. To be engaged, clinicians must perceive CDS to be useful or beneficial. As stated by the National Academy of Medicine (formerly Institute of Medicine), technologies such as CDS should be designed to make it ‘easy to do the right thing’.9 To improve clinician adoption and the success of CDS, there are established best practice principles for CDS design, which includes tailoring CDS to the preferences of the end user and integration with their workflows.1 10 Incorporation of input from end users regarding CDS content, presentation and functionality is one important step to achieving this goal stated by the National Academy of Medicine and is referred to as user-centred or human-centred design.11 12

Unfortunately, the best practice principles for CDS design are inconsistently followed and there is limited literature describing preimplementation end user perceptions, needs or preferences of CDS for chronic medications in primary care. Several publications describe CDS features that improve end user engagement and outcomes and many of which have formed the basis for the best practice principles in CDS design.13–18 However, these findings are largely based on reviewing publications describing specific features of CDS that were successful or unsuccessful or focused on identification of barriers to using CDS. Further, CDS developers seeking to implement CDS have generally sought end user feedback on clinical content or validation in the form of usability testing following initial CDS design prototyping.19–22 Such input after prototyping introduces potential bias in end-user feedback of what the end product could be. While there are published examples of obtaining end-user input into the design of a paper-based CDS prior to prototyping,23 those results may not apply to electronic CDS workflows. Much of the literature and best practices in CDS design also represents findings from a time period when electronic health records (EHRs) and CDS were new or not a standard part of clinical workflow, which may not be relevant to today when EHRs and CDS are commonplace to clinical workflows.

Understanding the current day preferences and needs of clinicians for CDS targeting chronic medications in primary care and comparison to established best practices in CDS design can be used to optimise design and adoption by end users. This information is critical given inconsistent application of best practices in CDS design and the extensive resources invested in designing CDS despite numerous published examples of CDS that result in suboptimal clinical outcomes. Of CDS implemented, medication-focused CDS is one of the most common types.24 Here, we describe current clinician perceptions regarding beneficial features of CDS for chronic medications in primary care prior to prototyping and implementation as one component of a user-centred CDS design.

Methods

The purpose of the focus groups was to inform the optimal design of medication-related CDS for primary care within a large health system that includes academic, suburban, community and underserved settings. The health system cares for an estimated 3.5 million outpatients annually and has used one EHR platform (Epic) for the past 5 years. The Ottawa Decision Support Framework Needs Assessment25 and the grounded theory principle of saturation were used to conduct the focus group.26 To achieve saturation, we conducted three focus groups.27 28 To provide context, CDS for chronic heart failure (HF) medications in primary care was used as a case study during the focus groups. Chronic HF medications were chosen because there is a national need to improve prescribing to align with evidence-based recommendations29 and because these principles can be generalised to management of other chronic diseases in primary care.

Each focus group was planned to include five to 12 participants to facilitate effective discussion, which is aligned with best practices in focus group conduct.30 Eligible participants were prescribing clinicians from a variety of primary care practice settings whose workflow would be impacted by the CDS or who have pertinent clinical expertise (eg, cardiology). Clinical experts were invited to assist with thinking about more challenging clinical needs for the HF case study. The investigators identified participants by ongoing professional relationships and invited clinicians via email to participate based on clinician type (ie, physician, pharmacist, advanced practice provider), practice setting and clinical expertise to ensure broad representation.

The focus groups were 1.5 hours each and followed a moderator guide, led by an investigator (KET) with training and experience in conducting focus groups.30 Each session included five consecutive components: (1) overview of purpose and rationale for focus group and informed consent, (2) brief written clinician demographic questionnaire, (3) open-ended discussion of beneficial CDS features for chronic medication management in primary care, (4) participants describing in writing their ideal CDS for chronic medications on paper and (5) open-ended discussion of potential unintended consequences of the CDS. In the fifth component, individual descriptions of the ideal CDS served to validate the findings of the open discussion and evaluate features that resonated most with individuals, separate from influences from peers that can present in a group environment.

Focus groups were audio recorded and field notes were taken by an investigator (AGV) to identify key issues. Audio recordings were transcribed and clinicians de-identified prior to thematic analysis. Accuracy of the transcriptions was validated by an independent investigator (KET) who reviewed 20% of the transcription and by comparison with field notes. A thematic approach26 31 using ATLAS.ti software (V.7, Scientific Software Development GmbH) was used to analyse the transcripts by one investigator (JAN). The transcriptions were categorised into major themes iteratively using topic coding. Clinician demographics were also coded. Topic coding included inductive general categorisation of text from the transcripts into themes with some connection or pattern, with successive recoding for more specific subcategories.31 Throughout topic coding, interpretation or analytical conclusions were applied to assess the meaning of codes and emergent themes.31 A 20% sample of the coding was reviewed by a second independent investigator (KET) for validation.

The major categories identified in the open discussion of the focus groups were then used to evaluate the individual clinician written descriptions (component five). Characteristics of the individual descriptions were reviewed for consistencies and discrepancies of the categories identified in the open discussion of the focus groups. Counts were generated by tabulating instances of a written description describing categories discussed in the focus group and noting new themes. The written descriptions were reviewed by one clinical pharmacist-investigator (KET) with expertise in chronic medication management and CDS design. The focus group participants provided informed consent.

Results

Twenty-one clinicians participated in the focus groups, including 11 primary care physicians, four advanced practice providers, four pharmacists and two cardiologists. The individual focus groups consisted of five, six and 10 participants each. Primary care clinician subspecialties included geriatrics, internal medicine and family medicine (table 1).

Table 1
|
Clinician characteristics

Open-ended discussion of beneficial features

Clinicians described four main beneficial features during the open discussion: (1) non-interruptive alerts; (2) clinically relevant and customisable support; (3) summarisation of pertinent clinical information and (4) improving workflow.

Non-interruptive alerts

Clinicians overwhelmingly asserted they did not want interruptive or active CDS alerts because they interrupted clinician workflow. The term non-interruptive alerts used by participants was broadly defined as passive alerts that a user chooses to review, whereas interruptive was defined as active or disruptive alerts that interrupted workflow and one is forced to review before completing a task.32 33 Each focus group deliberated on this point for a prolonged period of time compared with discussion of other CDS features. Clinicians repeatedly reported the alerts were “one more thing to get through” to complete a task, reporting interruptive alerts to be a barrier to completing their tasks. One clinician stated ‘…[the alert] not only disrupts your flow but it actually paralyzes you, that I think is the worst of all’.

Clinicians reported ‘alert fatigue’, referred to interruptive alerts as an ‘annoyance’ and stated an overwhelming number of alerts for ‘every patient’. They stated the alerts were not helpful, ‘tell you the obvious’ and often are redundant or irrelevant. Multiple clinicians stated the interruptive alerts were not ‘smart enough’ to recognise that a given task was already complete and sometimes fired multiple times after a task was completed. Some notable ‘irrelevant’ examples included a lactation warning for a 60-year-old patient and an outdated alert regarding foreign travel and exposure to the Ebola virus. Irrelevance was also noted when an alert was out of context with the reason for visit or clinician type.

Despite overwhelmingly negative attitudes towards interruptive alerts, clinicians in one of the three focus groups expressed some positive benefits of interruptive alerts, being careful to emphasise the infrequency of helpfulness. The clinicians reported a reminder alert was ‘sometimes’ helpful and one clinician stated ‘every once in a while I am brainlessly doing something and it pops up and I’m like, oh, oh, that’s helpful’. While not universally endorsed, some participants indicated they liked one interruptive alert for lung cancer screening and found it helpful, because it interrupted workflow at the right time and ‘it does the work for me’, including ability to place the order.

There was no consensus regarding the most appropriate timing of an interruptive alert. Roughly equal numbers of clinicians preferred the CDS to alert at different times such as: (1) the first opening of an encounter; (2) when ordering a medication or reviewing medications, (3) entering the patient’s visit diagnosis or (4) at the end of the encounter. Proponents of alerting on first opening the encounter wanted the information to inform their visit agenda, while opponents stated it was premature. Proponents of alerting on ordering a medication or entering a diagnosis believed it would improve context, whereas opponents expressed this would be too late because ordering tasks are often completed at the conclusion of the visit after a plan was already established. Proponents of alerts appearing at the end of the visit argued the alert serves as a double check while minimising workflow interruptions, whereas opponents stated this was too late. There was also concern that in situations involving medical residents, the attending physician might not see or receive the alert until after the patient was dismissed/discharged.

In lieu of interruptive alerts, clinicians indicated a strong preference for passive CDS. They want ‘a gentle reminder that doesn’t actually interrupt you’, or ‘just sort of there and available’. Many clinicians desired CDS they could access in a way that fits into their workflow. A number of clinicians desired CDS accessible before a patient’s visit and prior to opening an encounter, stating this would be more helpful in planning the visit. Clinicians stated they want summary information in the form of clinical dashboards and summative or snapshot screens. One clinician stated if CDS were ‘…more pro-active, preventative,…something that is not in that moment where you’re trying to just take care of the patient acutely, we could do more outreach and prevention’. Clinicians also expressed favourable views of checklists that could ‘default to fill in the [information] that the system already knew’.

All clinicians from a single focus group asserted that passive alerts would be used more frequently and be more effective than interruptive alerts, whereas clinicians in the other two focus groups felt passive alerts would be less widely seen and therefore less effective. Proponents of interruptive alerts noted they must be properly designed and implemented to be effective and minimise alert fatigue.

Clinically relevant and customisable support

Clinicians wanted the option to make the CDS smarter, informing the CDS when to alert rather than interruptive CDS alerting all clinicians in the same way for all patients. Clinicians reported CDS might be better accepted if the CDS could be temporarily dismissed. One clinician stated, ‘if the patient is going through an acute thing and you don’t want to address [the interruptive alert] right now, say remind me in ninety days or something like that, I think would be helpful’. Clinicians also looked favourably on CDS with response options allowing them to permanently disable CDS for a specific patient whom they felt the recommendation would never be appropriate. However, clinicians wanted the option to explain why it is not appropriate so they can access these responses in the future as a means of documentation. Clinicians also wanted more flexibility of how to respond to CDS, stating ‘I always want to do something else’, other than the options given to select. Some clinicians suggested having an ‘other’ response option that allowed them to explain and prevent them from feeling forced to respond in a given way.

To minimise instances when the CDS is considered irrelevant, they desired the CDS to be patient specific and provide assistance only in the appropriate context, considering the setting and clinician. Clinicians expressed the desire to tailor the CDS based on the reason for visit. Many indicated they do not want to see a chronic-medication CDS alert for patients who are not assigned to their patient panel or is being seen for an acute reason. In addition, clinicians commented several times that they ‘know the patient better than the computer’, and want CDS to be smarter and patient specific, not disease specific.

Presentation of pertinent clinical information

Participants wanted the CDS to present pertinent patient-specific information to inform their decision, such as pertinent laboratory results, vital signs, and drug allergy information. Having this information ‘saves you from going to Chart Review and pulling them out’. Participants reported having dates for information such as vitals or labs as a reminder to order new tests, if warranted, and to assess clinically meaningful trends, such as serum creatinine changes for acute kidney injury.

For CDS recommendations that are accepted, clinicians indicated they wanted additional information: information regarding monitoring would serve as a reminder and information on medication costs could improve selection of more affordable medications. Clinicians thought it would be helpful for the CDS to include actual or general costs of medications when making recommendations, which could improve patient adherence and avoid pharmacy requests for less expensive alternatives.

Optimisation of workflow

Clinicians discussed the role of clinic staff filtering CDS that would otherwise be presented to the clinician. They suggested that clinic staff could handle some of the interruptive CDS while preparing the patient for the clinician’s visit, such as during medication reconciliation. Staff responses to CDS would then inform and streamline the CDS presented to clinicians. One clinician stated clinic staff could do ‘non-physician work that would free up physicians to do some of the higher level patient care management that we just sometimes don’t get to’.

Clinicians also requested easy access to calculations otherwise routinely done manually, such as daily morphine-equivalent doses and 10-year cardiovascular risk estimates. Participants emphasised that information be easy to find. It was recognised that some routine calculations, such as 10-year cardiovascular risk estimates, are available in existing EHRs, but the information was not conveniently located or it was challenging to recall the steps to access them.

Clinicians expressed interest in CDS that would populate text in their clinical documentation or patient instructions to make documentation easier. Clinical documentation and the patient instruction sections of the EHR encounter would automatically be populated to reflect the plan, counselling points and necessary follow-up including labs if they accepted a CDS recommendation to order a new medication for a patient. For example, accepting a CDS recommendation to start spironolactone would lead to standardised text populated in the clinical documentation reflecting the plan to start spironolactone, monitor pertinent labs and include patient instructions. Clinicians specifically expressed interest in CDS that would reduce clinician documentation burden by generating patient instructions, especially for medications requiring titration, with timing of monitoring and common adverse effects.

Clinicians also wanted the ability to take action within the CDS with the ‘least amount of clicks’ to make it easier to follow through with an accepted CDS recommendation. One clinician suggested a CDS recommendation ‘…could just say ‘would you like to order it?’ and you click ‘yes’ and boom, it just happens’ instead of needing to leave the current screen and enter the orders module. One clinician stated ‘if you save the clinician time, you'll always win’.

Key components of individual CDS written descriptions

When asked to independently describe their ideal, medication-related CDS, clinicians generally incorporated features discussed during the group discussion (table 2). Most clinicians drew figures of the ideal CDS supplemented with written descriptions. Despite the sentiment of the group discussion, 12 clinicians described their ideal CDS as interruptive compared with seven who described a passive CDS. Of note, some clinicians drew more than one CDS and some expressed wanting both interruptive and passive CDS, whereas some clinicians did not clearly articulate whether the CDS they drew was interruptive or passive. Features most commonly incorporated into the descriptions were: (1) the CDS being patient specific (n=15 clinicians), (2) inclusion of pertinent summary information, including current medications (n=16 clinicians) and diagnostics (n=12 clinicians), (3) actionable links to place orders (n=13 clinicians) and (4) the ability to permanently disable or defer the alert to a later time (n=10 clinicians). Although six clinicians incorporated cost in their descriptions, two articulated they wanted it as optional information they could access on demand instead of having it presented on the CDS user interface.

Table 2
|
Features of a medication-related clinical decision support (CDS) described individually by clinicians (N=21 clinicians)

Unintended consequences

When asked about potential unintended consequences of CDS for a chronic medication, clinician concerns were related to alert fatigue and blindly following the CDS without appropriate knowledge, and/or incomplete or inaccurate information.

One clinician stated, ‘in a lot of ways we are creating clinicians that don’t necessarily know pathophysiology and patient assessment and what they are learning is algorithms for treating patients and best practice alerts that tell them what to do’. This sentiment was not only in reference to clinicians in general but also with particular reference to medical residents. The clinicians acknowledged that uniform acceptance of the CDS recommendations ‘can lead to a lot of harm’, especially if the recommendation was based on inaccurate or incomplete information. It was believed this has a negative effect on their rapport with the patient. To prevent the thoughtless acceptance of CDS recommendations, participants indicated a need for recommendations to include situations where the recommendations should not be accepted and links to supporting evidence.

Participants also reported situations where alert fatigue has led to negative health outcomes. They expressed need for CDS, especially interruptive alerts, to be systematically evaluated for efficacy and end user use to facilitate needed revisions or discontinue them.

Discussion

This study reinforces the modern day relevance of the best practice principles in CDS design to CDS for chronic medications in primary care and the need to apply these principles to realise the benefits of CDS, which are pervasive in healthcare, yet do not currently produce consistently positive outcomes. We identified four main categories of CDS features for chronic medication management that clinicians find beneficial: clinically relevant and customisable, presentation of pertinent information and optimisation of workflow. These categories of features align with the best practices in CDS design and highlight aspects that clinicians perceive to be more or less beneficial, which should be prioritised when designing and implementing future CDS for chronic medications in primary care.

Interestingly, focus group participants expressed strong dislike for interruptive CDS alerts during the open group discussion, which was the most common CDS suggested when individuals were asked to design the ideal CDS. This discrepancy may be due to effects of peer influence during the group discussion or realisation that interruptive alerts were helpful when designed well and that non-interruptive alerts are more likely to be overlooked or ignored. The discussion of whether the alerts should be interruptive or not aligns with literature supporting improved use and effectiveness of interruptive CDS16 22 34 35 and non-interruptive CDS being less likely to be viewed and therefore less likely to be effective.16 22 24 36 37 However, whether an alert is interruptive or not is only one feature of CDS and as the focus group conversation progressed, clinicians discussed other features of CDS that would improve usefulness of an interruptive alert, such as timing of the alert and the ability to customise the alert. These features may have led individual clinicians to favour interruptive alerts if the design incorporated these features. Although not explicitly discussed during the focus groups, the sentiment regarding passive versus active CDS highlights the importance of potentially reserving active CDS for high priority or high-risk patient care situations13 38–40 and carefully designing active CDS such that the other features are optimised to minimise clinician frustrations.

Although there was no consensus in the timing of alerts, the discussion supports efforts to accommodate variations in individual clinician workflow and to personalise CDS to clinical workflows. However, to date, CDS have not been personalised to individual workflows, given the complexity required to account for the highly variable clinical workflows. Future research is needed to evaluate feasibility of such personalisation to workflows. For non-interruptive alerts, personalisation could include designing the CDS such that clinicians can access the information in the alert at a variety of different time points (eg, before entering the patient chart and/or on order entry and/or when signing a note), whereas such an approach with interruptive alerts would increase the frequency of interruptions and augment alert fatigue. Thus, for interruptive alerts, individualised personalisation of the alert timing by the clinician to meet their workflow needs would be the ideal state, but has not been realised yet. The ideal timing of passive or active CDS facilitates clinicians using the CDS without them noticing it, which can be challenging to accomplish in many situations, but nonetheless should be the goal in CDS design. Often personalisation is limited by technical constraints of the CDS software or EHR, which is continually evolving and may be less of a barrier with time and creative solutions.

To improve relevance and optimise their workflow, clinicians want CDS to be smarter, synthesising and presenting pertinent patient-specific information, with actionable recommendations that generates automated text in their clinical documentation and patient instructions. These findings are consistent with published recommendations on the ‘grand challenges’ that must be overcome for CDS to reach their potential and positively impact healthcare.39 Summarising pertinent information such as labs, vitals and allergies makes it easier for clinicians to quickly evaluate the appropriateness of a CDS recommendation and actionable links, such as ordering medications or labs, makes it easy to take action immediately. Further, the ability to populate clinical documentation and patient instructions with the plan would decrease duplicate work. Clinicians were specifically interested in auto-population of patient instructions to include titration schedules, timing of lab monitoring and pertinent adverse effects. Although not specifically mentioned in the focus groups, the clinical documentation could include routine monitoring that would assist in planning follow-up care. Some clinicians also expressed interest in presenting medication cost information with CDS recommendations. Although there are efforts to integrate medication costs into EHRs and CDS,41 there is currently no published literature describing the efficacy of such efforts. However, in other situations, such as ordering laboratory tests, clinicians did change ordering behaviour when informed of associated costs,42–45 suggesting the same may be true with medication ordering.

Another desired feature of the CDS was flexibility in response options, which is aligned with best practice principles in user-interface design. At times, clinicians felt forced to answer in a way not aligned with their intentions or the clinical scenario and wanted the option to better explain their actions in the form of free text that could be viewed later to trigger recall of their decisions. They also strongly wanted the option to permanently disable or temporarily dismiss CDS to alert at a more opportune time of their choosing. Clinicians who believed CDS would improve usefulness felt the ability to customise the CDS to the clinical scenario and individual workflow was very important. Principles of user-interface design includes ensuring clear and actionable response options,13 but flexibility in options is not well emphasised in the literature and can be critical to support the often nuanced clinical scenarios.

Clinicians had some concerns about unintended consequences, especially related to alert fatigue, mindlessly following CDS and inaccurate or incomplete information being presented by CDS. To avoid these problems, clinicians suggested implementing systematic processes for evaluation of CDS and inclusion of optional information and references. While inclusion of references in CDS are key to building trust, there are many additional strategies recommended to cultivate trust.13 Prioritising CDS recommendations according to specificity, urgency and relevance minimises obtrusiveness and increases trust.13 As CDS becomes increasingly pervasive in healthcare and we drive towards a Learning Health System, provision of a rationale with an assessment of the certainty or quality of the recommendation is important to engender trust.46 Further, as CDS are modified in response to clinician feedback, it is important to monitor for unintended consequences. For example, converting all CDS to be passive or giving clinicians the ability to permanently ‘snooze’ all CDS could result in missed opportunities to optimise evidence-based care for patients.

The findings of this study are limited by generalisability, given that participants represent clinicians at a single large academic medical centre using one integrated EHR platform for over 5 years and are exposed routinely to CDS. The results may be less representative of clinicians with less experience using an EHR and CDS, or using a different EHR platform, despite conversations focusing on broad concepts related to CDS that were not specific to an individual EHR. Further, participants represent a convenience sample based on ongoing professional relationships with the study investigators, thus introducing selection bias. However, a notable strength is that participants represent a variety of disciplines across outpatient practice settings and various expertise, with varying degrees of clinical practice responsibilities.

Clinicians characterised CDS for chronic medications as beneficial when it is clinically relevant and customisable, presents pertinent clinical information (eg, labs, vitals) and optimises workflow. Although clinicians preferred passive, non-interruptive alerts, most acknowledged that these may not be widely seen and may be less effective. The design features align with literature describing best practices in CDS design and emphasise features that primary care providers prioritise when using CDS for chronic medication use. Despite awareness of these best practices in CDS design, many CDS are not designed with application of these best practices, which is thought to be one reason for the mixed outcomes of CDS to date.14 19 47 48 This study reinforces the modern day relevance of the best practice principles in CDS design to CDS for chronic medications in primary care. When designing CDS for chronic medications in primary care, developers should consider these user-centred design features and continually re-evaluate CDS design as technical capabilities of CDS and EHRs become more sophisticated.

Overview

Article metrics
Altmetricopen-url
Dimensionsopen-url