National questionnaire study on clinical ICT systems proofs: Physicians suffer from poor usability

Abstract

Objective

In the health informatics field, usability studies typically focus on evaluating a single information system and involve a rather small group of end-users. However, little is known about the usability of clinical information and communication technology (ICT) environment in which healthcare professionals work daily. This paper aims at contributing to usability research and user-oriented development of healthcare technologies with three objectives: inform researchers and practitioners about the current state of usability of clinical ICT systems, increase the understanding of usability aspects specific for clinical context, and encourage a more holistic approach on studying usability issues in health informatics field.

Methods

A national web questionnaire study was conducted in Finland in spring 2010 with 3929 physicians actively working in patient care. For the purposes of the study, we described three dimensions of clinical ICT system usability that reflect the physicians’ viewpoint on system usage: (1) compatibility between clinical ICT systems and physicians’ tasks, (2) ICT support for information exchange, communication and collaboration in clinical work, and (3) interoperability and reliability. The dimensions derive from the definitions of usability and clinical context of use analysis, and reflect the ability of ICT systems to have a positive impact on patient care by supporting physicians in achieving their goals with a pleasant user experience. The research data incorporated 32 statements with a five-point Likert-scale on physicians’ experiences on usability of their currently used ICT systems and a summative question about school grade given to electronic health record (EHR) systems.

Results

Physicians’ estimates of their EHR systems were very critical. With the rating scale from 4 or fail to 10 or excellent, the average of the grades varied from 6.1 to 8.4 dependent on the kind of facility the physician is working. Questionnaire results indicated several usability problems and deficiencies which considerably hindered the efficiency of clinical ICT use and physician's routine work. Systems lacked the appropriate features to support typical clinical tasks, such as decision making, prevention of medical errors, and review of a patient's treatment chart. The systems also required physicians to perform fixed sequences of steps and tasks, and poorly supported the documentation and retrieval of patient data. The findings on ICT support for collaboration showed mainly negative results, aside from collaboration between co-located physicians. In addition, the study results pointed out physicians suffering from system failures and a lack of integration between the systems.

Conclusions

The described study and related results are unique in several ways. A national usability study with nearly 4000 respondents had not been conducted in other countries in which healthcare technologies are widely adopted. The questionnaire study provided a generalized picture about the usability problems, however, it should be noted that there were significant differences between legacy systems in use. Previously, researchers had not approached contextual aspects of usability the context of clinical work, where numerous systems are in use. The described usability dimensions and the presented study results can be considered as the first step towards conceptualizing ICT usability in the unique setting of clinical work.

Introduction

Today, hundreds of information and communication technology (ICT) systems are used in healthcare organisations to serve physicians and other professionals in their daily work with patients. These systems cover a broad range of applications, from widely used electronic health records (EHR) and computerised physician order entry (CPOE) systems, to modern speech recognition technologies and mobile applications. Among other industries, healthcare has already profited extensively by the development of ICT. Although the technology-related benefits are obvious in theory, it seems that they are not clearly associated in the operating situations nor appeared in healthcare professionals’ daily work. Indeed, what is known about the usability of the healthcare technology environment – the jungle of electronic systems and applications in which physicians, nurses and other professionals work daily?

User-oriented research in the health informatics field has tended to focus on ICT adaptation and user satisfaction issues, rather than on usability. Since the early 2000s, however, there has been growing interest towards usability and human-centred design issues. Typically, researchers have applied traditional usability evaluation methods [1], [2] and focussed on healthcare information systems, particularly on EHRs, and their use in clinical settings (e.g. [3], [4], [5], [6]). Additionally, several articles have aimed at promoting the adaptation of the evaluation approach by describing instructions for usability activities in healthcare settings (e.g. [7], [8], [9]).

The motivation for usability studies derives from empirical study results as well as worrying statements about failures in technology development and adaptation (e.g. presented by professors Heeks [10] and Haux [11]). The results from recent studies suggest that currently used healthcare information systems suffer from a high number of usability flaws which considerably hinder the use of computer systems. The evidence is strong: an EHR system has been reported with 103 flaws related to the complexity of information, a poor relation to work activities, and lack of support for mobility [3]; a commercial EHR in a large paediatric hospital has been reported with 134 flaws related to consistency, user control, flexibility, efficiency, and natural dialogue [4]; a CPOE system has been reported with 33 flaws related to user interface and user interaction issues [5]; and a handheld prescription writing application has been reported with 73 flaws related to interface design [6].

Despite the increasing trend towards usability issues, relatively little systematic data has been gathered on the usability of the clinical technology environment. Generally speaking, the focus of usability research seems to be heavily affected by the traditional approach of human–computer interaction evaluation. As an illustration of this, several evaluation studies share the following characteristics in common: they focus on a single healthcare information system already in use, apply traditional usability evaluation methods (user testing or usability inspections), are conducted in a specified use context, and involve a rather small number of end-users (typically healthcare professionals within the same area of expertise). This observation also applies to those few reported usability questionnaire studies used to evaluate hospital information technology (IT) systems [12], [13], [14].

It is easy to argue that more research is needed to investigate usability of ICT systems and the impacts current systems have on clinical work. The ISO 9241-210 standard describes the objectives of designing systems for usability as follows: Usable systems can provide a number of benefits, including improved productivity, enhanced user well-being, avoidance of stress, increased accessibility and reduced risk of harm [15]. When reflected on in the domain of health informatics, the following question can be raised to address the success of ICT development and adaptation: Do clinical ICT systems support professionals’ operative work? Are professionals able to conduct their work in an efficient and satisfactory way using these systems? From the viewpoint of healthcare professionals, research on the usability of a single system is somehow contradictory when considering their daily work and surroundings. The technology environment in clinical work consists of tens of ICT systems, of which several are used simultaneously (e.g. the process of electronic prescribing using EHR and other ICT systems). With the end-users’ perspective in mind, it is easy to argue that research should address the usability of ICT systems in clinical contexts from a broader viewpoint: consider applications as integrated parts of the technology environment and describe the objectives of usability considerations with regards to user's tasks and goals while utilizing numerous ICT systems in their daily work.

It seems that no empirical research has addressed the usability of clinical ICT systems from such perspective. Further, in health informatics literature, the definition for usability presented by the ISO standard [15], [16] is often referred to; however, no specific clarification or definition has been presented on the concept of the usability of healthcare ICT. For instance, the HIMMS 2009 report describes nine principles of EHR user interface design with references to the ISO standard [16], and emphasises efficiency, effectiveness, safety, user satisfaction and cognitive workload attributes [17]. Likewise, a consultative report by Schumacher et al. [18] refers to the ISO stating that the usability goals for an EHR system must be set by specifying target values for effectiveness, efficiency, and satisfaction. Additionally, in their paper on the contextual nature of usability, Svanæs et al. [19] illustrate the context of use in healthcare settings with help of three case descriptions and highlight the importance of taking into consideration the contextual nature of usability. Based on these findings, one can argue that the contextual nature and the applicability of the definition for the health informatics domain tend to be poorly understood and inadequately addressed by researchers and practitioners.