Review article
Factors influencing implementation success of guideline-based clinical decision support systems: A systematic review and gaps analysis

https://doi.org/10.1016/j.ijmedinf.2016.12.001Get rights and content

Highlights

  • A synopsis of evidence on the impact of factors impeding or facilitating implementations of guideline-based CDSS used by physicians is absent from the scientific literature.

  • This systematic literature review provides an integrated understanding of these factors, based on a socio-technical framework.

  • Research on factors influencing CDSS implementations has mainly focused on human and technology factors and less on organizational factors.

  • Future research within the field of guideline-based CDSS should focus on evaluating implementations through the use of socio-technical models.

Abstract

Objective

To provide an integrated and differentiated understanding of factors influencing guideline-based CDSS implementation and illustrate the gaps in the current literature.

Materials and methods

A systematic literature review and in-depth exploration of factors impeding or facilitating successful implementation of guideline-based CDSS supporting physicians in clinical decision-making was performed. Factors were identified thematically by textual analysis of the included publications and were individually mapped to the human, organization and technology-fit (HOT-fit) framework for evaluating implementations of health information systems.

Results

A total of 421 factors were found in 35 included publications from a total of 3676 publications. The mapping of factors concerning CDSS implementation on the HOT-fit framework revealed gaps in each domain of the framework and showed that research has mainly focused on human and technology factors and less on organizational factors.

Conclusions

Future research within the field of guideline-based CDSS should focus on evaluating implementations through the use of socio-technical models to study guideline-based CDSS system implementations from a multidimensional view. Furthermore, research is needed to explore whether use of these models during the planning phases of a CDSS project is useful in anticipating and preventing implementation barriers from occurring and exploiting facilitators to a successful implementation of the system.

Introduction

Evidence-based guidelines are developed and implemented to guide and support physician’ clinical decision making in improving quality of care. However, evidence exists that shows that paper-based clinical guidelines are still underutilized in practice [1], [2]. A Clinical Decision Support System (CDSS) can offer physicians patient-specific advice based on guideline recommendations, thereby overcoming obstacles in the use of traditional paper-based guidelines and improve physicians’ adherence to recommendations. [3], [4]. Yet, even though the evidence of CDSSs improving clinical performance and patient outcomes is convincing, the failure rate in introducing CDSS in clinical practices is still over 50 percent [5].

Introducing a CDSS seems fraught with obstacles among which low ease of system use [6], negative end-user attitudes towards the system and negative impact on clinical workflows [5]. Studies evaluating CDSS implementation in clinical care continue to provide insight into factors influencing or issues surrounding CDSS introduction [7], [8]. However, these insights are revealed and discussed from various perspectives [9], among which the perspective of the CDSS technology, the user’s experience with CDSS and cultural and management issues within the healthcare organization. But while evidence on the impact of factors is wide-spread, a synopsis is absent. Building on the knowledge base of evaluation studies of Health Information systems (HIS), Yusof et al. proposed a framework to evaluate HIS while incorporating the concept of fit between Human, Organization and Technology (HOT-fit) [10]. The concept of fit focuses on the alignment between and compatibility of the human, technology and organization. In doing so, the HOT-fit framework provides an excellent framework to report on factors influencing CDSS implementation from these three perspectives.

In a previous conference paper, we already report on a literature study about factors associated with CDSS implementations used by physicians [11]. This paper extents the previous literature review with an in-depth exploration of factors associated with a successful implementation of guideline-based CDSS. We performed a mixed method research synthesis [12], focusing on factors revealed by both quantitative and qualitative evaluation studies of guideline-based CDSS. By combining evidence from quantitative, qualitative and mixed studies we aim to provide a more integrated and differentiated understanding of implementation factors influencing guideline-based CDSS implementation success and, in doing so, illustrate the gaps in the current literature. We extracted these from the studies through a systematic approach and mapped them to the HOT-fit framework to provide a more integrated and differentiated understanding of factors impeding or facilitating CDSS implementation success. This research synthesis ultimately aims to support software teams in development and implementation of guideline-based CDSS.

Section snippets

HOT-fit

The human, organization and technology-fit (HOT-fit) framework aims to assist researchers in conducting thorough evaluation studies of HIS. The HOT-fit framework can be used as a reference model for evaluating the performance, effectiveness and impact of HIS in a rigorous, systematic and continuous manner. The HOT-fit framework is based on earlier models for evaluating information systems (IS); the IS Success Model by DeLone and McLean [13] and Information Technology (IT)-Organization Fit Model

Search results

The literature search generated a total of 3676 publications. After removing the duplicates and reviewing the abstracts, 556 publications were selected for full text review. The measured Cohen's Kappa for the inter-observer agreement between reviewers was 0.754, indicating good agreement [18]. Finally, 35 publications were found eligible for inclusion. Fig. 2 shows the study’s search flow and an overview of the included publications can be found in Supplemental Table S1. Most studies used

Main findings

This literature synthesis provides an in-depth exploration of factors facilitating or impeding implementation of guideline-based CDSS. Previous researchers have advocated that evaluation studies of CDSS should be performed from multiple perspectives [8], [52], [53], [54]. Our mixed studies research synthesis revealed that evaluation studies of guideline-based CDSS have mainly focused on Technological and Human factors, reflected in the high number of factors reported in these domains (210 and

Conflicts of interest

The authors declare that they have no financial or personal relationships that could influence this work.

Authors’ contributions

The authors declare that this manuscript is original, has not been published before and is not currently being considered for publication elsewhere. The manuscript has been approved by all authors. There are no other persons who satisfied the criteria for authorship. We confirm that the order of authors listed in the manuscript has been approved by all authors. The corresponding author will act as the sole contact for the editorial process and we understand that she is responsible for

References (65)

  • M. Peleg

    Using multi-perspective methodologies to study users' interactions with the prototype front end of a guideline-based decision support system for diabetic foot care

    Int. J. Med. Inform.

    (2009)
  • A.C. Li

    Integrating usability testing and think-aloud protocol analysis with near-live clinical simulations in evaluating clinical decision support

    Int. J. Med. Inform.

    (2012)
  • T.D. Sequist

    A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease

    J. Am. Med. Inform. Assoc.

    (2005)
  • A. Berlin et al.

    A taxonomic description of computer-based clinical decision support systems

    J. Biomed. Inform.

    (2006)
  • P. Carayon

    Human factors systems approach to healthcare quality and patient safety

    Appl. Ergon.

    (2014)
  • A.R. Gagliardi

    How can we improve guideline use? A conceptual framework of implementability

    Implement. Sci.

    (2011)
  • M.W. Jaspers

    Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings

    J. Am. Med. Inform. Assoc.

    (2011)
  • M. Prior et al.

    The effectiveness of clinical guideline implementation strategies—a synthesis of systematic review findings

    J. Eval. Clin. Pract.

    (2008)
  • M.H. Trivedi

    Development and implementation of computerized clinical guidelines: barriers and solutions

    Methods Inf. Med.

    (2002)
  • H. Varonen et al.

    What may help or hinder the implementation of computerized decision support systems (CDSSs): a focus group study with physicians

    Fam. Pract.

    (2008)
  • H. Shah

    Requirements for guidelines systems: implementation challenges and lessons from existing software-engineering efforts

    BMC Med. Inform. Decis. Mak.

    (2012)
  • M.P. Gagnon

    Systematic review of factors influencing the adoption of information and communication technologies by healthcare professionals

    J. Med. Syst.

    (2012)
  • S.A. Mohammed et al.

    Towards an evaluation framework for information quality management (IQM) practices for health information systems-evaluation criteria for effective IQM practices

    J. Eval. Clin. Pract.

    (2013)
  • E. Kilsdonk

    Factors known to influence acceptance of clinical decision support systems

    Stud. Health Technol. Inform.

    (2011)
  • M. Heyvaert et al.

    Mixed methods research synthesis: definition, framework, and potential

    Qual. Quant.

    (2013)
  • W.H. Delone et al.

    The DeLone and McLean model of information systems success: a ten-year update

    J. Manage. Inf. Syst.

    (2003)
  • M.S.S. Morton

    The Corporation of the 1990

    (1991)
  • D. O'Sullivan

    Decision time for clinical decision support systems

    Clin. Med. (Lond.)

    (2014)
  • N. Mays et al.

    Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field

    J. Health Serv. Res. Policy

    (2005)
  • J.R. Landis et al.

    The measurement of observer agreement for categorical data

    Biometrics

    (1977)
  • J.M. Langton

    How do medical doctors use a web-based oncology protocol system? A comparison of Australian doctors at different levels of medical training using logfile analysis and an online survey

    BMC Med. Inform. Decis. Mak.

    (2013)
  • D. Buenestado

    Evaluating acceptance and user experience of a guideline-based clinical decision support system execution platform

    J. Med. Syst.

    (2013)
  • Cited by (0)

    View full text