Review
Improving quality of care. A systematic review on how medical registries provide information feedback to health care providers

https://doi.org/10.1016/j.ijmedinf.2010.01.011Get rights and content

Abstract

Objective

To determine (1) how medical registries provide information feedback to health care professionals, (2) whether this feedback has any effect on the quality of care and (3) what the barriers and success factors are to the effectiveness of feedback.

Data sources

Original articles in English found in MEDLINE Pubmed covering the period January 1990 to August 2007.

Review method

Titles and abstracts of 6223 original articles were independently screened by two reviewers to determine relevance for further review.

Data extraction and analysis

We used a standardized data abstraction form to collect information on the feedback initiatives and their effectiveness. The effect of the feedback was only described for analytic papers, i.e. papers that attempted to objectively quantify the effect on the quality of care and to relate this effect to feedback as an intervention. For analysis of the effectiveness, we categorized the initiatives based on the number of elements added to the feedback.

Results

We included 53 papers, describing 50 feedback initiatives, of which 39 were part of a multifaceted approach. Our results confirm previous research findings that adding elements to a feedback strategy positively influences its effectiveness. We found 22 analytic studies, four of which found a positive effect on all outcome measures, eight found a mix of positive- and no effects and ten did not find any effects (neither positive nor negative). Of the 43 process of care measures evaluated in the analytic studies, 26 were positively affected by the feedback initiative. Of the 36 evaluated outcome of care measures, five were positively affected. The most frequently mentioned factors influencing the effectiveness of the feedback were: (trust in) quality of the data, motivation of the recipients, organizational factors and outcome expectancy of the feedback recipients.

Conclusions

The literature on methods and effects of information feedback by medical registries is heterogeneous, making it difficult to draw definite conclusions on its effectiveness. However, the positive effects cannot be discarded. Although our review confirms findings from previous studies that process of care measures are more positively influenced by feedback than outcome of care measures, further research should attempt to identify outcome of care measures that are sensitive to behaviour change as a result of feedback strategies. Furthermore, future studies evaluating the effectiveness of feedback should include a more extensive description of their intervention in order to increase the reproducibility of feedback initiatives and the generalizability of the results.

Introduction

Many medical registries give information feedback to healthcare professionals on a continuous basis [1], even though there is no empirical basis for deciding how this feedback is best provided [2]. Therefore registries and health care professionals need to ascertain which way works best, so that giving feedback will become a more reliable approach to quality improvement and opportunities for improvement will not be missed. Therefore, this paper focuses on how medical registries give information feedback and on its effect on the quality of care.

As a result of governmental regulations, public demand, but also at their own initiative, more and more health care organizations have started to collect data in medical registries [1], [3], [4]. We define a medical registry as a systematic and continuous collection of a defined data set for patients with specific health characteristics. The data are held in a central database for a predefined purpose and information is submitted by multiple units (e.g. hospitals or cardiac surgery departments). Whereas in the past the focus of these registries was on health care planning and epidemiological research, nowadays many of them are also used for accountability and quality improvement (QI). They may provide health care professionals with insight into their performance, motivate change and drive QI activities [5]. To gain insight into professionals’ performance, structured data from registries play a central role in the Plan-Do-Study-Act (PDSA) cycle, especially when planning improvement activities and studying if these activities have been effective [6], [7]. Information feedback is a way to present these data to caregivers in a structured way, varying from a yearly paper report containing data aggregated for all participants together to a website where participants can have access to the most recent data, with the possibility of comparing their own results to that of their peers or to a national average. In addition to being a common approach to quality improvement, giving feedback to the original data providers is also a basic requirement for registries when aiming to increase the quality of the data and to motivate providers to collect data as part of their routine work [1], [8], [9]. In their review, Jamtvedt and co-workers concluded that audit and information feedback can be effective in improving professional practice. However, they also found that decisions about how to provide information feedback must be guided by pragmatic factors and local circumstances [2]. So from their review it remains unclear exactly which information feedback strategy works best.

Jamtvedt et al. reviewed information feedback based on any health care data source, while we focused on feedback based on data from medical registries. As medical registries combine data from different facilities, benchmarking the performance of individual facilities is an important feature. Due to their ability of being used for benchmarking, registries are often used in continuous QI initiatives. In addition, Jamtvedt's review was limited to randomized controlled trials (RCTs). Although randomized studies are seen as the optimal design for evaluating the effect of improvement strategies [10], observational studies are also valuable in understanding and evaluating such interventions [11], [12], [13]. Hence, we aimed to include not only RCTs, but any peer-reviewed paper on information feedback within the context of a medical registry. Furthermore, where Jamtvedt et al. only reported on the effectiveness of information feedback, we also aimed to identify the barriers and success factors to this effectiveness as reported in the literature. Knowledge on possible barriers and success factors might influence both type and content of the feedback strategy [14]. QI strategies tailored to such potential barriers and success factors are more likely to be effective [15], [16], [17].

Our systematic review is aimed at health care professionals and others that are starting or running a medical registry and (are planning to) provide information feedback to their participants. As we identified barriers and success factors, this paper is also relevant for health care providers who receive feedback from a registry and who wish to use this information for their local quality improvement practice.

The objectives of this paper were to determine (1) how medical registries provide information feedback to health care professionals, (2) whether this feedback has any effect on the quality of care and (3) the barriers and success factors for using this feedback for quality improvement.

Section snippets

Literature search strategy

We searched MEDLINE Pubmed for original articles in English for the period January 1990 to August 2007. We used MeSH terms referring to medical registries (databases (factual), registries), combining them with MeSH terms referring to QI strategies (quality indicators, concurrent/utilization review, total quality management, benchmarking, program evaluation, peer-review health care, medical/nursing audit) and MeSH terms related to other aspects of quality of care (outcome/process assessment

Search

After removing duplicates our search strategy resulted in 3459 original articles and 145 reviews and editorials. Initial screening of titles and abstracts resulted in 146 original articles for full text screening and 108 reviews eligible for hand searching the reference lists. These reference lists contained 2764 references, 41 of which we selected for full text screening. Finally, we included 53 papers in total (see Fig. 1 for the search flow), 24 of which were classified as analytic. The most

Discussion

In this review we aimed to determine how medical registries provide feedback to health care professionals, assess its effect on the quality of care and identify barriers and success factors for this effectiveness.

In our systematic review 53 papers were included, describing 50 diverse feedback initiatives. We found that medical registries mainly provided quarterly paper reports to healthcare facilities. The reports typically contained data aggregated on the facility level regarding process and

Conclusion

Our review shows that the literature on the methods and effects of information feedback by registries is heterogeneous, which makes it difficult to make straightforward comparisons between feedback initiatives and to draw definite conclusion on the effectiveness of feedback. Although the effect of feedback on the quality of care remains unclear, the positive effects of feedback provided by medical registries cannot be ignored. Our review confirms the findings from earlier studies that process

Authors’ contributions

KJ, NK and AR had the basic idea for this study. SV and ST were primarily responsible for designing and executing the search strategy. SV, KJ, NK and AR reviewed the titles and abstracts, included the relevant papers and completed the data abstraction forms. SV drafted the manuscript. All authors were involved in the critical revision of the paper for important intellectual content and its final approval before submission.

Summary points

What was known before this study?

  • there is no empirical

Glossary

Analytic papers
Papers attempting to objectively quantify the effect on the quality of care and relating this effect to feedback as an intervention.
Benchmark
A comparative standard that can be used as a target when trying to improve performance.
CQI
Continuous quality improvement.
Level of data aggregation
Level at which data are summarized, e.g. per health care provider.
Descriptive papers
Papers only describing the development or application of information feedback.
EHR
Electronic health record.

References (82)

  • V.T. Stewart

    Use of a prototype acute stroke registry to improve care: profile of receptive stroke programs

    Am. J. Prev. Med.

    (2006)
  • C. Kania et al.

    Using clinical and functional data for quality improvement in outcomes measurement consortia

    Jt. Comm. J. Qual. Improv.

    (1996)
  • R. Grol et al.

    From best evidence to best practice: effective implementation of change in patients’ care

    Lancet

    (2003)
  • J. Van der Lei

    Information and communication technology in health care: do we need feedback?

    Int. J. Med. Inform.

    (2002)
  • G Tripepi et al.

    Bias in clinical research

    Kidney Int.

    (2008)
  • G. Jamtvedt et al.

    Audit and feedback: effects on professional practice and health care outcomes

    Cochrane Database Syst. Rev.

    (2006)
  • A.E. Powell et al.

    Using routine comparative data to assess the quality of health care: understanding and avoiding common pitfalls

    Qual. Saf. Health Care

    (2003)
  • J.B. Reitsma, Registers in cardiovascular epidemiology, Chapter 1, Thesis/Dissertation, University of Amsterdam,...
  • Institute of Medicine

    Using Information Technology. Crossing the Quality Chasm

    (2001)
  • E.H. Bradley et al.

    Data feedback efforts in quality improvement: lessons learned from US hospitals

    Qual. Saf. Health Care

    (2004)
  • G.J. Langley et al.

    Skills to support improvement

    The Improvement Guide. A Practical Approach to Enhancing Organizational Performance

    (1996)
  • D.G. Arts et al.

    Defining and improving data quality in medical registries: a literature review, case study, and generic framework

    J. Am. Med. Inform. Assoc.

    (2002)
  • E. Hemminki et al.

    Need for and influence of feedback from the Finnish birth register to data providers

    Qual. Assur. Health Care

    (1992)
  • M. Eccles et al.

    Research designs for studies evaluating the effectiveness of change and improvement strategies

    Qual. Saf. Health Care

    (2003)
  • M. Petticrew et al.

    Evidence, hierarchies, and typologies: horses for courses

    J. Epidemiol. Community Health

    (2003)
  • K. Walshe

    Understanding what works – and why – in quality improvement: the need for theory-driven evaluation

    Int. J. Qual. Health Care

    (2007)
  • J.M. Grimshaw et al.

    Systematic reviews of the effectiveness of quality improvement strategies and programmes

    Qual. Saf. Health Care

    (2003)
  • M. Bosch et al.

    Tailoring quality improvement interventions to identified barriers: a multiple case analysis

    J. Eval. Clin. Practice

    (2007)
  • J.M. Grimshaw et al.

    Changing provider behavior. An overview of systematic reviews of interventions

    Med. Care

    (2001)
  • M. Wensing et al.

    Multifaceted interventions

  • T. Weijden et al.

    Feedback and reminders

  • M. Hulscher et al.

    Process evaluation of change interventions

  • M. Mugford et al.

    Effects of feedback of information on clinical practice: a review

    BMJ

    (1991)
  • A.D. Oxman et al.

    No magic bullets: a systematic review of 102 trials of interventions to improve professional practice

    Can. Med. Assoc. J.

    (1995)
  • M.N. Marshall et al.

    Use of performance data to change physician behavior

    JAMA

    (2000)
  • M.D. Cabana et al.

    Why don’t physicians follow clinical practice guidelines? A framework for improvement

    JAMA

    (1999)
  • R. Grol et al.

    Theories on implementation of change in healthcare

  • C.A. Beck et al.

    Administrative data feedback for effective cardiac treatment: AFFECT, a cluster randomized trial

    JAMA

    (2005)
  • J.E. Hux et al.

    Confidential prescriber feedback and education to improve antibiotic use in primary care: a controlled trial

    CMAJ

    (1999)
  • N.J. Pimlott et al.

    Educating physicians to reduce benzodiazepine use by elderly patients: a randomized controlled trial

    CMAJ

    (2003)
  • J.M. Schectman et al.

    The effect of an education and feedback intervention on group-model and network-model health maintenance organization physician prescribing behavior

    Med. Care

    (1995)
  • Cited by (140)

    • National registries: Lessons learnt from quality improvement initiatives in intensive care

      2020, Journal of Critical Care
      Citation Excerpt :

      To do so, the proportion of ICU admissions within each contributing hospital, and the proportion of ICUs within each country must be high. This is essential to ensure valid data, maintain stakeholder trust, and improve the effectiveness of feedback [2]. Timely, structured feedback, often classified into indicators of how patient care is organised (structure), conducted (process), or what is achieved (outcome), is essential.

    View all citing articles on Scopus
    View full text