Structured data quality reports to improve EHR data quality

https://doi.org/10.1016/j.ijmedinf.2015.09.008Get rights and content

Highlights

  • SDQR and feedback sessions support improvements in the recording of clinical records.

  • Significant improvements were made in the recording of risk factors.

  • Improvements not sufficient to meet RACGP data quality standards.

  • An ecological approach across the data cycle is needed to improve DQ in EHRs.

Abstract

Objective

To examine whether a structured data quality report (SDQR) and feedback sessions with practice principals and managers improve the quality of routinely collected data in EHRs.

Methods

The intervention was conducted in four general practices participating in the Fairfield neighborhood electronic Practice Based Research Network (ePBRN). Data were extracted from their clinical information systems and summarised as a SDQR to guide feedback to practice principals and managers at 0, 4, 8 and 12 months. Data quality (DQ) metrics included completeness, correctness, consistency and duplication of patient records. Information on data recording practices, data quality improvement, and utility of SDQRs was collected at the feedback sessions at the practices.

The main outcome measure was change in the recording of clinical information and level of meeting Royal Australian College of General Practice (RACGP) targets.

Results

Birth date was 100% and gender 99% complete at baseline and maintained. DQ of all variables measured improved significantly (p < 0.01) over 12 months, but was not sufficient to comply with RACGP standards. Improvement was greatest with allergies. There was no significant change in duplicate records.

Conclusions

SDQRs and feedback sessions support general practitioners and practice managers to focus on improving the recording of patient information. However, improved practice DQ, was not sufficient to meet RACGP targets. Randomised controlled studies are required to evaluate strategies to improve data quality and any associated improved safety and quality of care.

Introduction

Current Australian and international health policies emphasise the importance of electronic health records (EHRs), information sharing and the smart use of data, information and communication technologies in improving the coordination, quality and efficiency of healthcare through the secure use and sharing of information [1], [2], [3]. The use of EHRs can assist with clinical decision making, improve adherence to guidelines and reduce errors in prescribing and most of the evidence for improvements in quality of care are made in primary and secondary prevention. [4] The routine collection of clinical information from EHRs is increasingly being stored in large data repositories and used for research and quality improvement. Quantifying, understanding and improving the quality of routinely collected clinical information in EHRs is crucial if they are to support effective clinical care, monitor safety and quality and be useful for audit and research purposes.

The Royal Australian College of General Practice (RACGP) Standards for General Practice 4th edition [5] were developed by a National Expert Committee in consultation with key stakeholders in primary care. The indicators, explanations and practical resources for general practices include key elements of a National Accreditation Scheme and e-health initiatives: patient records, health summaries, patient identification, clinical handover, governance and quality use of medicines. Improving the quality of data in EHRs requires the use of consistent coding systems and terminology, the recording of ethnicity and Aboriginal or Torres Strait Islander status, minimum requirements for health summaries such as relevant family and social history and current medications, preventive care (e.g. smoking and blood pressure) and the documentation of consultations so they include the reason for visit, referrals, clinical findings and other information necessary for good decision making, clinical handover and integrated care.

The University of New South Wales (UNSW) electronic Practice Based Research Network (ePBRN) extracts and links data from information systems (IS) and electronic health records (EHR) in general practices, hospitals (ED and admissions) and ambulatory/community health services in the Fairfield health neighborhood in South Western Sydney, Australia. The quality of data and information from participating general practices is routinely examined for completeness, correctness, consistency and duplication of records [6]. Generally, the quality of demographic (except for ethnicity), service, prescribing and investigation data were better than those of clinical measures such as BMI and smoking status. Completeness of records and data varied between the practices. There was a lack of consistent coding rules and standards for data entry within and across the practices. These findings are consistent with other studies on the quality of data in primary care Clinical Information Systems (CIS) [7], [8]. A literature review included twelve studies which reported some change in quality of data after feedback, training or audit type interventions. However, it was not clear whether it was the intervention, participation or time that explained the changes in data quality [9].

We report on a study to examine whether a structured data quality report (SDQR) and feedback sessions with practice principals and managers improved data quality of EHRs amongst general practices participating in the Fairfield neighborhood ePBRN.

Section snippets

Materials and methods

The GRHANITE™ extraction and linkage software pseudonymised patient records, using a standard hashing technique [10]. Clinical and managerial data were extracted, encrypted and sent securely to the ePBRN data repository at UNSW. Patient records were linked within and between the practices, allowing the identification of patients with duplicate records or clinical records at more than one practice. For this study, we used data extracted at four time points (baseline, 4, 8 and 12 months) from the

Results

Seven GPs (including practice principals) and two practice managers from the four practices consented to participate in the SDQR feedback study. These practices varied in size with the number of GPs ranging from 4 to 7; the two largest practices employed practice nurses and full time practice managers.

The aggregate RACGP-active population for the four ePBRN practices at baseline was 27,042. The size of the RACGP-active populations varied across the practices (Practice #1: 870; Practice #2

Discussion

The recording of risk factor information improved significantly with the intervention, but it was not sufficient to meet the RACGP targets. The negative trend found with the recording of Aboriginal and Torres Strait Islander status reflected a changing RACGP-active practice population as well as multiple factors such as staff inertia and patients not volunteering information. GPs confirmed they do not routinely collect this information as they believe they know their patients and only ask those

Conclusion

SDQRs with benchmarks and peer-comparisons and feedback sessions with practice principals and practice managers support general practitioners and practice managers to focus on improving the recording of patient information. However, improvements did not result in compliance to RACGP targets. It may be that the targets are unrealistic, but further mixed methods studies are needed to understand the reasons why in various contexts, target populations and practice organization models these are not

Conflict of interest

No conflict of interest

Contribution of authors

Jane Taggart contributed to the design of the study, the collection and interpretation of extracted and qualitative data, the drafting and revising of the paper and final approval for submission. Siaw-Teng Liaw contributed to the conception and design of the study, the interpretation of extracted and qualitative data, drafting and revising the paper and final approval for submission. Hairong Yu contributed to the management, manipulation and interpretation of the extracted data and revising the

Acknowledgements

All participating general practices who contributed their data and perspectives and Oluyemisi Ijamkinwa (OI) who assisted with the feedback sessions and data synthesis.

Not all numbers in the references are the same size fontReferences (10)

  • Australian Government Department of Health. Primary Health Care reform in Australia. Report to support Australia's...
  • National Health & Hospital Commission. A Healthier Future For All Australians—Final Report of the National Health and...
  • D. Blumenthal

    Wiring the Health System—origins and provisions of a new federal program

    New Engl. J. Med.

    (2011)
  • B. Chaudhry et al.

    Systematic review: impact of health information technology on quality, efficiency, and costs of medical care

    Ann. Intern. Med.

    (2006)
  • Royal Australian College of General Practitioners. Standards for General Practices (4th Edition) 2013....
There are more references available in the full text version of this article.

Cited by (32)

  • Predictive Abilities of Machine Learning Techniques May Be Limited by Dataset Characteristics: Insights From the UNOS Database

    2019, Journal of Cardiac Failure
    Citation Excerpt :

    We think that our results, when considered in conjunction with these previous findings, provide an insight into the inability of machine learning methods to overcome key systemic limitations of clinical datasets. In this case, whereas UNOS is a robust clinical registry, it is based on administrative data, which can blunt phenotyping of complex patients and attenuate the predictive ability of both traditional methods as well as machine learning techniques; indeed, this has been noted several times in efforts aimed at extracting meaningful information from the EHR.15 Our analysis illustrates this limitation as we see the AUC for neural networks plateau after inclusion of ∼25 variables in the model.

  • The informatics capability maturity of integrated primary care centres in Australia

    2017, International Journal of Medical Informatics
    Citation Excerpt :

    A critical analysis with comparison among IPCCs, using inductive and deductive methods, was conducted to understand associations between the ICM and extent of integration of data, systems and services. EHR data quality (completeness and consistency) was assessed and benchmarked using in-house methods [19,24]. Interpretation was in context, considering factors such as patient, provider, team organisation, health financing and regulations as enablers and barriers to adoption/use of eHealth tools [25].

  • Impacts of structuring the electronic health record: Results of a systematic literature review from the perspective of secondary use of patient data

    2017, International Journal of Medical Informatics
    Citation Excerpt :

    In addition, several major developments have taken place in recent years, which have not necessarily been reflected adequately in the studies we reviewed – namely the emergence of national e-health service programs, cross-border health projects (in the EU), mobile health care applications, personal health records and health data recorded by citizens. A continuation of our review from 2010 onwards would show what the currently trending areas of research are in the secondary use context [e.g. 103–105]. With regard to the evaluation methods used, our review revealed the absence of common methodologies – despite there being common evaluation topics or expected benefits.

  • A guide for the application of analytics on healthcare processes: A dynamic view on patient pathways

    2016, Computers in Biology and Medicine
    Citation Excerpt :

    Our case study starts with a prepared event log. The aggregation of EHRs brings along its own challenges [15–17]. Nevertheless, PA techniques can help identifying and solving data quality issues.

  • Data quality assessment framework to assess electronic medical record data for use in research

    2016, International Journal of Medical Informatics
    Citation Excerpt :

    One of the limitations in assessing data completeness is that there is no guidance as to what the metrics or benchmarks for data concordance, completeness and correctness should be. Currently, implementing approaches such as the Structured Data Quality Report method [20], or the Data Quality Probe [10] method over time have been shown to improve provider practice for clinical data entry and thus data completeness, additional assessment efforts are necessary. Future work should focus on identifying what constitutes acceptable missingness, and/or establish benchmarks of acceptable inclusion levels of patient records (e.g. what are acceptable confidence intervals for predictors within a decision support system that is developed and then runs on EMR data, and when can it be introduced into practice).

View all citing articles on Scopus
View full text