Effect of electronic health records in ambulatory care: retrospective, serial, cross sectional study

Electronic health record systems

Before implementation, individual paper based medical records were manually delivered to multiple sites throughout the integrated Kaiser Permanente system. These sites were often several miles distant from each other and from the medical records warehouse; availability of records for same day and unscheduled care was unreliable.

Paper medical records were essentially eliminated after implementation. Patients' electronic charts became available for unscheduled phone contacts, same day ambulatory care, and emergency department visits at Kaiser Permanente facilities. Although the regions implemented different systems, an internally developed clinical information system in Colorado and a externally supplied system (EpicCare) in the Northwest region, they shared important characteristics:

• Integrated documentation and reporting of clinical results reporting, including comprehensive recording of use of primary and specialty care, telephone contact, urgent care, and emergency department

• Computerised entry of physician orders for tests and prescriptions

• 24 hour availability of medical records at the point of care

• Immediate availability for all potential users—for example, staff in telephone advice centres, pharmacists, and staff reporting clinical results

• Easy searching.

Decision support functions varied between systems in terms of scope and function.

Design

To evaluate the effect of electronic health records, we conducted retrospective serial cross sectional studies for each region. We used administrative data to assess usage and the Health Plan Employer Data and Information Set to assess quality of care before and after implementation. This dataset is collated by the National Committee for Quality Assurance, which is recognised as the sole authority on the comparative performance of managed healthcare plans in the United States. The Health Plan Employer and Data Information Set is a series of standardised performance measures covering all population needs and pressing public health problems, such as appropriate treatment for patients with chronic diseases and care delivered to elderly people. Managed healthcare plans participate voluntarily. Results are based on statistically valid random samples of members and are rigorously audited.

The electronic health records were introduced progressively over more than a year and in different calendar years in the two regions. We reviewed data for entire calendar years, delimiting the implementation period by the calendar year in which implementation began (year I¹) and the calendar year in which it was completed (year I²). Given natural lags in implementation and impact, the baseline period includes year I¹ and the intervention period begins in year I².

To determine meaningful utilisation measures, we conducted interviews with roughly 100 clinical and operations leaders, managers, frontline clinicians, and staff from medicine, nursing, pharmacy, clinical laboratory and radiology services, risk management, medical records, and other areas after implementation of electronic health records. The interviews suggested the electronic system increased the efficiency of ambulatory care visits and reduced redundant or marginally productive healthcare services by allowing patient issues to be resolved during the first contact, enabling more services to be offered per visit and reducing the need for separate health maintenance visits. Informants also reported reduced use of clinical laboratory, radiology, and emergency services and increased effectiveness of scheduled and unscheduled telephone contact with patients.

To evaluate these impressions, we examined total office visits and separated use into primary and specialty care. We included visits to doctors and other health professionals. We also examined the frequency distribution of primary care visits. Data on total office visits and use of emergency departments, radiology departments, clinical laboratories, and telephone call centres were obtained from regional data warehouses.

Most informants perceived that the electronic health record improved quality of care. To assess quality of care, we extracted data from the Health Plan Employer and Data Information Set. From the 30 measures available in the dataset, we selected three that represented activities in primary and specialty care, care of routine and chronic conditions, and care requiring a cross departmental referral. These were percentages of relevant members receiving advice on stopping smoking, screening for cervical cancer, and retinal examinations in diabetes. These three measures were among the few that remained consistently defined over the eight year study period; the definitions of many measures have evolved over time.

Statistical analysis

We stratified the annual rates of total office visits per region by primary care and specialty care and adjusted to a fixed age distribution over the study period using two age categories (0-64 years and ≥ 65). Adjusting for smaller age increments and sex did not affect the results. We calculated the difference between the age adjusted visit rates in year I¹ and the observation period.

Year I¹ was selected as the baseline year for several reasons. In keeping with national trends, use of ambulatory care had been rising at Kaiser Permanente before electronic health records were introduced (fig 1). The reasons for the rise include changing reimbursement mechanisms, policies to review usage, and new technologies.12 Year I¹ represents maximum capture of this upward trend. It also represents the last point of substantially steady state operational practices in ambulatory care. Although implementation began in year I¹, it began slowly in isolated segments of ambulatory care, and it was some time before electronic records affected health care; most of the implementation occurred in year I².

Fig 1

Rates of office visits adjusted for age before and after introduction of electronic health records in primary care

• Download figure
• Open in new tab
• Download powerpoint

We selected year 2 as the primary comparison year because it was the last point for which consistent data from both regions were available. Regional measures of use of ambulatory care subsequently changed. In the Northwest region, data for two additional years were available because of the earlier implementation dates.

Because of the large sample size (around 400 000 members and 2 million visits a year in each region), the difference between the two rates approximated the normal distribution; we therefore used a z test to assess the significance of the difference. We also examined use of emergency departments, radiology, laboratory services, and call centres by this method.

Use of ambulatory care

Both regions had significant decreases in use of services. The age adjusted number of total office visits per member in year 2 decreased by 9% compared with year I¹ (P < 0.0001, in both regions), and age adjusted primary care visits decreased by 11%. Age adjusted specialty care visits decreased by 5% in Colorado and 6% in the Northwest (both P < 0.0001). In year 4, the total office visit rate in the Northwest region was 8% lower than before electronic records became available (fig 1). Partial implementation had minimal effect during year I¹.

The frequency pattern of ambulatory primary care visits suggested a general decrease in use across all patients, with larger reductions among patients making three or more visits (fig 2). The percentage of members making three or more visits a year decreased by 10% in the Colorado region and 11% in the Northwest region between year I¹ and year 2. In year 4, the rate in the Northwest region had decreased by an additional 2%. Moreover, the percentage of members with ≤ 2 visits a year increased. This finding is particularly striking in light of the ageing Colorado membership (a disproportionate number of people aged over 65 were enrolled during the study period) and is consistent with the effects of electronic health records described by clinical and operational leaders.

We reviewed data relating to other factors that could potentially explain decreased use of ambulatory care. Rates of visits to emergency departments (internal and external to Kaiser Permanente) did not rise over the study. To rule out inadvertent reductions in access to services or shifting of care to other providers, we reviewed the ratio of all primary care providers (physicians, nurse practitioners, and physician assistants) to members and the ratio of referrals to outside providers in both regions throughout the study. Both ratios remained stable.

To rule out other global influences, we compared the changes in the Colorado and Northwest regions with trends in Kaiser Permanente regions without electronic health records and with national trends. Inconsistent definitions of office visits precluded direct comparisons between regions. We examined the rate of change in office visits, as independently defined by three other Kaiser Permanente regions, for which trend data were available for the same period. The data did not show comparable decreases. The rate of ambulatory care visits by people aged 45 or older increased by 14% across the United States between 1992 and 2002, which encompasses our study period.13

Telephone contact

The electronic health record enabled more effective telephone contacts. In the Northwest region, telephone encounters scheduled at the discretion of physicians increased from a baseline of 1.26 per member per year to 2.09 after two years. In the Colorado region, staffing of call centres briefly shifted from primarily nursing staff to include doctors with access to electronic health records. Appointments needed by patients after telephone contact decreased by 7% when contact was with a doctor with access to electronic health records. Doctors reported being able to resolve health issues by phone more readily with the electronic health record. Rates of appointments after telephone contact rose when staffing reverted to nurses. Comparable data on telephone contact from other Kaiser Permanente regions were not available.

The effect of telephone contact on use of ambulatory care is shown through a contemporaneous operational evaluation of the telephone treatment of uncomplicated urinary tract infections. In the Colorado region, a previously available nursing protocol was built into an electronic health record template in year I¹, increasing ease of access for nursing staff. Between year I¹ and year 2, the age adjusted visit rate for urinary tract infection among women fell by 31%. The records of women treated with antibiotics for a urinary tract infection during three months in year I¹ were randomly audited. Of 262 women whose records were audited, 73 were prescribed antibiotics by a nurse; 67 of these did not require a return visit within eight weeks of treatment, indicating appropriate resolution.

Radiology and clinical laboratory services

Age adjusted rates of use of radiology services decreased by 14% in the first two years after introduction of electronic health records in the Northwest region. Despite more recent increases in general use of imaging inside and outside Kaiser Permanente, the age adjusted rate remained 4% lower than before implementation. The chief of radiology in the Colorado region believed strongly that availability of electronic records to all carers improved interpretation of films.

Laboratory usage in the Northwest region had decreased by 18% four years after electronic health record were introduced; rates subsequently increased 5-7% annually. Rates of laboratory usage in the Colorado region remained generally stable, rising 14% before electronic health records were introduced and falling 2.9% in the two subsequent years.

Quality of care process measures

Intermediate measures of quality of health care remained unchanged or slightly improved after electronic health record were introduced (table 2). This allays any fleeting concerns that decreased usage compromised quality of care.

Possible explanations

The observed changes in use of services have many potential explanations. Efficiency of outpatient care may have been increased by the readily available comprehensive clinical information. This conclusion is supported by the shifting frequency distribution of visits, the increased use of phone contact in lieu of a visit, and doctors' consistent observations across the two regions that electronic health records enabled them to identify and resolve patients' health issues in the first contact or with fewer contacts.

We were able to eliminate several confounding factors as potential causes of reduced use of ambulatory care. These include changes in age distribution, reduced practitioner availability (as measured by the ratio of all primary care providers to members), and possible shifting of primary care to specialty, emergency, or outside care (as measured by rates of visits and outside referral rates).

Organisational pressure to reduce use of ambulatory care could have caused similar effects. Despite differing operating systems and organisational goals, neither region had an implicit or explicit goal to reduce outpatient visits during the study. No meaningful changes occurred in copayments for primary care or specialty care, and neither the health plan products nor the products offered by employer purchaser groups changed substantially during the study period.

Decision support embedded in the electronic health records also potentially influenced usage. The role of decision support systems in guiding clinical decision making has been documented.14 15 Decision supports were more developed and widespread in the Northwest system than in the Colorado system, but their contribution to the reduced rates of use was not a focus of this study.

Applicability

A non-representative population could potentially limit the applicability of our findings. However, neither region represents an unusual patient population. The members in both regions have an average or above average illness burden, as measured by the nation's Medicare hierarchical category classification scores.

Administrative level data leave some questions about the effect of electronic health records unanswered. For instance, keyinformants expected that improved availability of complete laboratory data would eliminate redundant testing, yet we found no conclusive evidence that this was the case. Decreases in use of laboratory services were neither consistent in the two regions nor sustained over time. However, inappropriate use of clinical laboratory services before the availability of electronic health records may also include underuse of some tests (such as glycated haemoglobin and lipid screening). Examination of ordering patterns for specific tests may better reflect the effect of electronic health records than overall trends.

Effect on doctors

Introduction of electronic health records represents a substantial change in doctors' workflow. Some research indicates that electronic health record systems impose a greater burden on clinicians.16^–18 This effect may be temporary19 or situation dependent.20 However, internal Kaiser Permanente work indicates that the effect of electronic health records on clinician workload is highly variable but time neutral on average (unpublished data). Additionally, Geisinger Health System noted a significant improvement in productivity after introducing electronic records, as measured by McGladrey relative value units (Joseph Bisordi, personal communication, 2003).

Interviews with clinical and operations leaders indicated that strong leadership support for realising potential efficiency gains was key to successful implementation. Organisational structure supporting free flow of information and efficiency gains is key to realising the benefits of electronic health record implementation.21 However, the organisational dynamics affecting implementation require further study.

Effect on quality

Despite perceptions of improved quality of care after electronic health records were introduced, we found generally stable and only occasionally improved performance on selected measures. Both regions are high performing, which may make it more difficult to identify marginal quality improvements.22 Health Plan Employer Data and Information Set data for reporting year 2003 placed Colorado among the 10 top performing health plans in the country for effectiveness of care measures and the Northwest among the Pacific region's five top performing plans.23 At a minimum, we can assert that electronic health records and the resulting effects on usage do not reduce the quality of care and may in fact increase appropriate use of healthcare services.