Methods
The trial evaluating the HST’s effects was designed as a phased, cluster-randomised (‘step-wedge’) trial with each of six randomly determined clusters of nine hospital admitting services becoming successively included in the intervention group over contiguous 4-month periods. PWH hospitalised on services not yet included in the intervention group served as controls. We designed the EMR alert to identify all PWH on both intervention and control services. In this manuscript, we describe the EMR alert’s function in identifying PWH on all services included in the randomised trial. Further description of the methods and results of the trial itself will be the focus of future manuscripts.
Our hospital uses Epic corporation’s Hyperspace software as its EMR in both inpatient and outpatient settings. Our alert, programmed by an Epic physician builder within our health system (TG-B), screened records of adult patients admitted to inpatient status (excluding observation hospitalisations) to determine if they met criteria of PWH. We excluded observation hospitalisations (<48 hours) because these might be too short for the HST to be effective. The alert’s output consisted of a message with patient name and medical record number delivered to the EMR’s ‘In-Basket’ system.
The alert was designed to trigger for any one of four criteria, chosen to identify PWH using discreet EMR data elements: (1) an HIV International Classification of Diseases 10th Revision (ICD-10-CM) code (B20, Z21, O98.711–O98.73) in any current or prior outpatient or inpatient problem lists, (2) any antiretroviral therapy (ART) medication(s) which can be used (but are not necessarily specific) for HIV (identified from pharmaceutical subclasses maintained and updated externally by the EMR vendor, online supplemental S1) on the patient’s current inpatient or historical outpatient medication lists, (3) an HIV-1 RNA level assay ordered (regardless of result) during the hospitalisation or (4) a positive HIV-antibody (Ab) result during the hospitalisation or at any prior point in time. The third criterion was intended to capture both instances of the clinical team performing virological monitoring of individuals with diagnosed HIV, (in which case the result could be either detectable or undetectable) and instances of diagnosing acute HIV infection during the hospitalisation (in which case the result would be detectable). We did not look at HIV-1 RNA level testing prior to the hospitalisation because we felt this may introduce a high number of false-positives due to prior attempts at diagnosing acute HIV, and because we felt these individuals would be well captured by the fourth criterion.
We initially considered a fifth criterion, a laboratory order for a CD4 cell count during the index admission. In 2 weeks of predeployment testing (40 hospitalisations alerted), this criterion triggered for 3 hospitalisations of HIV-uninfected persons, all of whom had CD4 cell counts ordered to assess immunodeficiency in the setting of cancer chemotherapy. The CD4 criterion did not identify any PWH who were not identified by one or more other criteria; thus, it was eliminated as an alert trigger for subsequent hospitalisations.
In May 2017, we deployed the alert and began the randomised trial. We reviewed all alert instances in the first month of deployment (approximately 60 charts). We identified a single instance where the alert was activated by the HIV-1 RNA criterion but failed to recognise that the patient also had a positive antibody. In this case, we identified and fixed a coding error that resulted in an unintended upper age limit for the antibody criterion. We then considered our specifications for the HIV alert criteria finalised. The alert build and post go-live support required 74 total hours of physician builder time.
The alert separately notified a data abstraction team tasked with confirming the patient’s HIV status (through reviewing chart notes and/or lab results) and the intervention team (HST members), who used the alerts to know which patients to see at the bedside. Rather than manually define individual data abstraction team members and intervention team members, the alert sent messages to separate recipient pools for each of these teams. Individuals could be added and removed from each pool as needed for team member turnover.
To analyse sensitivity, two nurses (EH and KH) conducted manual chart reviews of hospitalisations selected without regard to the EMR alert. A manual review of all adult patients admitted during the 2-year intervention period (approximately 100 000 hospitalisations) was beyond our capacity, so we collected a random sample of charts over a 4-month interval during the midpoint of the intervention period, from admitting services that averaged more than 10 hospitalisations among PWH per year. We aimed to review 1500–2000 charts, approximately 3%–4% of annual hospital volume. The protocol for each review began with reading the admission history and physical note and the most recent progress note looking for HIV (or AIDS) described as an active or historical diagnosis. If there was no indication of HIV (or AIDS) in the clinical notes, the reviewers then examined laboratory, medication and problem list chart sections and finally screened outpatient visits for any visits at the hospital-affiliated HIV clinic.
We also determined the proportion of hospitalisations identified by the full alert that was identified by each alert criterion alone and in two-way combinations. Assuming the full alert would approach 100% sensitivity, these results would then approximate sensitivity estimates for the individual criteria.
For the analysis of PPV, we started with the abstractor team’s manual reviews of each alerted hospitalisation indicating whether the patient was, indeed, living with HIV. We then performed a secondary review of all 199 hospitalisations the abstractors initially classified as false-positives (not having HIV infection despite the alert triggering). The secondary review involved detailed examination of current and prior discharge summaries, inpatient progress notes, outpatient office visits, medication lists and a search of outside records (available through EMR links) including antibody measurements and viral loads. Finally, we performed a retrospective EMR query of the charts that were identified by the alert to analyse which criteria (ICD-10-CM, ART, HIV-1 RNA, antibody or a combination) activated the alert.
Through this post-deployment analysis, we observed that the antibody criterion had never alerted, confounding our expectations. On a targeted review of a sample of 10 charts, 4 of which were known to have a positive antibody result within our EMR, we determined that the alert, as coded, was not successfully capturing antibody tests. Our 2017 audits did not include charts in which the only positive criterion was the antibody, and thus we failed to identify this issue prior to or during the intervention period. The error appears to have originated from failure of a function within the EMR to translate textual results of antibody tests into discrete normal/abnormal data.
To evaluate the potential impact of the antibody criterion failure, we determined from our retrospective EMR query that only 95 of 110 028 (0.09%) adult hospital admissions during the study period (29 unique patients) were associated with a positive HIV antibody and no other criteria. Thus, we considered the percentage of charts missed by the failure of this criterion to be negligible. We limited our analysis of individual criteria to the information available for the three functioning criteria: ICD code, HIV-1 RNA or prescription of any ART. We performed analyses using Stata V.16.1 software (StataCorp)11 with an α value of 0.05 for significance and CI calculations of sensitivity and PPV.