Article Text
PDF
Abstract
Objective The study aimed to measure the validity of International Classification of Diseases, 10th Edition (ICD-10) code F44.5 for functional seizure disorder (FSD) in the Veterans Affairs Connecticut Healthcare System electronic health record (VA EHR).
Methods The study used an informatics search tool, a natural language processing algorithm and a chart review to validate FSD coding.
Results The positive predictive value (PPV) for code F44.5 was calculated to be 44%.
Discussion ICD-10 introduced a specific code for FSD to improve coding validity. However, results revealed a meager (44%) PPV for code F44.5. Evaluation of the low diagnostic precision of FSD identified inconsistencies in the ICD-10 and VA EHR systems.
Conclusion Information system improvements may increase the precision of diagnostic coding by clinicians. Specifically, the EHR problem list should include commonly used diagnostic codes and an appropriately curated ICD-10 term list for ‘seizure disorder,’ and a single ICD code for FSD should be classified under neurology and psychiatry.
- Electronic Health Records
- Health Information Systems
- Decision Support Systems, Clinical
- BMJ Health Informatics
- Healthcare Common Procedure Coding System
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- Electronic Health Records
- Health Information Systems
- Decision Support Systems, Clinical
- BMJ Health Informatics
- Healthcare Common Procedure Coding System
Introduction
Epilepsy is the fourth most common neurological disorder after Alzheimer disease, migraine and stroke.1 Overall, 20%–30% of people seen at epilepsy centers for drug-resistant seizures are diagnosed with functional seizure disorder (FSD).2 FSD is often misdiagnosed as epilepsy with several years of delay before a correct diagnosis.3 Subsequently, FSD is incorrectly documented and miscoded as epilepsy in the electronic health record (EHR). The International Classification of Diseases, 10th Edition (ICD-10) introduced specific codes for the diagnosis of FSD and epileptic seizures, respectively, ICD-10 code F44.5—FSD, conversion disorder with seizures (code F44.5) and ICD-10 code G40.9—epilepsy, unspecified.4 The differentiation of a code for FSD in the ICD-10 was intended to improve the validity of FSD diagnostic coding in the EHR.
A problem list is a compilation of diagnoses selected by clinicians during patient encounters and updated when a diagnosis changes.5 Outpatient records rely on clinician-inputted problem lists in the EHR to identify and document medical conditions.5 A single diagnosis may be represented by multiple, ICD-coded diagnostic terms. Correct diagnostic coding requires active maintenance of EHR problem lists and clinician judgement.6 An assessment of the quality of diagnostic coding supports better patient care and improved outcomes. The study aimed to measure the precision of code F44.5 in the VA Healthcare System (VA) EHR.
Methods
Setting
An informatics search tool and a natural language processing (NLP) algorithm identified potential cases of FSD through data extraction of VA inpatient, outpatient and pharmacy EHR charts across 170 VA medical centers in fiscal years 2002–2018.3 7 The development and validation of the NLP tool is described elsewhere.3 Briefly, the NLP classifier was validated using 2200 notes of veterans evaluated for seizure disorders. Reviewers used Yale cTakes Extension to annotate syntactic constructs, named entities and their negation context in the EHR. These annotations are passed to a classifier to detect NES patients. The achieved a positive predictive value (PPV) of 93%, a sensitivity of 99% and a F-score of 96%.
Sample
Of the 12 000 veterans diagnosed with FSD or epilepsy, a sample of 876 veterans coded with F44.5 were manually reviewed.5 FSD classification was based on the International League Against Epilepsy (ILAE) Nonepileptic Seizures Task Force levels: definite (clinically established diagnosis of FSD with video electroencephalogram (vEEG)), probable (seizure witnessed by a neurologist), possible (some mention of FSD in the chart), not (not FSD), epilepsy and both (mention of epilepsy and FSD in the chart).8
Statistical analysis
The PPV of code F44.5 was calculated with the true positive value to include definite (vEEG) and probable (seizure witnessed by a neurologist) groups, while the false positive (FP) value included not (not FSD) and epilepsy groups. Although the classification groups both and possible capture some cases of F44.5, the groups were excluded from the overall definition of F44.5 due to the possibility of captured FPs. Code F44.5 was used by 39 medical centres. Those medical centers were deidentified and stratified according to the frequency of code F44.5 usage (figure 1). Patient charts with missing data (n=3) for FSD classification and code F44.5 were removed from the analysis.
Frequency of FSD code F44.5 by Veterans Affairs Connecticut Healthcare System West Haven Medical Center. FSD, functional seizure disorder.
Results
Results indicated a PPV of ~0.439 with a 95% CI of (0.391 to 0.487). This PPV demonstrated a low precision rate for code F44.5 in the VA EHR. The sample of patients (N=876) included: definite n=99 (11%), probable n=128 (15%), possible n=347 (40%), not n=206 (24%), epilepsy n=83 (9%) and both n=10 (1%) (online supplemental figure 2). Among the medical centers, the highest accuracy was 65% (17/26) (figure 1). Conversely, the medical center with the most FSD diagnoses had a poor accuracy of 14% (7/48) (figure 1).
Supplemental material
Discussion
FSD is poorly documented in the VA EHR, as evidenced by the 44% precision rate for code F44.5. Many people with FSD who are misdiagnosed with epilepsy are prescribed unnecessary medications that are harmful and costly to the patient. Correct diagnostic coding of FSD leads to appropriate, timely treatment, as well as the appropriate allocation of healthcare resources. After auditing the documentation workflow, we speculated that the low precision rate for code F44.5 is in part due to coding errors in the lookup diagnosis and problem list functions.
Most EHR systems provide a lookup diagnosis function. This function allows clinicians to search for a keyword which yields a problem list of diagnostic terms to select from. When a clinician uses the lookup diagnosis function for a keyword search, some problem lists yield a lengthy list of diagnoses. A problem list with too many diagnoses to scroll through may overwhelm the user.5 For instance, a lookup diagnosis for the keyword epilepsy yielded a lengthy problem list with diagnoses ordered alphabetically. Conversion disorder with seizures or convulsions (a diagnostic term for FSD) was listed first (online supplemental figure 3). Clinicians may have inadvertently coded some epilepsy patients with an FSD diagnosis due to its convenient placement on top of the problem list.
Supplemental material
In contrast to lengthy problem lists, some problem lists exclude relevant diagnoses. When a lookup diagnosis yields a problem list with a single diagnosis, that diagnosis may be selected by default. For example, a lookup diagnosis for seizure disorder resulted in a problem list populated with only one term—conversion disorder with seizures or convulsions (FSD)—which was unequivocally wrong in many cases (online supplemental figure 4). The selection of an incorrect diagnosis by default suggests that some patients with seizure disorder or epilepsy were miscoded for FSD. This default selection is due to the exclusion of relevant diagnoses in a problem list. The optimisation of the lookup diagnosis and problem list functions may improve clinician coding.
Supplemental material
ILAE, ICD and Diagnostic and Statistical Manual of Mental Disorders DSM) have different diagnostic criteria for FSD.4 This lack of consistency may lead to diagnostic ambiguity and coding error. For example, the ILAE diagnostic classification system has distinct codes for FSD, epilepsy and seizure disorder. However, ICD-9 did not have a code for FSD and ICD-10 classified FSD under psychiatry instead of neurology.2 4 The shift in ICD-10 classification of FSD from a purely psychiatric disorder to a functional neurological disorder in the ICD-11 has not yet aligned with the DSM-5 classification of FSD as a mental health condition.9 Consequently, the variability among classification systems and in coding practices of clinicians have likely undermined the validity of EHR-coded data.2
The lessons learned from this implementer report demonstrate the necessity of routine audits on ICD coding for real-world healthcare system applications. In fact, due to the proven inaccuracy of FSD coding, the VA did not include FSD within their internal VHA Support Service Center Neurology Cube, a web-based capital project application and tracking database.10 Additionally, organisations should incentivise and support clinicians to maintain problem lists. Problem lists help facilitate patient care among clinicians and organisations. The standardisation of EHR problem lists and clinician coding practices can improve the quality of EHR-coded data and clinical processes.10 Finally, the low precision rate of code F44.5 suggests that the EHR-coded data for the differential diagnoses of seizures (ie, epilepsy, focal seizures, generalised seizures) may be inaccurate (online supplemental figures 3,4).
There are some limitations to the study and to this assessment. First, the errors in the lookup diagnosis function were not tracked by individual medical centres. Thus, which medical centers were impacted by which errors are unknown. Second, the problem list errors identified in this report were of one medical center’s VA EHR, and problem lists vary across medical centres. Finally, the unavailability of data on false negative diagnoses of FSD made it impossible to calculate the accuracy of code F44.5.
Conclusion
The low precision rate of FSD code F44.5 was affected by errors in the VA EHR’s lookup diagnosis and problem list functions, and by variations in FSD criteria across diagnostic classification systems. This implementor report demonstrated a health informatics approach to troubleshooting data validity. In brief, three key recommendations to promote FSD code validity emerged from the analysis: the problem lists should be composed of the most common and most inclusive diagnostic codes; the problem list results of the lookup diagnosis function for seizure disorder must yield all relevant ICD-10 terms; and a single ICD code for FSD should be classified under neurology and psychiatry. Overall, implementing information system improvements will increase the validity of diagnostic coding by clinicians and of EHR-coded data.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors HHA, CB and YB conceived of the presented idea. HHA, YB and SFA developed the theory and performed the computations. YB, DG and EJ-S conducted medical chart review. SA facilitated data mining and management. YB, EJ-S, SA, JG, MG and SFA verified the analytical methods. CB encouraged the investigation of using the NLP method and supervised the findings of this work. SFA took the lead in writing the manuscript with support from HHA. All authors provided critical feedback and helped shape the research, analysis and manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.