Article Text
PDF
Abstract
Background Primary care in UK is expected to use tools such as the electronic Frailty Index (eFI) to identify patients with frailty, which should be then validated and coded accordingly.
Aim To assess the influence of organisation and software on how eFI score and direct clinical validation occurs across practices in Leeds.
Method The ‘minimum necessary’ anonymised patient data required for the study (recorded eFI scores and frailty codes – mild, moderate or severe – with their dates of entry) was requested to the Health and Care Hub of the NHS Leeds Clinical Commissioning Group. Data from 44 185 patients from 104 practices using two different clinical software were collected. Descriptive statistics was carried out using SPSS software.
Results 42 593 patients had a frailty code, 8881 had an eFI code. 7341 had both types of entry, and correlation between eFI and coded level of frailty was as expected high (85.3%), but there was statistically significant variation depending on practice and software used. When results did not match, there was a tendency to overstate, to code a level of frailty above the value to be assigned based on the numeric value of eFI, and it was more so on those practices using SystmOne software compared with those using EMIS Web.
Conclusions Although correlation was generally good, the variability encountered would indicate the need for training and also for software improvements to reduce current disparity and facilitate validation, so frailty level is adequately recorded.
- electronic frailty index
- electronic health record
- frailty
- primary care
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
Frailty can be objectively assessed in different ways, like using the PRISMA-7 questionnaire1 or the Groningen Frailty Indicator,2 but more recently a computerised way has been suggested: the electronic Frailty Index (eFI), which 'can be automatically populated with routinely collected primary care electronic health record (EHR) data'.3 The eFI has been validated on patients older than 65 years of age based on 36 areas or deficits (including activity limitation, atrial fibrillation, diabetes, falls, housebound, polypharmacy, respiratory disease). It was constructed based on the presence or absence of 2171 codes on the electronic record. The algorithms used in eFI would allocate a score below 0.12 if the patient is fit,>0.12–0.24 if mildly frail,>0.24–0.36 if moderately frail and above 0.36 if severely frail.
Frailty assessment in general practice in England is now contractual4 and it is a challenge for primary care.5 A suggestion is to use eFI as a screening tool and to follow it with direct clinical verification4 to deal with potential data quality issues. In other words, to carry two independent processes to assess the same condition, the presence of frailty. While eFI is an automatic process objective, based on recorded data, the recommended clinical verification is a manual process, likely to be subjective, without a clear recommended tool to assess record entries or to question the patient in search of parameters indicative of frailty.
The aim of this article is to assess how GPs are coding frailty, whether based on eFI or not, and also what variability exists.
In Leeds, primary care use the following two clinical software:
SystmOne. Using this software it is not possible to obtain the eFI on a particular patient, but it has to be accessed running a report on the practice population. The software will calculate the score for every patient in the organisation but the user would indicate the chosen percentage of patients to be presented, with scores from high to low.6 Once the output on the partial number of patients with a score is available, the clinician needs to decide how to use the data, such as whether to bulk-code patients with the score or even a frailty code or with an alert about the need to validate the score. Scores will not be transferred to patient records automatically.
EMIS Web. In this case eFI is not run on population, but it is used in a case-by-case approach. On opening a patient record the software will automatically calculate, and – if the patient is frail – provide an alert to indicate the eFI score.7 The clinician needs to decide how to manage that alert, such as adding a frailty code, a score or both to the patient record. Scores once again will not be transferred to patient records automatically.
In both scenarios, the clinician is unable to see how the score was calculated as none of the mentioned software would provide any additional tool for corroboration. The clinician would need to look into the record and/or consult the patient to validate the score, to code the perceived level of frailty (mild/moderate/severe).
Aim
There are three recognised objectives:
To assess validation of eFI.
To assess variability among practices on the use of eFI and coding.
To assess if software used influences validation.
Method
The Health and Care Hub of the NHS Leeds Clinical Commissioning Group was approached to obtain the ‘minimum necessary’ anonymised patient data for the study. The inclusion criteria were patients of any age with a recorded score of eFI or one of three frailty codes (mild, moderate, severe) over the past 2 years. Patients were excluded if codes were older than 2 years or required codes were absent. The written request was made in late November 2018. Two weeks' later the obtained document contained eFI scores and frailty codes with dates, as well as clinical software used, all grouped by anonymised GP practice.
There were 44 185 cases among the 104 practices in Leeds (18 using EMIS Web and 86 using SystmOne) where data regarding eFI scores and/or frailty scores recorded over the past 2 years were available.
IBM SPSS version 24 was used for all statistical analyses and as being the primary endpoint the validation of frailty, the level of correlation between eFI and frailty coding, the analysis focused on those patients with both types of entry. Secondary endpoints were the effect of practice and software. In all cases descriptive statistics was used to compare the relevant groups, compiling frequency tables and crosstabulating eFI scores against frailty codes not only as a whole, but also in relation to particular practice and software.
Results
Among the 44 185 records, 42 593 patients had a frailty code, including 35 130 patients who had a code indicating frailty but without eFI score. The eFI was done in 8881 patients: among them 1471 individuals who had a score below 0.12, meaning they were not frail. 7341 cases had both eFI and frailty code, representing 17.2% of patients with recognised frailty.
Practices varied in the number of patients with a recorded eFI score between one patient and 1059 (average 90.6 patients) while regarding the number of patients with a frailty code it varied between one and 2762 (average 321 patients).
As an average, a record entry of eFI score was (minus) 22.9 days before the entry of a code of frailty severity, but it varied between minus 707 and 602 days (SD 182.9).
Among the 42 593 patients who had a code of frailty, 17 509 patients were coded to have mild frailty (41.1%), 17 788 patients had moderate frailty (41.8%) and 7297 patients had severe frailty (17.1%).
There were 7341 patients with both eFI scored frailty and coded level of frailty entry, representing 17.2% of the patients recognised as frail. Assessing the correlation between these two values (see table 1), it was noted that they were equivalent in 85.3% of cases. In 11.7% of cases clinicians overstated, that is, coded a level of frailty that was above the level that corresponded to the numeric value of eFI, while in 3% of cases, patients were considered less frail than the score indicated.
Correlation between eFI score and frailty code assigned.
A different perspective was obtained when data were assessed depending on practice (figure 1). There were practices that consistently coded frailty level above the corresponding score (such as practice ID6, overstating 97.1% of their 137 cases, or practice ID58, overstating 100% of their 256 cases). There were practices that only agreed on about half the patients (practice ID9, 55.8% concordance among their 156 cases). Understating, coding a level of frailty below the corresponding score, was less commonly encountered (Practice ID87, with 28.6% understatement and 71.4% agreement among their 56 cases).
Correlation between eFI and frailty codes among practices.
Pearson Chi-Square Test analysis confirmed practice differences were statistically significant (X squared (164, n=7341)=5123.68, P<0.001).
When software used was assessed (see table 2), the concordance was higher among EMIS Web practices than among SystmOne surgeries. This was mainly the result of significant overstating among the latter (22%).
Concordance between eFI and frailty code depending on software used
Pearson Chi-Square Test analysis confirmed it was also statistically significant (X squared (2, n=7341)=690.43, P<0.001).
Discussion
Summary
This study confirms that general practice does not seem to use eFI as the main tool to diagnose frailty, as only 17.2% of frail patients had a score present. Although 'it would be expected that a subjective judgement of frailty as documented in GP records would agree poorly with a more formal method of determining frailty' (Professor David Stott, personal communication, 21 December 2018), the correlation between the two assessments is good (85.3%). Unfortunately it is influenced by software used and by organisation considered, so current validation of eFI, as suggested by NHS England4 is flawed:
Although there is an overall agreement between the frailty coding and the scores obtained automatically by the clinical software, when data is looked into in more detail, it is clear there is wide variation among practices.
Frailty assessment is a new way of predicting medical risk in patients, and clinicians need better support and training to evaluate it.
Clinical software influences results. The facts that those potential patients are obtained in different ways, and that there is no way to assess why the score came up to a particular number, no doubt contribute to variability in results. There is a need to correct two potential sources of variability:
Why is it not possible to obtain scores individually and in reports simultaneously in practices?
Why is there no more information available to clinicians when presented with an eFI to facilitate validation, to assess the used codes are relevant for the individual patient at hand?
Validation should be confirming data quality of the codes assessed, which is not feasible at present on the two software tested, and variability is the result of it.
Strengths and limitations
To the best of our knowledge this is the first assessment of the validation of eFI scores and coded entries of frailty in a large population. It has allowed assessing variability, its roots and potential solutions in a large sample, with more than 100 practices, and results are likely to reflect what is the situation nationally.
Due to the nature of the study, obtaining just the minimum possible data required to carry the analysis without getting complementary information that would require a long process of ethical approval, there is the possibility more data has been assessed by clinicians without coding, particularly checking eFI scores but not adding their values to patient records. It could influence the percentage of patients having eFI and the level of correlation, although it must be argued a sample has been done regarding that particular relationship between the two assessments.
Comparison with existing literature
NHS England guidance4 is clear on what general practice should aim for, but clinical validation needs robust mechanisms, and it could be of concern the fact the majority of patients did not have eFI scores when coded as frail, but it clear there is heterogeneity of tools available to primary care8 and there is variability on primary care electronic records data quality.9 10 Obtained data pairs in any case provided a clear insight on the relationship between the two, and the influence of software, an area not looked at in the literature.
Acknowledgments
This study was possible thanks to the approachable support of the data Health and Care Hub of the NHS Leeds Clinical Commissioning Group.
Footnotes
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.