Discussion
This study provides knowledge not otherwise obtainable about patients with diabetes. First, we were surprised that we identified only one other study that developed and/or validated a prediction rule related to HbA1c among patients with diabetes. Second, without the rule, the provider does not know a given patient’s likelihood of having an HbA1c<75 mmol/mol within or by the end of the next 2 years.
While sensitivity and specificity provide the probability of the model showing attainment of HbA1c<75 mmol/mol, or lack thereof, in a patient who has such a measurement, the predictive value provides the probability of actual attainment of HbA1c<75 mmol/mol, given the model showing, or not showing, such attainment. The potential contribution of these prediction rules to a learning healthcare organisation is dependent on accurate differentiation between patients who will, and will not, continue to have HbA1c measurements ≥of 75 mmol/mol. As seen in tables 3 and 4, the predicted probabilities compared with the observed are fairly accurate for each of the tertiles for each of the outcomes.
If aggregate data from the healthcare system were available to providers, they would know 55% of these patients maintain HbA1c values<75 mmol/mol at 2 years (ie, the pretest probability). The usefulness of a prediction rule is based on the proportion of patients whose probability has substantially shifted from the pretest probability. When predicting who will maintain HbA1c values of <75 mmol/mol, the majority of patients’ probabilities shifted. This did not occur for predicting who will ever have an HbA1c<75 mmol/mol; of the two, maintenance is more imperative clinically.
Most importantly, this rule identifies patients for whom an alternative approach is more appropriate. Resources should be reallocated for the 15% of patients with predicted probabilities of 0–0.4 and who are interested in meeting treatment goals: they have less than one-third chance of maintaining HbA1c values of <75 mmol/mol using the approach employed by PCPs (table 3). This implies a substantial shift in workload as it represents multiple visits currently occurring between PCPs and patients addressing treatment intensification. Our model suggests the presence of underlying drivers of poor glycaemic control that operate over the course of years. These factors may be psychosocial and should be addressed before such treatment is again attempted; our recommendation is psychosocial screening. In this way, patients are also being offered treatment intensification: once these issues have been addressed, they may be more likely to attain an HbA1c value of <75 mmol/mol.14 By identifying the problem from a population health perspective, we are recognising a gap in care and naturally risk-stratifying patients, so the greatest benefit goes to those with the most need, with the resources available. The application of this rule across the VA could result in reduced costs for diabetes care and increased costs for psychosocial support. Future studies should be done to identify such factors, assessing whether their management results in better outcomes than routine primary care.
When incorporated into our already-existing reporting mechanism, providers can receive the model results specific to each patient during, or in preparation of, the clinic visit. With these predictions, a provider can assess the patient’s values and motivation in treatment intensification. If interested, they can set a goal based on the patient’s age and comorbidities, discussing treatment options. If the patient is still above goal, then psychological, social and cultural assessment can be performed to identify treatment barriers. If psychological screening indicates a psychiatric or personality disorder or substance abuse, the provider should refer the patient to behavioural health. Alternatively, if social assessment indicates an occupational problem or homelessness, the provider should refer the patient to social service. If a cultural barrier is identified, the patient should be referred to a support group for culturally sensitive recommendations.
As evidenced by the models retaining time HbA1c≥75 mmol/mol, consecutive number of HbA1c values of ≥75 mmol/mol, baseline HbA1c value and receiving insulin treatment, it appears intermediate and short-term factors play an important part on whether a patient with an HbA1c≥75 mmol/mol will have subsequent HbA1c values of <75 mmol/mol. Fortunately, modifiable factors such as diet, exercise, treatment regimen and recent change in health status (eg, pneumonia) are captured among these, highlighting their importance. Since time-weighted average HbA1c entered the model predicting HbA1c measurement ever <75 mmol/mol, but not the model predicting final HbA1c measurement <75 mmol/mol, another predictor variable potentially is more statistically, but not clinically, significant. Also noteworthy, minority status changed directions across the models. Our cohort has 4738 (3.58%) patients whose qualifying HbA1c during baseline was their first documented HbA1c in the VA. Future studies should be conducted to further elucidate these relationships.
In our review of the literature, we found only one other paper reporting a prediction rule for HbA1c in patients with diabetes.15 Investigators of that study followed up 684 European patients with type 2 diabetes mellitus 12 months from clinical appointments; their dependent variable was ‘an observed increase in terms of HbA1c %≥0.5’. While we considered higher-risk patients to have baseline HbA1c≥75 mmol/mol, their patients had microvascular complications with a median (range) HbA1c of 55 mmol/mol (48–72 mmol/mol). The authors of that paper do not report sensitivity, specificity, positive predictive value or negative predictive value to allow comparison of model performance to ours.
Limitations
This study has several limitations. First, a significant Hosmer-Lemeshow indicates the potential presence of a systematic pattern of bias, such as model mis-specification; however, several criticisms exist about the Hosmer-Lemeshow test, including its use in large datasets as any small departure between observed and predicted frequencies is magnified.16 Although we do not know whether bias exists, a c-statistic is acceptable at the threshold of 0.7. Therefore, the c-statistics for each outcome indicate a satisfactory probability of correctly classifying a randomly selected pair of cases from those who did and did not attain HbA1c<75 mmol/mol. Second, although we have studied the population of patients with diabetes with an HbA1c≥75 mmol/mol, the findings may not reflect patients receiving care in other healthcare systems. For example, a comparison of demographics and health characteristics of the 2013 National Health Interview Survey suggested more similarity with VA patients and Medicare beneficiaries than patients with employer-based health insurance or Medicaid beneficiaries.17 Third, as with all secondary database analyses, the results depend on data entry of people independent of the study. To address this, the investigators applied clinical knowledge and understanding of the processes generating the data when dealing with issues of data integrity. Finally, no prediction rule should replace clinical judgement, especially when factors not available in a data source may play an important role. Rather, the findings are meant to inform the conversation between patient and provider. An improvement of this model can be the development of candidate predictor variables of modifiable patient characteristics. In particular, effects of various therapeutic classes will be helpful as the current ADA guidelines recommend many options, dependent on patient preferences and target HbA1c.4