Article Text

Personalised prediction of maintenance dialysis initiation in patients with chronic kidney disease stages 3–5: a multicentre study using the machine learning approach
  1. Anh Trung Hoang1,
  2. Phung-Anh Nguyen2,3,4,
  3. Thanh Phuc Phan5,6,
  4. Gia Tuyen Do1,7,
  5. Huu Dung Nguyen1,
  6. I-Jen Chiu8,9,10,
  7. Chu-Lin Chou9,10,11,12,
  8. Yu-Chen Ko13,
  9. Tzu-Hao Chang14,
  10. Chih-Wei Huang14,15,
  11. Usman Iqbal16,17,
  12. Yung-Ho Hsu8,9,10,11,
  13. Mai-Szu Wu8,9,10 and
  14. Chia-Te Liao8,9,10
  1. 1Nephro-Urology and Dialysis Center, Bach Mai Hospital, Hanoi, Vietnam
  2. 2Clinical Data Center, Office of Data Science, Taipei Medical University, Taipei, Taiwan
  3. 3Clinical Big Data Research Center, Taipei Medical University Hospital, Taipei, Taiwan
  4. 4Research Center of Health Care Industry Data Science, College of Management, Taipei Medical University, Taipei, Taiwan
  5. 5International PhD program of Biotech and Healthcare Management,College of Management, Taipei Medical University, Taipei, Taiwan
  6. 6University Medical Center, Ho Chi Minh City, Vietnam
  7. 7Department of Internal Medicine, Hanoi Medical University, Hanoi, Vietnam
  8. 8Division of Nephrology, Department of Internal Medicine, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
  9. 9Division of Nephrology, Department of Internal Medicine, School of Medicine, College of Medicine, Taipei Medical University, Taipei, Taiwan
  10. 10TMU-Research Center of Urology and Kidney (TMU-RCUK), Taipei Medical University, Taipei, Taiwan
  11. 11Division of Nephrology, Department of Internal Medicine, Hsin Kuo Min Hospital, Taipei Medical University, Taoyuan City, Taiwan
  12. 12Division of Nephrology, Department of Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan
  13. 13Division of Cardiovascular Surgery, Department of Surgery, Shuang Ho Hospital, Taipei Medical University, New Taipei City, Taiwan
  14. 14Graduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical University, Taipei, Taiwan
  15. 15International Center for Health Information Technology, College of Medical Science and Technology, Taipei Medical University, Taipei, Taiwan
  16. 16School of Population Health, Faculty of Medicine and Health, University of New South Wales (UNSW), Sydney, New South Wales, Australia
  17. 17Global Health & Health Security Department, College of Public Health, Taipei Medical University, Taipei, Taiwan
  1. Correspondence to Dr Chia-Te Liao; ctliao19386{at}tmu.edu.tw; Professor Mai-Szu Wu; maiszuwu{at}tmu.edu.tw

Abstract

Background Optimal timing for initiating maintenance dialysis in patients with chronic kidney disease (CKD) stages 3–5 is challenging. This study aimed to develop and validate a machine learning (ML) model for early personalised prediction of maintenance dialysis initiation within 1-year and 3-year timeframes among patients with CKD stages 3–5.

Methods Retrospective electronic health record data from the Taipei Medical University clinical research database were used. Newly diagnosed patients with CKD stages 3–5 between 2008 and 2017 were identified. The observation period spanned from the diagnosis of CKD stages 3–5 until the maintenance dialysis initiation or a maximum follow-up of 3 years. Predictive models were developed using patient demographics, comorbidities, laboratory data and medications. The dataset was divided into training and testing sets to ensure robust model performance. Model evaluation metrics, including area under the curve (AUC), sensitivity, specificity, positive predictive value, negative predictive value and F1 score, were employed.

Results A total of 6123 and 5279 patients were included for 1 year and 3 years of the model development. The artificial neural network demonstrated better performance in predicting maintenance dialysis initiation within 1 year and 3 years, with AUC values of 0.96 and 0.92, respectively. Important features such as baseline estimated glomerular filtration rate and albuminuria significantly contributed to the predictive model.

Conclusion This study demonstrates the efficacy of an ML approach in developing a highly predictive model for estimating the timing of maintenance dialysis initiation in patients with CKD stages 3–5. These findings have important implications for personalised treatment strategies, enabling improved clinical decision-making and potentially enhancing patient outcomes.

  • Machine Learning
  • Primary Health Care
  • Preventive Medicine
  • Patient Outcome Assessment
  • Electronic Data Processing

Data availability statement

Data may be obtained from a third party and are not publicly available.

https://creativecommons.org/licenses/by/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

  • Early prediction of dialysis initiation in patients with chronic kidney disease (CKD) is invaluable for tailoring personalised treatment plans. Despite several prediction models that have been developed, they almost lack sufficient accuracy and fail to include several crucial factors related to CKD progression.

WHAT THIS STUDY ADDS

  • Developing a machine learning predictive model that incorporates comprehensive clinical data and standardises the selection of the index date may lead to even more accurate predictions. So far, this study has involved the largest patient enrolment and the broadest range of clinical parameters.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

  • These findings enable highly accurate estimation of the timing for initiating maintenance dialysis in patients with CKD stages 3–5. As a result, they have significant implications for personalised treatment strategies, facilitating improved clinical decision-making and potentially enhancing patient outcomes.

Introduction

Chronic kidney disease (CKD) and end-stage renal disease (ESRD) are significant global health problems that burden healthcare systems worldwide. In Taiwan, the overall prevalence of CKD was 8.2%, and the incidence of treated ESRD was 529 per million population, according to the US Renal Data System annual report published in 2021.1 International guidelines recommend referring patients with CKD to nephrology for pre-ESRD care upon reaching an advanced stage to improve the quality of care and reduce costs.2 3 One critical component of pre-ESRD care is counselling patients on choosing kidney replacement therapy (KRT) following shared decision-making. It may involve preparing vascular access for haemodialysis (HD) at least 6 months before HD initiation, placing a peritoneal dialysis (PD) catheter at least 2 weeks before PD initiation, or identifying suitable donors for pre-emptive kidney transplantation before dialysis is required to replace failing kidney function. Therefore, personalising the timing of referral for maintenance dialysis initiation is essential for each patient. However, early personalised estimation of the optimal timing for maintenance dialysis initiation presents a significant challenge for patients with CKD. This decision relies not only on the glomerular filtration rate (GFR) level but also on symptoms of uraemia syndrome and the ability to manage complications such as electrolyte imbalance, acid–base disturbances and fluid overload through medical treatment.3–5

Numerous traditional and artificial intelligence (AI) models have been developed and evaluated to estimate the duration until the initiation of KRT among patients with CKD.6–10 Among these models, machine learning (ML) approaches that use complex computer algorithms are effective in identifying the most critical factors and developing predictive models with superior performance.11 12 However, existing models rely primarily on laboratory data and comorbidities to inform their analyses and predictions, neglecting important indicators of CKD progression such as an annual decline in GFR, proteinuria and medication use. Furthermore, the timing (index date) chosen for these models is heterogeneous due to the diverse stages of CKD represented in the patient cohorts, resulting in relatively low predictive power. Therefore, developing an accurate and reliable predictive model that incorporates comprehensive clinical data and standardises index date selection is crucial to improving personalised decision-making for patients with CKD.

In this study, we aim to develop and validate an ML model for early personalised prediction of maintenance dialysis initiation within 1-year and 3-year timeframes among patients with CKD stages 3–5 using a multicentre longitudinal cohort.

Materials and methods

Study data source

We conducted a retrospective analysis of the Taipei Medical University (TMU) clinical research database (TMUCRD), which comprises comprehensive medical claims data for patients across three affiliated hospitals: TMU Hospital (TMUH), Wan Fang Hospital (WFH) and Shuang Ho Hospital (SHH). The TMUCRD encompasses structured and unstructured data for over 4 million patients, spanning the period from 1998 to 2021. All data were fully anonymised prior to analysis, with patients’ identity codes and medical facility information scrambled to ensure patient privacy. This study was authorised by the joint institutional review board (IRB) committee of TMU (IRB#: N202105032).

Study population

We identified patients diagnosed with CKD between 1 January 2008 to 31 December 2017, based on the International Classification of Disease, ninth revision (ICD-9) code 585 and 10th revision (ICD-10) code N18. First-time patients diagnosed with CKD stages 3–5 and who had not undergone KRT, defined as the index date, were included in the study. We confirmed the diagnosis based on the G-stages, in which the estimated glomerular filtration rate (eGFR) was calculated using the CKD-EPI creatinine equation.13 To define correctly the stage of CKD, only patients who had been diagnosed with CKD prior to the index dates and had their eGFRs observed within 3–6 months after the index date were included. Patients who were younger than 20 years at the time of diagnosis of CKD stages 3–5 or underwent pre-emptive kidney transplantation were excluded from the study (figure 1).

Figure 1

Enrollment process of the first study. The index date is the point of time at which patients with CKD were first diagnosed with stages 3–5. CKD, chronic kidney disease; eGFR, estimated glomerular filtration rate; KRT, kidney replacement therapy.

Observation period

Patients were followed up from the time of stages 3–5 CKD diagnosis (ie, the index date), and their data were censored at the start of maintenance dialysis, loss to follow-up, termination of insurance or the end of the study period (ie, 3 years from the index date). Additionally, patients who died before initiating KRT during the follow-up were excluded from the study.

Outcome measurement

The primary outcome of this study was the initiation of maintenance dialysis among patients with CKD stages 3–5. The initial dialysis point was defined as the first day of maintenance dialysis treatment (eg, the first day of long-term HD, the day of catheter insertion for PD or the first day of PD) based on the procedure-related codes under Taiwan’s National Health Insurance (online supplemental table S1). The patients who underwent maintenance dialysis were defined as ‘receiving dialysis’ and others as ‘living without dialysis’.

Supplemental material

Variables and data processing

Patient demographics, comorbidities, medications and laboratory data were collected from the database within 1 year before the diagnosis date. The major comorbidities were identified using diagnostic codes (ICD-9 and ICD-10) from outpatient and inpatient databases. The analysis included all diseases listed in the Charlson Comorbidity Index (CCI),14 along with additional conditions such as essential hypertension, glomerular diseases, lipid metabolism disorders and septicaemia. These diseases were considered confirmed if at least one outpatient or inpatient visit was documented within 1 year prior to the diagnosis date.

The TMUCRD contains comprehensive information on prescribed medications from three affiliated hospitals. Patients’ medication prescription claims were tracked using the Anatomical Therapeutic Chemical codes for 1 year preceding the diagnosis date of CKD stages 3–5 (online supplemental table S2).

We also retrieved some routine blood tests from laboratory datasets, including haemoglobin (Hgb), white blood cells (WBC), neutrophils, platelets (PLT), blood urea nitrogen (BUN), creatinine (CREA), cholesterol (CHOL), triglyceride (TG), albumin, calcium, phosphorus, sodium and potassium. The average value of each blood test was calculated based on the results collected within 1 year before the diagnosis date. Blood tests that had over 50% missing values were excluded from the analysis. To manage the missing continuous features, we used the Multiple Imputation by Chained Equations method to fill in these gaps in the data.15 The eGFR was measured at the diagnosis date (the baseline eGFR) and compared with the previous eGFR, from which the decline of eGFR was calculated using the formula:

The decline of eGFR = (Previous eGFR–Baseline eGFR)/the day interval

In addition, we also collected urine tests and classified albuminuria based on albuminuria categories according to the Kidney Disease Improving Global Outcomes (KDIGO) classification13 (online supplemental table S3). Patients with missing values on albuminuria were defined as the group ‘unknown’.

Modelling

The classification models were used to predict the initial dialysis, including logistic regression, linear discriminant analysis, gradient boosting machine (GBM), light GBM, AdaBoost, random forest (RF), extreme gradient boosting machine (Xgboost) and artificial neural networks (ANN). A detailed description of different models and their parameters is shown in online supplemental appendix S1.

Model training and testing

To ensure robust model development and account for sample selection bias, we divided the dataset into two parts: the training set and the test set. The training set consisted of patient data from two hospitals, TMUH and WFH, and was used for model development. To evaluate the performance of different ML models and estimate generalisation errors, we applied the stratified fivefold cross-validation method within the training set. This involved dividing the patients into five groups while ensuring that each group represented a proportional distribution of patient characteristics. Each group was then used as the internal validation set for one of the five replications. On the other hand, the test set comprised patient data obtained from SHH and served as an independent dataset for external model validation.

Statistical analysis and evaluation of model performance

Continuous variables were provided as the mean±SD, and categorical variables were provided as absolute (n) and relative (%) frequency; it is described in table 1.

Table 1

Demographic characteristics of patients with CKD stages 3–5

The area under the receiver operating characteristic (ROC) curve (AUC), accuracy, sensitivity (recall), specificity, positive predictive value (precision), negative predictive value and F1 score were computed to evaluate and compare the performance of all prediction models. The model with the highest AUC was selected as the best model through comparison using the external testing set. Furthermore, the impact of features in the best model was analysed using Shapley Additive exPlanations (SHAP) values.16

Data processing was conducted using SQL Server Management Studio V.18.6 (Redmond, Washington, USA), while model training and testing were performed using Python V.3.8.8 software (Wilmington, Detroit, USA) with scikit-learn V.1.1 (Paris, France).

Results

Data extraction

We identified 16 244 eligible patients with CKD stages 3–5 who were diagnosed for the first time in three TMU-affiliated hospitals from 1 January 2008 to 31 December 2017. Among those, we excluded 9703 patients due to being younger than 20, having no medical histories a year before the index date or lacking evidence for the diagnosis of CKD. Furthermore, we excluded 438 and 1282 patients for the 1-year and 3-year prediction models, respectively, who had died prior to undergoing maintenance dialysis. Finally, 6123 and 5279 patients who met all inclusion criteria were included in model development for 1-year and 3-year prediction performances, respectively (figure 1).

Study population characteristics

The patients’ demographics, comorbidities, medications and laboratory data were summarised in table 1. The study population was over 65 years, with a mean (SD) ages of 71.7 (13.2) and 70.5 (13.1) years for the 1-year and 3-year prediction models, respectively. Most patients were male (59.1%) and had baseline G-stages of G3 (eg, 56% for the training set and 43% for the testing set). Patients had a high prevalence of comorbidities such as diabetes (51%), hypertension (61%), disorders of lipid metabolism (47%) and glomerular disease (24%). The outcome of initiation of maintenance dialysis was observed at 10.5% and 24.1% for the 1-year and 3-year prediction models, respectively.

The performances of different prediction models

The performances of different prediction models are shown in table 2. For a 1-year prediction of successful dialysis treatment, the highest AUC value of 0.96 was observed for the ANN model (ie, sensitivity, 0.88; specificity, 0.75; precision, 0.39 and F1 score, 0.6), followed by the GBM and RF models with an AUC value of 0.89. Likewise, the ANN model was performed with a better AUC value of 0.92 (ie, sensitivity, 0.87; specificity, 0.79; precision, 0.63 and F1 score, 0.73) than other ML models in the 3-year prediction of receiving maintenance dialysis. The ROC curves of varying prediction models for 1-year and 3-year successful dialysis treatment are shown in figure 2.

Table 2

Summary of different classification models

Figure 2

The performance of the prediction models in the testing dataset. (A and C) 1-year prediction with machine learning and ANN models; (B and C) 3-year prediction with machine learning and ANN models. ANN, artificial neural network; AUC, area under the receiver operating characteristic curve; LGBM, light gradient boosting machine; XGB, extreme gradient boosting.

Features importance

The lists of the top 20 important features that might impact the prediction model’s performance for 1-year and 3-year successful dialysis are shown in figure 3. The essential features of the 1-year and 3-year follow-up models were baseline eGFR, BUN, creatinine, triglyceride, age, gender, Hgb, CHOL, PLTs, albuminuria, diabetes disease, hypertension and related medications (eg, diuretics, insulin, dipeptidyl peptidase-4 and calcium channel blockers).

Figure 3

Feature importance of the ANN prediction model. (A) 1-year prediction model and (B) 3-year prediction model. ANN, artificial neural network; eGFR, estimated glomerular filtration rate; DPP-4, dipeptidyl peptidase

Discussion

Our study findings demonstrated that ML classification models are well suited for a meaningful prediction of the initiation of maintenance dialysis in patients with CKD stages 3–5. The ANN method showed a better performance level for 1-year and 3-year prediction of dialysis commencement with a higher AUC (0.96 and 0.92), good sensitivity (0.88 and 0.87) and specificity (0.75 and 0.79).

In previous studies, AI models have been applied to predict CKD progression and start KRT. In 2015, Jamshid Norouzi et al used an adaptive neuro-fuzzy inference system to predict renal failure progression. Their model could accurately (>95%) predict the GFR for 6-month to 18-month intervals. However, only 465 patients with CKD were included in their study, and it was noted that proteinuria was not an important feature in their model.8 In 2019, Jing Xiao et al developed ML models to predict CKD progression. Their model used only the patient’s demographics and biochemical blood features, not features derived from a urinalysis. Besides, the predictive power of the model was not high (AUC: 0.873, sensitivity: 0.83 and specificity: 0.82).17 Another model was performed using only comorbidity data from 8492 patients to predict the onset of KRT, and their results were even lower (AUC, sensitivity and specificity were only 0.773, 0.623 and 0.781, respectively).7 Recently, Qiong Bai et al also conducted an ML model to predict the risk of ESRD. This model selected many important factors associated with the progression of CKD, including demographics, blood tests and comorbidities, but not proteinuria. However, the predictability has not improved compared with previous models.18 It could be explained by the fact that many patients in that study were in the early stages of CKD, resulting in a low percentage of those who progressed to ESRD when followed for a short period of time. The imbalance in the outcome can significantly affect the model’s predictive power.

In this study, we only focused on patients with CKD stages 3–5, and their risk of progression to ESRD is high. Hence, predicting the time of their dialysis commencement is very practical in our daily clinical care. Moreover, we carefully identified the model features associated with CKD progression and KRT based on the clinical setting and traditional logistic regression analysis. Forty-five significant prognostic factors were selected, including patient demographics, comorbidities, routine blood and urine tests and commonly used medications. Therefore, the predictive ability of our model has higher accuracy.

In further analysis of the ANN model, we also ranked all predictors according to their influence on the 1-year and 3-year models using SHAP values.19 Notably, several distinct features have been identified, respectively. For example, age, comorbidity (CCI score), PLT counts and WBC counts were important contributing factors in the 1-year prediction model, whereas gender and other medications such as proton pump inhibitors, beta-lactam antibacterial agents, organic nitrates and H2-receptor antagonists were relevant factors in the 3-year model. Common important factors identified in both models included eGFR at baseline, blood urea, serum creatinine and albuminuria (see figure 3). These are also key determinants for the risk classification of CKD according to the 2012 KDIGO guidelines.13 20 Other contributing factors in both models included serum Hgb level, TG or CHOL levels, hypertension, diabetes mellitus, diuretic use, antihypertensive agents and medications for controlling blood glucose levels (see figure 3). Anaemia typically develops during the course of CKD; a decrease in serum Hgb is significantly associated with the progression of CKD.21 22 Diabetic nephropathy is the leading cause of ESRD in adults.23 24 In patients with diabetic CKD, blood glucose levels are associated with poor outcomes such as serum creatinine doubling, ESRD and mortality, and intensive glycaemic control could reduce these risks.25–29 Additionally, several studies have demonstrated that certain levels of dyslipidaemia is independently associated with rapid renal progression, KRT, all-cause mortality and cardiovascular death in predialysis patients.30–33 Hypertension may occur early during the course of CKD and is related to a more rapid decline of kidney function, the development of cardiovascular disease and death in patients with CKD.34 35 Early intervention and tight control of blood pressure could lessen the risk of CVD and all-cause death in patients with and without CKD.36 37 Diuretics are an important part of guideline-directed medical therapy for patients with CKD with hypertension, oedema and hyperkalaemia.38 In terms of adverse effects, whether diuretics are an independent risk factor for CKD progression remains controversial. However, these medicines played important roles in both our models.39–41 Therefore, diuretics should be used with caution in patients with CKD stages 3–5. Finally, the GFR decline rate is also influenced by some immutable patient factors. The Kidney Disease Outcomes Quality Initiative guideline has provided ample evidence that African-American race (not justified in this study), male gender and older age are related to a more rapid GFR reduction.20 In summary, our models take advantage of the important factors involved in the progression of CKD, are consistent with current clinical practice guidelines and are highly applicable. They could be a good screening tool to determine the likelihood of initiating long-term dialysis by using the available clinical data on the patient. Several limitations need to be addressed. First, due to the patient’s lack of weight and height, the body surface area was not adjusted for eGFR. As a result, the determination of G-stage using unadjusted eGFR may be inaccurate for oversized patients. Second, using the decline in eGFR between baseline eGFR and previous eGFR may not accurately capture the progression of CKD when compared with the annual decline in eGFR during the follow-up period. Consequently, this factor did not significantly contribute to our model. Third, we only used retrospective data from three hospitals in Taipei to create our models, and it is widely recognised that racial and regional variables also influence CKD progression. Further work should involve training and validating the models through multinational and multiracial data before the clinical application is generalised. Fourth, we incorporated all important features into the prediction model, acknowledging that this approach might not be practical for clinical implementation. Nevertheless, these features underwent meticulous screening and hold varying degrees of significance in relation to CKD progression. Additionally, we assessed the model using only the top 10 important features and obtained comparable results (online supplemental tables S4 and S5, online supplemental figures S1 and S2).

Conclusion

We have shown that using the machine learning approach can develop a highly predictive model for estimating the timing of maintenance dialysis initiation in patients with CKD stages 3–5, which provides a further step towards personalised treatment in this population.

Data availability statement

Data may be obtained from a third party and are not publicly available.

Ethics statements

Patient consent for publication

Acknowledgments

The authors thank the staffs at Office of Data Science, Taipei Medical University for their assistance in data collection and processing.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • ATH and P-AN contributed equally.

  • M-SW and C-TL contributed equally.

  • Contributors THA, CTL, Y-HH and M-SW conceptualised the study and wrote the first draft. THA, PAN and C-TL completed and edited the final manuscript. THA, PAN, PPT, I-JC, C-LC and T-HC collected and assimilated necessary data. CTL is guarantor.

  • Funding This research was funded by Taipei Medical University (TMU109-AE1-B31) and the Higher Education Sprout Project by the Ministry of Education (MOE) in Taiwan.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.