Original Research•

Designing and validating a clinical decision support algorithm for diabetic nephroprotection in older patients

Abstract

Background Older patients with diabetic kidney disease (DKD) often do not receive optimal pharmacological treatment. Current clinical practice guidelines (CPGs) do not incorporate the concept of personalised care. Clinical decision support (CDS) algorithms that consider both evidence and personalised care to improve patient outcomes can improve the care of older adults. The aim of this research is to design and validate a CDS algorithm for prescribing renin-angiotensin-aldosterone system inhibitors (RAASi) for older patients with diabetes.

Methods The design of the CDS tool included the following phases: (1) gathering evidence from systematic reviews and meta-analyses of randomised clinical trials to determine the number needed to treat (NNT) and time-to-benefit (TTB) values applicable to our target population for use in the algorithm. (2) Building a list of potential cases that addressed different prescribing scenarios (starting, adding or switching to RAASi). (3) Reviewing relevant guidelines and extracting all recommendations related to prescribing RAASi for DKD. (4) Matching NNT and TTB with specific clinical cases. (5) Validating the CDS algorithm using Delphi technique.

Results We created a CDS algorithm that covered 15 possible scenarios and we generated 36 personalised and nine general recommendations based on the calculated and matched NNT and TTB values and considering the patient’s life expectancy and functional capacity. The algorithm was validated by experts in three rounds of Delphi study.

Conclusion We designed an evidence-informed CDS algorithm that integrates considerations often overlooked in CPGs. The next steps include testing the CDS algorithm in a clinical trial.

What is already known on this topic

  • The care of older adults could be improved through the utilisation of clinical decision support (CDS) algorithms that consider both evidence and personalised care, aiming to enhance patient outcomes. Current clinical practice guidelines (CPGs), which do not incorporate the concept of personalised care, often result in suboptimal pharmacological treatment for older patients with diabetic kidney disease.

What this study adds

  • We designed an evidence-based CDS algorithm that integrates considerations often overlooked in CPGs, including life expectancy, functional status and personalised goals of therapy. The CDS algorithm covered 15 possible scenarios, and produced 36 personalised and 9 general recommendations.

How this study might affect research, practice or policy

  • This study introduces a validated CDS algorithm for personalised prescribing to older adults with varying health profiles. The algorithm integrates personalised care criteria and evidence-based metrics derived from a meta-analysis. It provides general and tailored recommendations to guide prescribing decisions for older adults with diabetes, enhancing patient-specific treatment. This novel approach not only influences diabetes care but also establishes a framework for designing similar algorithms for different health conditions among older populations, benefiting research and practice.

Introduction

Diabetic kidney disease (DKD) is the leading cause of kidney failure, responsible for approximately 40% of incident cases.1 2 To prevent and control DKD, clinical practice guidelines (CPGs) recommend prescribing renin-angiotensin-aldosterone system inhibitors (RAASi) to slow the progression of the disease.3–7 Generally, older adults are less likely to receive antihypertensive therapy as recommended by CPGs for different reasons, which include the preventative nature of some indications, cost, questionable efficacy and safety in older adults, and limited clinician–patient relationship.8–10 Moreover, although there is mounting evidence on the importance of prescribing RAASi in patients with diabetes, 57% of older patients in the USA with an indication for RAASi were not receiving it, while 62% of those on a RAASi received suboptimal doses.11 12

DKD poses an example of the decision-making challenges when prescribing medications for older patients due to patient complexity and the number of possible treatment options. These challenges have resulted in the presence of a gap between evidence-based medicine (EBM) and practice.13–16 CPGs are produced and promoted by many organisations as a means to reduce this gap.17 Despite these efforts, clinicians are hesitant to apply those recommendations to older patients because they are aware that these guidelines are often not appropriate for this population.18 19 Furthermore, applying the recommendations of different CPGs to each comorbidity of older patients can result in a multitude of conflicting recommendations,20 potentially leaving the ultimate decision to expert opinion, rather than EBM. Consequently, older patients are at high risk of receiving suboptimal or inappropriate treatments.21 22

CPGs commonly face the challenge of primarily employing explicit criteria without accounting for patients’ unique characteristics or embracing personalised care. Personalised care for older adults may involve assessing functional status, caregiver support, genomics and patient preferences. Consequently, treatment plans hinge on risk-to-benefit ratios tailored to each patient.23 Clinical decision support (CDS) systems can facilitate the integration of both EBM and personalised care, optimising therapy for patients. Studies indicate that CDS tools enhance overall clinical practice, guideline adherence, patient therapy compliance, reduce polypharmacy and enhance patient safety.24 25

In this project, we aimed to design a novel CDS algorithm that merges EBM and personalised care concepts to support decision-making related to the use of RAASi in older adults with diabetes. The process of developing this CDS algorithm was studied, applied and validated, which can inform the development of further CDS algorithms for medication use in older patients.

Research objectives

The objective of this study was to develop and validate a CDS algorithm for clinicians and patients to help decide when to start, resume or stop RAASi for the prevention of DKD in older patients with diabetes. The purpose of this manuscript is to discuss one part of a large project to design and validate the CDS algorithm, and it focuses on the building of the algorithm.

Methods

The study consisted of six phases: (1) building the foundation of the algorithm; (2) the systematic review (SR) and meta-analysis (MA); (3) the extraction of CPGs’ recommendations; (4) exploring healthcare providers (HCPs)’ input; (5) the design of the CDS algorithm; and (6) the validation of the CDS algorithm. In this paper we focus on results from phases 5 and 6.

Foundation of the algorithm

To design the algorithm, we followed the American Geriatrics Society (AGS) approach for treating older patients with multi-morbidity.26 This approach emphasises considering a patient’s unique factors based on their daily living activities and independence level, categorising them into a functional status group, and estimating their life expectancy.26

To achieve a balance between EBM and AGS’s personalised care principles, the other aspects of the CDS algorithm focused on medication-related factors drawn from relevant randomised clinical trials (RCTs) and CPGs. These factors encompassed the number needed to treat (NNT) and time-to-benefit (TTB) for RAASi medications. NNT gauges a medication’s effect size, while TTB measures the time required for a statistically significant benefit to emerge in RCTs comparing therapy recipients to a control group.26 27

Systematic review and meta-analysis

We conducted an SR/MA to identify and summarise all relevant RCTs on the role of RAASi in DKD.28 In summary, we searched all published RCTs that studied the renal protective effects of RAASi in patients with diabetes, extracting information from 46 studies. We used the results of our SR/MA to calculate NNT and TTB for each clinical case in the algorithm.28

Extraction of CPGs’ recommendations

The selection of the guidelines recommendations was carried out through a survey of six HCPs from different disciplines (medicine, pharmacy and nursing). The participants were recruited through the researchers’ professional network in Canada, and they worked in different areas of specialisations (nephrology, endocrinology and geriatrics). The survey asked the participants to rank conflicting recommendations that cover the same area of clinical care from different CPGs. The most highly ranked recommendations then were used in building the CDS algorithm in the next phases.

We conducted a search for CPGs on DKD, CKD and diabetes in PubMed, Canadian Medical Association Infobase, G-I-N International Guideline Library, Google Scholar and NICE databases from January 2010 to September 2021. Additionally, we manually sought relevant CPGs from professional organisations in nephrology, endocrinology, geriatrics and family practice. Inclusion criteria comprised CPGs published in English within the past 5 years or the latest versions of landmark CPGs, aligning with the Institute of Medicine’s definition of CPGs.29

A single researcher (NA) performed the search, while two researchers (J-PL and NA) screened CPGs against inclusion criteria, removing duplicates and older editions. Recommendations regarding RAASi prescription for DKD prevention or slowdown were extracted. An extraction form facilitated categorisation, incorporating recommendation strength and evidence level.

Quality assessment using the International Appraisal of Guidelines, Research and Evaluation (AGREE II) checklist and platform involved two researchers (NA and SL). Only guidelines with an average rigour score exceeding 50% were considered.30

To select guideline recommendations, a survey involving six HCPs from diverse fields (medicine, pharmacy and nursing) was conducted. These participants, recruited from the authors’ professional network, specialised in nephrology, endocrinology and geriatrics. They ranked conflicting recommendations covering the same clinical care area from various CPGs, with the highest-ranked ones informing the CDS algorithm development in subsequent phases.

Exploring the key components of the CDS algorithm according to HCPs

We conducted an anonymous, online, cross-sectional survey of Canadian HCPs who were affiliated with a provincial regulatory body, nephrology organisation or advocacy body. A 59-item questionnaire composed of a mix of question types was designed to obtain HCPs’ perspectives about the use of CDS tools in their daily clinical practice.31

Design of the CDS algorithm

Case definitions and building

We built a list of all possible potential cases based on the presence or absence of the following conditions: albuminuria, CKD and hypertension. The cases also addressed the prescribing questions for these cases on starting, adding or switching to RAASi in patients with controlled or uncontrolled hypertension. Next, we screened the inclusion criteria of the 46 RCTs of the SR to identify which RCT can answer the clinical question of each case, and matched each RCT with one of the cases (table 1).

Table 1
|
The definitions of clinical cases and the corresponding trial(s)

Calculating NNT and TTB for each case

Data extraction was performed for 46 RCTs in the SR and MA. Statistical analysis was performed to calculate the effect size for each individual RCT then to calculate the pooled effect size for subgroups of RCTs that matched specific clinical cases. We reported the results as ORs, which were then used to calculate NNT for each clinical case in the algorithm using Moore’s method.32 33 The TTB was calculated using the statistical process control method as illustrated by van de Glind et al.34 While NNT provided the probability of the benefit from the treatment, TTB identified the time frame to witness this benefit. We used the NNT and TTB to estimate specific cut-points for the benefit of using RAASi for each case. Cut-points were used to generate the personalised recommendations based on patients’ estimated life expectancy and functional status, which were estimated by the Lee Index and the Barthel Index for Activities of Daily Living, respectively.35 36 For example, patients with short life expectancy (below a specific cut-point of years identified by TTB) and with poor functional status would probably not benefit from medications with long TTB and high NNT. Table 2 shows the NNT and TTB for each clinical case.

Table 2
|
The studies used in determining the pooled effect sizes, NNT and TTB for each clinical case

Validation of the CDS algorithm

We conducted a Delphi study to determine the content validity of the first draft of the proposed CDS tool.37 A series of questionnaires and controlled feedback is used as part of the communication between the researchers and the group of experts, where experts show their level of agreement on specific concepts, phrases or decisions.37 Based on the feedback the statements are revised and the survey is re-distributed along with individual positions.37

Panel experts participated anonymously in the process using LimeSurvey – an online survey programme. Each panellist was provided access to all of the relevant reference articles resulting from the MA, which form the evidence base for each recommendation in the CDS draft. Each participant was asked to indicate their level of agreement on each recommendation at each decision point of the proposed algorithm using a 5-point Likert scale, where 1=strongly agree; 2=agree; 3=neutral; 4=disagree; and 5=strongly disagree. The median and IQR values were calculated for each item. Recommendations with a median value of <3 and a 75th percentile value <3 were included in the algorithm, while recommendations with a higher median value were modified and used for the subsequent rounds of the Delphi process. A second and third rounds of the Delphi process were conducted using the modified recommendations developed following the first round.

Results

Systematic review and meta-analysis

In the MA, evidence was drawn from 26 551 patients with diabetes from 46 studies published between 1991 and 2016. The majority of trials recruited patients from outpatient clinics, and included patients with type 2 diabetes mellitus. The trials were mostly conducted on patients with microalbuminuria at baseline and patients with normal kidney function. The mean follow-up of the studies was 36 months (range 12–72). The mean sample size of all studies was 577 patients (range 50–5231). The average age of the patients was 51 years (range 28.7–82.5).28

The studies were categorised based on their inclusion criteria and study outcomes into categories as in online supplemental table 1. This MA provided the required data from 33 studies on the efficacy of RAASi which was used to calculate NNT and TTB values applicable to our target population for the use in the algorithm.

Extraction of CPGs’ recommendations

The electronic search retrieved 165 documents pertaining to different CPGs. After duplicate removal, the CPGs were screened for eligibility according to the prespecified inclusion criteria, and 23 were included as follows: 9 CPGs on CKD, 6 for diabetes, 5 for hypertension and 3 for primary care (table 3).

Table 3
|
A list of Included clinical practice guidelines

Exploring the key components of the CDS algorithm according to HCPs

According to the published abstract of the survey study,38 63 participants from various disciplines (nephrology, family medicine, geriatrics and internal medicine) completed the questionnaire. Over 80% of respondents agreed that patient willingness to adhere to therapy, remaining life expectancy and functional capacity were pivotal in prescribing decisions for older adults. Most participants also expressed agreement with the importance of validating CDS tools and providing supporting evidence to facilitate their use in daily practice.

These study findings influenced the CDS algorithm’s design as follows: each CDS recommendation included supporting evidence and ORs with 95% CIs. Justifications for the recommendations were presented in both professional and lay terms to promote shared decision-making. Personalised recommendations took into account the patient’s life expectancy, level of independence, and therapy goals.

Design of the CDS algorithm

We built 16 clinical cases. The clinical cases of the CDS algorithm were categorised into two types: those with matching RCTs, and those without matching RCTs. The former are the ones that were represented by at least one RCT from the SR (table 1). We generated 36 personalised recommendations and 9 general recommendations for 9 clinical cases. NNT and TTB were calculated and incorporated in the personalised recommendations of the clinical cases and justifications were added in professional and lay languages. A list of the recommendations generated by the first draft of the CDS algorithm are presented in online supplemental table 2. The latter type denotes cases of patients that were not represented by any of the RCTs from the SR. These cases were added to the CDS algorithm and were provided prescribing recommendations from the CPGs (online supplemental table 3).

Validation of the CDS algorithm

Our study recruited a panel of 17 healthcare professionals The demographics of the expert panel are provided in online supplemental table 4.

Delphi round 1

Experts were given a link to a 54-question online questionnaire covering general and personalised recommendations for clinical cases. They offered input on recommendation content and wording. Based on their responses, 35 statements reached consensus and were included in the final algorithm draft, while 19 did not achieve consensus on content or wording and were moved to the second round. Online supplemental table 5 displays the median and IQR for each statement. Experts generally agreed on recommending RAASi for patients with microalbuminuria or macroalbuminuria, and for those with controlled or uncontrolled hypertension. However, consensus was not reached for statements advising against RAASi for patients with normoalbuminuria and hypertension, or those with microalbuminuria and no hypertension. Recommendations were modified or entirely changed for statements lacking consensus, with justifications and relevant CPGs provided in Delphi round 2.

Delphi round 2

Twelve experts participated in round 2, which consisted of 19 questions concerning statements of the CDS algorithm. Based on the responses of the experts, consensus was not reached for nine statements for either content or wording or both, and therefore were moved to round 3. Online supplemental table 6 shows the median and IQR for each statement.

Modifications were made to the recommendations, so that we provided more neutral language. We provided information to the experts that we acknowledged that there are variations in clinical practice, but we stressed that our purpose was to obtain their responses based on their assessment of the evidence provided. Also provided were the CPG’s recommendations as well as the justifications for each recommendation.

Delphi round 3

Round 3 involved nine experts providing input on nine statements. Ultimately, a consensus was reached for all nine statements, concluding the validation process for the CDS algorithm (figure 1). Each statement’s median and IQR are shown in online supplemental table 7.

Figure 1
Request permissionrequest permission icon
Figure 1

Flow diagram of Delphi rounds.

Design of the final draft of the CDS algorithm

The validated recommendations of the CDS algorithm were incorporated into the final draft of the CDS algorithm in a decision-tree with justifications of the recommendations (online supplemental table 8 and figure 2).

Figure 2
Request permissionrequest permission icon
Figure 2

Clinical decision support algorithm for RAASi-related Prescribing Decisions for older adults with diabates.

Discussion

This article provided the rationale and details of the design and validation process of a novel CDS algorithm designed to help in making RAASi-related prescribing decisions for older adults with diabetes for the prevention and management of DKD. The final version of the algorithm included 9 general recommendations and 36 personalised recommendations considering the variation of life expectancies and functional capacities among older adults.

This pioneering CDS algorithm incorporates personalised care parameters using NNT and TTB values from an MA of RCTs. It leverages a wide range of RCTs as its knowledge base, accommodating various scenarios with differing albuminuria levels, hypertension and kidney disease. Developed by a team of experts in geriatrics, nephrology, pharmacoepidemiology and patient-reported outcomes, it also draws insights from clinicians across disciplines via surveys and Delphi studies. This interdisciplinary approach shaped the algorithm’s design systematically and structurally.

Acknowledging the lack of guidelines or standard methods for CDS algorithm development, referring to prior research offers insights. Lefebvre et al crafted nine deprescribing CDS algorithms for haemodialysis patients employing Lynn’s three-step approach.39 40 They conducted a systematic literature review to address research queries and engaged multidisciplinary experts for content development.39 Farrell et al developed four deprescribing CDS algorithms.41–44 The algorithms were designed using a guidelines development approach through applying the Grading of Recommendations, Assessment, Development and Evaluation process.45 The same research group developed an instruction manual to guide the efforts of future researchers interested in the development of deprescribing CDS algorithms.46

A SR conducted by Souza-Pereira et al provided examples of CDS algorithms used to manage chronic diseases.47 The most addressed chronic diseases were diabetes, cardiovascular diseases and pain. According to their review, most of the algorithms focused on the follow-up management by providing treatment recommendations or guidelines on the management of the disease. The SR showed that most of the algorithms were developed using databases to feed the algorithms, and they were designed with the help of software engineers.47 Yan et al conducted a review on the CDS algorithms that were developed for inpatient pharmacy services.48 The review discussed studies that compared the CDS algorithm interventions to control and showed that around 80% of the evaluated CDS algorithms were associated with statistically significant favourable outcomes compared with control. These outcomes included correct doses based on renal function, the detection of drug–drug interactions and reduced length of stay.48

What distinguishes this CDS algorithm from the previous examples, is that beside providing a structured approach to prescribing, our CDS algorithm will also enable prescribers to incorporate personalised care concepts into their decisions. That is, the algorithm requires the prescriber to consider a variety of patient factors, including goals of therapy, quality of life, remaining life expectancy, and functional capacity. Considering these factors in the care of older adults is crucial, as older adults are heterogeneous in terms of these factors.26

As with other interventions, CDS algorithms aim to improve patients’ care and standardise practice. While algorithms and protocols are helpful, they are not a substitute for clinical expertise.49 It is normal for prescribers to disagree with some of the guidelines or algorithms. Despite this, clinical decision-making should follow a transparent, consistent approach.50 The rationale for deviating from guidelines and algorithms should be well-founded and properly documented. By providing documentation, CDS algorithms could be optimised in the future.50

This algorithm inherits limitations from the MA it was derived from, covering RCTs spanning over 20 years with methodological variations. Inclusion criteria led to diverse participant characteristics, managed through subgroup and sensitivity analysis. Generalisability may be limited as the expert panel were exclusively from Canada. Regular updates and adaptation of the CDS algorithm are essential to ensure relevance.

Conclusion

We designed and validated a novel CDS algorithm for RAASi-related decisions for older patients with diabetes. The phased design ensures a validated algorithm, incorporating older patient-centred care concepts and clinician input on prescribing decisions. Future research should assess its performance in clinical trials, aiming for realistics outcomes, reduced healthcare costs and maintained quality of life in older adults. Enhancing accessibility involves integrating it into a web-based decision support tool, facilitating use by HCPs.

Overview

Article metrics
Altmetric data not available for this article.
Dimensionsopen-url