Protocol•

Using algorithms to initiate needs-based interventions for people on antipsychotic medication: implementation protocol

Abstract

Introduction Non-adherence to antipsychotic medications for individuals with serious mental illness increases risk of relapse and hospitalisation. Real time monitoring of adherence would allow for early intervention. AI2 is a both a personal nudging system and a clinical decision support tool that applies machine learning on Medicare prescription and benefits data to raise alerts when patients have discontinued antipsychotic medications without supervision, or when essential routine health checks have not been performed.

Methods and analysis We outline two intervention models using AI2. In the first use-case, the personal nudging system, patients receive text messages when an alert of a missed medication or routine health check is detected by AI2. In the second use-case, as a clinical decision support tool, AI2 generated alerts are presented as flags through a dashboard to the community mental health professionals. Implementation protocols for different scenarios of AI2, along with a mixed-methods evaluation, are planned to identify pragmatic issues necessary to inform a larger randomised control trial, as well as improve the application.

Ethics and dissemination This study protocol has been approved by The Southern Adelaide Clinical Human Research Ethics Committee. The dissemination of this trial will serve to inform further implementation of the AI2 into daily personal and clinical practice.

Background

Schizophrenia and other serious mental illnesses (SMI) such as bipolar disorder and schizoaffective disorder, are fast emerging as one of the world’s most important health problems, with worldwide prevalence rates ranging from 0.8% to 6.8%.1 The Global Burden of Diseases, Injuries and Risk Factors Study 2010 revealed that SMIs were among the top 10 causes of disability, directly accounting for more than 7.4% of disease burden worldwide.2 In Australia, it is estimated that 2%–3% of the population (~600 000 people) are living with an SMI3 and facing numerous barriers to accessing and using health services.4 Treatment and management of SMI requires a multidisciplinary approach and communication between General Practice (GP), emergency departments, specialists, pharmacies, community clinics and allied health services.

Reducing risk of relapse and hospitalisation remains one of the greatest challenges in the treatment of SMI, in particular for people with schizophrenia, with an estimated 80% of this population reported to have relapsed multiple times within the first 5 years of initial treatment or remission from their index episode.5 Mental illness stigma constrains the use of available resources, as do inefficiencies in the distribution of funding and interventions. This combination of stigma and structural discrimination contributes to social exclusion and breaches of basic human rights of individuals with mental disorders.6 Detecting when people with SMI stop medication is a challenge given limited resources and suboptimal medication monitoring. Early detection of medication non-adherence is important to prevent: recurrence of negative symptomatology, relapse resulting in harm to self and others, decreased response to future treatment and for people with schizophrenia specifically neuro-degeneration.7

Information technology has the potential to improve effectiveness in the way people are monitored, treated and followed up8 and enhance self-efficacy in health management.9 For clinical decision support tools, patient-specific assessments or recommendations can play a critical role in improving prescribing practices, reducing serious medication errors, enhancing the delivery of preventative care services and improving adherence to recommended care standards.10 In a systematic review of 70 clinical studies, decision support systems significantly improved clinical practice in 68% of trials.10 Additionally, computer-based access to complete pharmaceutical profiles and alerts reduces the rate of initiation of potentially inappropriate prescribing, therapeutic duplication, excessive medication and the resulting adverse drug-related events.11 Such systems also enable information exchange within clinical teams, assisting in managing demand for health services and lowering direct medical costs for consumers.12

Consumer-facing information technology can provide pragmatic, accessible and scalable mobile health interventions.13 Furthermore, it has been suggested that the use of eHealth technologies allows individuals to be more proactively involved in health management, which ultimately leads to a greater likelihood of optimal healthcare outcomes.14

One of the most important factors for the successful implementation of such systems in healthcare is user’s acceptance and use of that technology. One of the major factors leading to the failed uptake of these systems is an inadequate understanding of the sociotechnical aspects, especially the understanding of how individuals and organisations adopt new technology.15 That is, a disparity between the model of healthcare ascribed by these systems and the actual nature of healthcare often result in decreased organisational approval. Sociotechnical theory provides a model against which system implementation into workflow can be better understood.

Rationale

There is considerable potential for the use of digital analytics systems to improve the monitoring of people with SMI between periods of illness, as well as to improve the overall health and well-being for people with SMI. However, a strong evidence base is essential before specific approaches are implemented. The aim of this study is to describe two different scenarios of AI2, and a protocol for a feasibility pilot of these use-cases in order to gather data and uncover pragmatic issues necessary to inform a larger randomised control trial, as well as improve the application.

Method

AI2 application

AI2 is a cloud-based application that sources GP appointments, laboratory tests, prescription and dispense records from Medicare data view in Australia’s national electronic health record, known as My Health Record (MyHR). Medicare data is only uploaded on the MyHR every 3–4 weeks so the system is near-real time. The AI2 algorithms systematically consider the combined effects of a set of prognostic factors (such as non-adherence to medication schedules or the absence of an appointment that are then grouped and mapped by service provider and prescription type16) to estimate the level of risk an individual with SMI has in relation to relapse, or if a patient has deviated from their individualised care trajectories. These visualisation algorithms are displayed on an internet-based dashboard, and are traffic light colour coded (red=potential high-risk urgent action required, yellow=potential moderate-risk, action required, green=unlikely risk, no action required). The application was developed using open-source hosting and development tools, Java Enterprise Edition using JBoss Seam and Hibernate Frameworks. For an overview of ethical workflow issues associated with this transfer of data, see Bidargaddi et al.16

Trial plan

Two use cases of the AI2 platform will run concurrently; in the first use case (patient-signup, a personal nudging system), patients will receive text messages directly when red flags are detected. In the second use-case (clinic-signup, a clinical decision support tool), the flags are presented as a dashboard for community mental healthcare clinics allowing, healthcare professionals to better prioritise their workload by focusing on patients most at risk (red/orange flags) while reducing interactions with those who are managing better (green flag). Summary of study designs for patient-signup and clinic-signup use cases are displayed in figure 1.

Figure 1
Request permissionrequest permission icon
Figure 1

Flow chart of study design comparisons of patient-signup and clinic-signup use case.

Use case 1: patient-signup—a personal nudging system

Recruitment

Patient participants must satisfy the following inclusion criteria; >18 years of age; have been diagnosed with SMI as defined by the Diagnostic and Statistical Manual of Mental Disorders17 by a psychiatrist; have sufficient command of the English language; be eligible for Medicare benefits (eg, an Australian citizen or permanent resident). A person will be ineligible to participate if their treating healthcare professional determines they are unable to provide informed consent themselves or with the assistance of a family member or an authorised representative. A stepwise recruitment strategy will be utilised to recruit patients in mental health settings in metropolitan, rural and remote South Australian locations.

Eligible individuals will be approached by their treating healthcare professional during their scheduled visit, where the healthcare professional will and state that participation in the trial is voluntary ask that the patient to carefully read over the patient information sheet and consent form. To avoid feelings of coercion, patients will be invited to take the information sheet and consent form home before deciding whether they wish to participate.

Design

Patient participants will be randomised by computer software with randomisation probability set at 50/50 to one of two care groups: (1) intervention (50%; where participants receive a clinician-mediated text message following an alert) or (2) control group (50%; usual care)(figure 2). Alerts will be automatically generated by the AI2 system as described above. The decision to send an alert is decided by psychiatrists within the study team acting as ‘monitors’. The psychiatrists are trained on how to use the AI2 system, how to correctly interpret the alerts and how to specify appropriate intervention pathways when participants are identified at risk of relapse and hospitalisation. The participant will be sent a text message from a bank of individually configurable messages.

Figure 2
Request permissionrequest permission icon
Figure 2

Flow chart of study design of patient-signup use case.

Procedure

After consenting to participate, patients must sign up to the AI2 system online providing the necessary information (Name, DOB, Medicare number) to enable the AI2 system to pull their Medicare data from MyHR system. Participants must also agree to fill in a post study questionnaire. Once the trial commences, the participant may or may not receive SMS alerts depending on their adherence to medication and care plans.

Outcomes

To assess the usefulness of the SMS nudges and experience with the AI2 system, patient recorded experience measures via both questionnaires and focus groups will be administered. Additionally, patients will complete the Medical Interview Satisfaction Scale18 at the end of the trial to assess their level of satisfaction with their healthcare professional in relation to distress relief, communication, rapport and compliance intent.

We will assess proportions of nudges that resulted in patient’s picking up a script or a GP appointment within 2 weeks of nudge. This information will be useful to estimate sample size and power calculations for future trials.

Use case 2: clinic-signup—clinical decision support tool

Recruitment

Healthcare clinics (rural and metropolitan) including community mental health, inpatient, primary care and /or hospitals or pharmacies will be approached to participate in the research. The head of each site must agree to the research and sign the required consent forms. Once the clinic has agreed to participate in the research, the study coordinators will run information sessions at participating sites to recruit individual clinicians. Information sessions will provide an overview of the AI2 study design, rationale, aims and intended outcomes and answer any question to recruit participating clinicians for each site.

Design

We will use interrupted time-series analysis—a quasi-experimental design that can evaluate an intervention effect by using retrospective and longitudinal health data.19 Enrolled health professionals will be trained to view and attend to flags of their patients displayed on the AI2 application dashboard during the intervention time period (12 months) and compare the indicators during these 12 months with the prior 12 months. Patients will not be randomised and will all receive the same level of care. Refer to figure 3 for this study design.

Figure 3
Request permissionrequest permission icon
Figure 3

Flow chart of study design for clinic-signup use case.

Procedure

Clinics that agree to participate in the trial will provide the research team with the necessary patient information (Name, DOB, Medicare number) to enable the AI2 system to pull their Medicare data from MyHR system. To participate in the research each clinician must provide online informed consent prior to gaining access to AI2. Importantly, AI2 does not replace or change usual care, it provides additional information. Healthcare professional participants will be provided with training on how to use the AI2 system and how to correctly identify and interpret the algorithms. Each clinic will be guided to discuss how the clinic can best incorporate AI2 into clinic practice, how best to use the information provided by AI2 and how best to intervene when patients are identified at risk of relapse and hospitalisation (red alert). After the healthcare professional has evaluated and reviewed an alert, they will record their response to the alert in the AI2 system. This might include no action taken, or details of communication with a patient. The AI2 generated green alerts, indicating ongoing adherence, can also be used when reviewing patients ready for potential discharge from community services to aid as a decision-making tool.

Outcomes

To assess the usefulness of AI2 in improving decisions of health professionals, we will calculate the proportion of the alerts generated by AI2 that were actioned and deemed useful by health professionals. To help with sample size calculations for a larger scale trial, differences between healthcare process and outcome metrics of patients of enrolled health professionals during the 12 months trial period with the prior 12 months period will be calculated by an interrupted time series analysis. Using linkages with health records, we will derive healthcare process, patient trajectory and outcome metrics for comparison purposes.

To evaluate the usability and acceptability of AI2 and to understand the implementation issues associated with new methods of healthcare delivery and decision support systems, consent for implementation research will be sought from participating clinicians to record, transcribe and analyse meetings and focus groups. The qualitative data will be analysed using an existing consolidated framework.20 Additionally, uptake and usage parameters will be automatically sourced by AI2 to understand uptake and patterns of use.

At the end of 12 months, all healthcare professionals involved in the intervention will be asked to fill in The Unified Theory of Acceptance and Use of Technology Scale21 that assesses their acceptance of AI2 on dimensions including: performance expectancy, attitude towards using technology, social influence, facilitating conditions and self-efficacy, anxiety and behavioural intention to use the system.

Discussion

This study will allow for the exploration of pragmatic issues that will inform a larger randomised control trial and enables us to assess the suitability of AI2 in two different use cases and, if necessary, identity how it can be refined and tailored for both clinical and consumer use.

Strengths

Current models of care for SMI are reactive. AI2, a unique third-party application based on Australia’s national electronic record, provides clinicians with the information and the opportunity to intervene early to prevent relapse and hospitalisation. Lack of connectivity between community and hospital information systems can result in ineffective handovers of medication schedules.22 Many individuals with SMI need regular monitoring and support to ensure they adhere to their medication and treatment plans. Although over 30% of people with SMI are managed by GPs, there is (1) no regular monitoring system for people managed in general practice23 and (2) GP’s are often not confident in treating or managing people with severe mental illness.24 Because of concerns over fear or relapse, community mental health professionals often delay referring people to primary care.24 As such, AI2 can support better strategic relationship to facilitate more effective initial assessments, care planning and transition of care.

In addition to the clinical application of AI2, the data collected in the course of this trial can, once de-identified, form the basis of a South Australian Mental Health Registry based on essential attributes recommended for clinical quality registries.25 There are a number of benefits to using a prospective design, most prominently that the investigative team will be able to study the process of implementation in real time rather than retrospectively, as most studies have done.26 Further, this study design allows for demonstrating the temporal sequence between the policy and the resulting outcomes.

Limitations

No power analysis was conducted for these pilot trials due to the fact that this intervention can have an effect on several different health process and health outcome metrics, and these effects are also moderated/mediated by the extent of health professional and patient participation in the intervention process. Given this complexity, we believe through the proposed pilot study we will be able to observe measures that are most likely to change in a realistic way. This would provide data needed to do sample size calculation for a realistic outcome measure, which is another objective of this pilot study. Additionally, diverging subjective interpretations of AI2-generated alerts by healthcare professionals is a potential source of bias, which will be mitigated by hands-on system training before the start of the trial.

Innovation

Innovative applications play an essential role in improving health outcomes and consumer-centred healthcare services through the use of electronic health records.16 AI2 is the first of its kid of analytic software in Australia, and national authorities are recognising the value of third-party applications for better engaging consumers in their healthcare and for aiding healthcare professionals with digital decision support tools.

Overview

Article metrics
Altmetricopen-url
Dimensionsopen-url