Limitations and areas of uncertainty

Many of the studies included in this paper are not necessarily generalisable to other geographies or demographics—this would require additional exploration and research. In a similar vein, the relationships outlined by these studies are only relevant to the data sets applied within said research; that is, the specific data sets used by the study reported. As such, it would be imprudent to infer that the results of the studies detailed in this paper indicate clinical applicability on their own. Rather, they offer guidance for promising avenues of further research with larger and more diverse data sets in specific patient populations, and/or by modifying the algorithmic methodology outlined to further improve accuracy. Finally, many studies use different AI techniques to analyse data or statistical methods for reporting data, which poses some limits on comparison of results between studies.

A note on units of measurement and definitions

Studies that examine AI suicide prediction models use different units of measurement when reporting results. Given this is a nascent area of research, it is not always possible to find studies that share the same units of measurement for comparison. Definitions are included here to provide context to the reader, with explanations relevant to their use within this paper.

• AUC (area under the receiver operator characteristics curve): AUC assesses chance‐level discriminative accuracy. AUC takes into account both true positives and true negatives. An AUC of 1.0 equates to a model with perfect discriminative accuracy, while an AUC of 0.5 means that the model produces results with the same accuracy as chance.

• Accuracy: measured by comparing the computed result (positive or negative) against its true value.

• Precision (also known as positive predictive value): precision reflects the proportion of positive results in a model that are true positives.

• Sensitivity (also known as recall): the proportion of total true positives that were registered as positive by the model.

AI for medical suicide prediction

With the proliferation of electronic medical records (EMRs), there is now a wealth of health data available. When linked with other data sources, analysis of these complex sets of information (known colloquially as ‘big data’) can provide a snapshot of the biological, social and psychological state of a person at one time. Machines can learn to detect patterns, which are indecipherable using traditional forms of biostatistics, by processing big data through layered mathematical models (AI algorithms). Algorithms can be designed to correct and learn from mistakes (training) to add the accuracy of an AI predictive model confidence—this is called machine learning.6 As such, AI—and machine learning more specifically—is well positioned to address the challenge of navigating big data for suicide prediction.

Results across multiple studies indicate that AI consistently outperforms doctors at predicting suicide completion and suicide attempts, highlighting the promise of AI-based medical suicide prediction. Research suggests a promising clinical application for AI in identifying risk of suicide completion. Kessler et al used machine learning protocols (Naive Bayes, random forests, support vector regression) to predict suicide completion among military veterans within 26 weeks following an outpatient mental health visit. The study demonstrated an AUC of 0.72 for those with prior hospitalisation for psychiatric issues, 0.61 for those without hospitalisation and 0.66 when both samples were combined. Relevant characteristics of hospitalisation and previous outpatient visits included suicidality, depression, bipolar disorder and non-affective psychosis. Interestingly, AUC improved to 0.75 when predicting suicide death within 5 weeks of the outpatient visit.7

In 2018, a study by Del Pozo-Banos et al used artificial neural networks (a type of machine learning technique) to analyse routinely collected information in EMRs to assess suicide risk in patients attending health services for any reason.8 Using only EMR and hospital data in the 5 years prior to a patient dying by suicide, the model accurately matched control patients and suicide cases (that is, whether patients committed suicide or not) with an accuracy over 73%. The authors noted that more complex models incorporating more data points would likely yield better results, and such a model will be built in the next stage of experimentation.

AI has achieved high accuracy when predicting suicide attempts, too. By applying machine learning to EHRs, Walsh et al created machine-learning algorithms (random forest and logistic regression) that achieved AUC values of 0.80-0.84when predicting whether a suicide attempt was likely to occur within the next 2 years and within the next week, respectively . Depression with psychosis, schizophrenia and prior suicide attempt were classified as important predictors in long and short term prediction.9 Ryu et al used a machine learning technique (random forest) to predict suicide attempts among those with suicidal ideation. The prediction model achieved strong results, with an AUC of 0.947 and accuracy of 88.9%.10 It is important to note that the clinical applicability of these tools in the real world remains unproven; however, initial results are extremely promising.11

Results across multiple studies indicate that AI consistently outperforms doctors at predicting suicide completion and suicide attempts, highlighting the promise of AI-based medical suicide prediction. One could imagine a future where initial screening tools such as those proposed by Del Pozo-Banos et al and Walsh et al/Ryu et al are combined to give an extremely accurate picture of an individual’s suicide risk.8 9 In turn, this could be used to inform treatment options for high risk patients.

The Department of Veteran’s Affairs in the USA is putting medical suicide prediction into practice. Rates of suicide among US military veterans are 1.5 times greater than those who have not served, even when adjusting for age and gender. In an effort to close this gap, the Recovery Engagement and Coordination for Health—Veterans Enhanced Treatment (REACH VET) programme uses an AI to examine millions of records on medications, treatment, traumatic events, overall health and other information. It then identifies veterans most at risk of suicide. Initial results have been impressive: those classified by the algorithm in the top 0.1% of risk were 15 times more likely to complete suicide in the next year, and 81 times more likely to attempt suicide in the next year, than the average veteran. Following risk assessment, clinicians then establish contact with at-risk veterans to offer resources and support, as well as an optional psychological consult. In the first year since implementing the programme, there were 250 less suicides (a 4% reduction) than what would have been expected from previous rates. While it is difficult to tell whether the REACH VET programme specifically contributed to this reduction, the Department has commissioned an independent evaluation of the programme’s effectiveness, and will look to expand the use of predictive analytics and share risk data to improve the AI’s modelling in coming years.12 13

An important question is what should be done when individuals are identified as being at risk of suicide. For example, hospitalisation may be the right step for some, but could cause more harm than good in other patients. Furthermore, forcibly detaining patients in a hospital or other medical setting could cause significant psychological stress and potentially hasten future suicide attempts. Identifying which types of treatment should be used for which patients is a valuable area of future research.

AI for social suicide prediction

A growing number of researchers and technology companies are using AI to monitor suicide risk through online activity. This builds on emerging evidence that language patterns on social media and methods of smart phone use can indicate psychiatric issues.14

A large number of studies have demonstrated the potential efficacy of applying social media to predicting suicide risk.15–22 In most cases, natural language processing is used to analyse the online activity of users on social media platforms for suicidal behaviours (such as a mention of suicide attempt, suicidal ideation, or discussion of suicidal themes). This may be combined with machine learning techniques to compare and contrast findings across and within platforms—for example, to determine patterns of behaviour and how this may relate to risk severity.

In the vast majority of studies examining the use of AI to predict suicidal behaviours on social media, it is not possible to verify against ‘ground truth’. That is, it is not possible to use medical records to determine whether an individual posting on social media has actually experienced what they are describing on the platform. Where verification against medical records is not possible, medical professionals with expertise in suicide can verify the likely veracity of user claims. This is, of course, based on their subjective, professional judgement. Some higher quality studies only include cases where there is unanimous agreement by medical professionals that the individual in question is legitimately at risk of suicidal behaviour; these cases are then included in a ‘gold standard’ sample to assess an AI model’s predictive power. One example is Gaur et al, where Reddit posts were examined for uses of suicidal language to determine suicide risk. Different clinical classification schemes were compared against machine learning techniques, including random forest and convolutional neural networks. Convolutional neural networks was the strongest performer, achieving an overall precision of 70%—40% better than baseline approaches that only applied medical classification systems.23

A landmark study published in Biomedical Informatics Insights by Coppersmith et al combined many of the insights of previous studies in this area. Coppersmith et al applied natural language processing and supervised and unsupervised machine learning methods to social media data from a variety of sources (eg, Facebook, Twitter, Instagram, Reddit, Tumblr, Strava and Fitbit, among others)—for which they were granted permission by test subjects—in order to determine the risk of attempted suicide. AUC was 0.89–0.93 for time periods ranging from 1 month to 6 months in length.24 As outlined by Coppersmith et al, i if a false alarm rate of 1%-2% is assumed, this model may be up to 10 times more accurate at correctly predicting suicide attempts when compared with clinician averages (40%–60% vs 4%–6%).25 ,24 26

Coppersmith et al cautioned that these results focused on an 18–24 age group of mostly American women, so may not be generalised to other demographics, cultures or norms. For example, stigma in different communities may influence whether people post about suicide on social media. Nonetheless, initial evidence suggests comparable results for men and lesbian, gay, bisexual, transgender, intersex, and questioning (LBGTIQ) people—albeit in a small sample size.More broadly, the model’s high accuracy in determining suicide risk, with access only to social media data, suggests a promising avenue for further research.

It is worth noting that the analytical tools deployed by Coppersmith et al are likely to be far less advanced and granular than those being undertaken by Facebook, Google, Twitter and other technology companies (which will be outlined later in this paper). This is on account of these firms having access to rich troves of online user data and cutting-edge analytical techniques. Given that these companies have not provided their results or techniques for independent evaluation (as will be discussed), it is not possible to draw further inferences. Yet, as more data becomes available through public forums, and algorithms for analysing this data advance, social suicide prediction is likely to yield significantly more accurate and clinically useful results than those described by Coppersmith et al.

Further, and finally, it is worth noting that the analytical power of such tools could be leveraged to enhance medical suicide prediction efforts—all that is required is that at-risk patients provide consent to access social media data. Padrez et al demonstrated the feasibility of such an approach; when asked in a hospital emergency setting, 37% of 2717 Facebook and/or Twitter users consented to share both their health record and social media data for the purpose of data linkage.27 Patient sensitivity around suicide and mental illness information may mean lower rates of consent in this cohort. However, the potential clinical usefulness of combining medical and social suicide prediction tools means that this topic deserves future research and consideration.

Suicidal ideation

A study by Lin et al examined the effectivities of machine learning techniques in detecting suicidal ideation based on six psychological stressors in EMRs. This study of Taiwanese military men and women used machine learning techniques, including logistic regression, decision trees, random forest, gradient boosting regression tree, support vector machine and multilayer perceptron; all machine learning methods achieved accuracies over 98% in predicting suicidal ideation. When compared with conventional clinical criterion for assessing the presence of suicidal ideation, the algorithms improved sensitivity by more than 35% and precision by 65%.28 In another study, researchers used a machine-learning algorithm (Naive Bayes) to identify those at risk of suicide ideation with 91% accuracy, based on their altered functional MRI neural signatures of death-related and life-related concepts.29

Turning to social media, Tadesse et al outlined a number of machine learning approaches for identifying suicidal ideation on Reddit using convolutional linguistics. One model, using a combination of long short-term memory and convolutional neural networks, achieved an accuracy and precision of 93% in identifying users with suicidal ideation.30 Ji et al found comparable results, demonstrating that machine learning techniques could leverage statistical, linguistic, word embedding and topic features to achieve 90% accuracy in identifying suicide ideation on Reddit and Twitter.31

Despite these promising results, the utility of identifying suicidal ideation may be limited due to low positive predictive value and modest sensitivity for suicide attempts. This is due to the low incidence of suicide attempts when compared with incidence of suicidal ideation.26 32 Saying this, such tools may still be useful. Many patients with suicidal ideation may not be willing to disclose this fact to their doctors, meaning that tools which can predict suicidal ideation based on psychological stressors could be valuable to medical practitioners, particularly when dealing with high risk populations such as military personnel—in turn, these tools could be combined with those which predict suicide attempts and completed suicide, as outlined above, to increase their clinical applicability. In this fashion, algorithms that deliver advantages in precisely identifying those who may be at risk of suicidal ideation could help to provide targeted care to more patients who are in need, and with subsequent benefits for the efficient allocation of scarce medical resources.

Mental illness + AI: prediction and diagnosis

While only a small fraction of those with mental illness die from suicide, more than 80% of people who die from suicide are thought to have mental illness. Risk increases for patients with multiple comorbid mental illnesses.26 As a result, there is clinical interest in better understanding the risk of mental illnesses in patients who may also be at risk of suicide (prediction) and correctly identifying mental illness when it is present (diagnosis).

One of the limitations of current psychiatric diagnosis of mental illness is that many conditions overlap with each other—at least 50% of patients receive more than one psychiatric diagnosis.33 AI prediction tools in medical settings could provide better diagnostic clarity, thus improving treatment efficacy in patients and reducing the impact of unnecessary side effects. As such, many researchers are excited by the potential of AI to improve access to mental health services and drive down the cost of diagnosis—particularly in rural/remote and low-income settings.34 35

A full analysis of the opportunities for using AI to predict mental illness is beyond the scope of this paper. However, examples of AI’s potential to predict and diagnose mental illness include the following.

Suicide amongst adolescents

A number of promising studies have analysed the potential for AI to predict suicide attempts among adolescents. Jung et al applied machine learning algorithms to a nationally representative sample of nearly 60 000 Korean adolescents to determine risk of suicide via history of suicide attempts/ideation. Taking into account 26 predictors of suicide risk, five different models (logistic regression, random forest, support vector machine, artificial neural network and extreme gradient boosting) achieved an accuracy between 77.5% and 79%.55 Walsh et al conducted a retrospective cohort study of 33 000 adolescents to predict suicide attempts; random forests achieved AUC values >0.80 across time frames that ranged from prediction windows of 7 days to 2 years.56 Finally, Bhat and Goldman-Mellor used deep neural networks to predict suicide attempts among Californian adolescents using a sample of over 500 000 medical records. The strongest performing model of the experiment achieved a sensitivity of 70%, specificity of 98% and AUC of 0.958.57

Non-suicidal self-injury and self-harm

Non-suicidal self-injury (NSSI) is defined as deliberate direct destruction or alteration of body tissue without conscious suicidal intent.58 Deliberate self-harm is an encompassing term for self-injurious behaviour, both with and without suicidal intent, that has a non-fatal outcome.59 NSSI has been linked to increased risk of severe self-harm and suicide attempts.60Ammerman et al used lasso regression (a type of regression with regularisation) and random forests to analyse NSSI patterns among 712 undergraduate students. Findings demonstrated that suicide plans and depression, both risk factors for suicide, were significant predictors of lifetime NSSI risk.61 Using a sample of 359 undergraduate students with a history of NSSI, Burke et al attempted to determine which NSSI factors were most salient to suicide risk.62 Three machine learning techniques (elastic net regression, decision trees, random forests) were used to determine that motivations, method lethality and scarring are likely the most important factors in ascertaining suicide risk. Further research is required to analyse the replicability of these results with larger sample sizes and across different geographies and age groups.

Physical illness

The presence of physical illness has been found to contribute to suicide risk.63 A study by Karmakar et al aimed to quantify the impact of a history of physical illness on suicide risk by using machine learning techniques to analyse EMR data of 7399 mental health patients with a history of physical illness. The best performing machine learning model combined data across all time periods to significantly outperform clinical baseline risk assessment in predicting suicide risk (AUC of 0.71 vs AUC of 0.56).64 This infers that AI suicide prediction tools are likely to be more effective when history of physical illness is taken into account.