The three fairness pillars, their attributes and relation to ML-based prediction of inpatient violence in psychiatric settings
| Pillar | Attribute | Relation to predictive care |
| Transparency | Interpretability | ML models achieve high accuracy in predicting violent behaviour in psychiatric settings.38 If these models achieve similar performance in new settings, they would be considered interpretable. However, if models are biased (ie, generating more false positives for inpatients defined by certain features), interpretability would be maintained even if biases carried forward to new samples.63 |
| Explainability | ML models are often trained on structured risk assessment scores.38 Scores may be biased against certain groups (eg, recent immigrants due to language barriers or cultural miscommunications), leading to biased models. Pairing predictions with feature explanations can lead clinicians to over-rely on ML models,78 which can exacerbate adverse impacts when models are biased. | |
| Accountability | ML models have been trained on actigraphy features to predict aggression in patients with dementia.178 However, patients should not be expected to advocate for themselves if models seem biased or are not generalisable, given their particularly vulnerable status. | |
| Impartiality | Provenance | Prior conviction and a diagnosis of schizophrenia are predictors of violence.38 179 Training models on these features could lead to certain groups being disproportionately classified as high-risk (eg, black men, due to residing in more policed areas,180 or being more likely misdiagnosed with schizophrenia181. Since these features are linked to other predictors, removing them does not remove model bias, nor does it address the social and political realities contributing to bias in the training data.111 182 |
| Implementation | ML modelling of violence risk is in part motivated by a desire to allocate staff resources to high-risk patients, but staff-patient interactions are known antecedents to violent behaviours.183 Most patients classified as high-risk do not become violent;40 however, pre-emptive interventions involving interactions with staff could precipitate violent behaviours. | |
| Inclusion | Completeness | A focus on legally protected categories may disregard biases related to unobserved characteristics (eg, sexual orientation or disability). Individuals with invisible or undiagnosed disabilities (eg, autism spectrum disorder) may display behaviours interpreted as precursors to violence or aggression.184–186 Additional marginalised groups might emerge when intersectional identities are taken into account. |
| Patient and family engagement | Collaboration in decision making during admission and maximising choice are important values for patients in settings where autonomy is limited.187–189 Patients may prioritise other aspects of care not captured by ML (eg, the caring relationships built with staff and peers, as compared with therapeutic interventions).190 |
ML, machine learning.