Original research•

Information processing by community health nurses using mobile health (mHealth) tools for early identification of suicide and depression risks in Fiji Islands

Abstract

Objectives High rates of depression and suicide and a lack of trained psychiatrists have emerged as significant concerns in the low-income and middle-income countries (LMICs) such as the Pacific Island Countries (PICs). Readily available smartphones were leveraged with community health nurses (CHNs) in task-sharing for early identification of suicide and depression risks in Fiji Islands, the largest of PICs. This investigation examines how CHNs can efficiently and effectively process patient information about depression and suicide risk for making diagnostic and management decisions without compromising safety. The research is driven by the theoretical framework of text comprehension (knowledge representation and interpretation) and decision-making.

Methods Mobile health (mHealth) Application for Suicide Risk and Depression Assessment (ASRaDA) was designed to include culturally useful clinical guidelines for these disorders. A representative sample of 48 CHNs was recruited and presented with two clinical cases (depression and suicide) in a simulated setting under three conditions: No support, paper-based and mobile-based culturally valid guideline support. Data were collected as the nurses read through the scenarios, ‘thinking aloud’, before summarising, diagnoses and follow-up recommendations. Transcribed audiotapes were analysed using formal qualitative discourse analysis methods for diagnostic accuracy, comprehension of clinical problems and reasoning patterns.

Results Using guidelines on ASRaDA, the CHNs took less time to process patient information with more accurate diagnostic and therapeutic decisions for depression and suicide risk than with paper-based or no guideline conditions. A change in reasoning pattern for nurses’ information processing was observed with decision support.

Discussion

Although these results are shown in a mental health setting in Fiji, there are reasons to believe they are generalisable beyond mental health and other lower-to-middle income countries.

Conclusions Culturally appropriate clinical guidelines on mHealth supports efficient information processing for quick and accurate decisions and a positive shift in reasoning behaviour by the nurses. However, translating complex qualitative patient information into quantitative scores could generate conceptual errors. These results are valid in simulated conditions.

Summary

What is already known?

  • Smart mobile phones have supported healthcare applications by health care providers in lower to middle-income countries.

  • The use of smart mobile phones can support automated calculations to reduce the total number of errors.

  • Although there have been experiments in mental health settings, their impact has not been rigorously evaluated.

What does this paper add?

  • The information processing involves filtering irrelevant information from the relevant, contributing to a lower cognitive load on the providers, thereby making the process efficient.

  • Safety may be compromised at the expense of efficiency due to conceptual errors even though the total number of errors is reduced.

  • The use of culturally appropriate guideline support on mobile phones can result in positive behavior change.

  • Although these results are shown in mental health settings, there are reasons to believe they are generalised beyond mental health.

Introduction

Mental health (MH) disorders are globally recognised as a significant public health concern yet receive inadequate attention, and more than 80% of people experiencing mental illness live in low-income and middle-income countries (LMICs.1

Pacific Island Countries (PICs) are classified as LMIC, where in recent years, depression and suicide have been reported to have reached an alarming level.2 With fewer than 20 qualified psychiatrists serving more than 11 million people, this situation is only likely to worsen with pandemic and post-pandemic-related conditions. The PICs alone account for 25% of the global burden of disease: Death by suicide comprises approximately 25% of the total disease burden,3 and depressive disorders approximately 5.7%.1

To increase coverage and access to evidence-based mental healthcare, community health nurses (CHNs), the front-line healthcare workers, assume a significant role in the early identification of depression and suicide risk in a collaborative care approach called ‘task sharing’. Despite the relative success of task-sharing initiatives globally, there is a lack of consistency in the quality of MH care, where individuals who have severe mental illness are not appropriately diagnosed and treated by community health workers.4 5 In the community clinics in the PICs, outpatients come with different ailments, and the CHNs are challenged to identify the ones with mental health risks. This task generates a high cognitive load on these nurses, necessitating a need to leverage cheap and easily available technology, as support to offload some of the burden.

Given the widespread accessibility to smartphones in Fiji, it is appropriate to use them to support CHNs in MH care delivery by developing mobile-based screening applications to quickly screen and provide care to patients at risk. However, while mobile technology intervention have great potential to help address current problems, little is known about its added value to provide efficient, effective and safe care in the PICs.

The optimal design of mobile health (mHealth) interventions requires an understanding of the cognitive underpinnings of the use of any technological devices since the interventions are most often ad hoc and lack a sound empirical basis for investigations.6 The use of mHealth has been successfully undertaken to support nurses in providing bedside documentation and to assist in their usual workflow.7 8 Although, smartphones and related mobile technologies are recognised as flexible and powerful tools when used sensibly in augmenting human cognition, there is also a growing concern that continued use of these devices, may have a negative impact on users’ cognition, namely, their ability to think and pay attention to specific tasks.9

Therefore, our studies are driven by theories and methods from cognitive science related with special focus on text comprehension (knowledge representation and interpretation) and patient management decisions within the cultural context.10 11 The cultural context refers to the social behaviour and norms practiced in the Fiji culture, including the peoples’ knowledge, beliefs, language and customs. It is the way they think, understand and communicate, which is important to consider for building any successful interventional tools.

Qualitative–cognitive studies involves collecting and analysing data from a fewer number of representative participants, with a focus on their knowledge structures and use of cognitive strategies in processing patient information.12 13 Cognitive strategies are the specific methods that people use while solving problems. This process of eliciting underlying thoughts while reading or solving a problem requires a well-established procedure in psychological literature of collecting lengthy ‘think-aloud’ protocols from the nurses, as they interpret, identify and resolve the patient problem.14

This ‘think-aloud’ method is used extensively in psychological and educational research and offers unique approach to capture participant thought processes when interacting with technology and interventions.15 16

Methodology

Rationale

We are using smaller sample sizes of participants with in-depth analysis in our studies rather than a large-scale epidemiological study, limiting our ability to run statistical analyses on the data. We aim to develop a basic understanding of a complex phenomenon under consideration before testing generalisable inferences from this data. It should be noted that in any complex clinical situation filled with idiosyncrasies, varied situations and unique constraints, there are a remarkable number of invariants on which we can capitalise once we identify the overbidding phenomenon that is task and people invariant. Although our sample size is small, we attempt to get data saturation with qualitative methods.17

To characterise cognitive strategies underlying diagnosis for depression and suicide by CHNs under three conditions: no guidelines (current practices) as the baseline and two guidelines, paper and mHealth tools. Representation of patient information is different in the three conditions, and thus the information will be interpreted differently.18 The patient data in the paper-based support tool is in a narrative form. The data in the mobile phone is discrete, requiring a shorter time to process information and require different nature of inferences to be drawn for making diagnostic decisions.

We expect that the nurses using mHealth will (a) process clinical information in a shorter time and (b) generate fewer errors in the calculation for more accurate diagnostic and therapeutic decisions.

Study design and procedures

Using an experimental design, CHNs were assigned to one of three conditions, nurses using to guidelines on mobile phones, on paper, and the last group were given no guideline support (control, reflecting the current status).

In all conditions, CHNs were presented with two clinical case scenarios, one representing severe depression and the other representing a high risk for suicide (described in materials and measures below). The CHNs in the control condition were tasked with four clinical problem-solving activities: (1) Think aloud while reading through each case (referred to as the ‘think-aloud’ task), (2) summarise each case (referred to as the ‘summary problem’ task), (3) provide a final diagnosis (referred to as ‘final diagnosis’ task) and (4) provide recommendation(s) for treatment (referred to as ‘recommendation(s) for treatment’ task).

After completion of all four activities, only the CHNs in the two experimental conditions were given guidelines and asked to engage in the same four activities mentioned above using the respective guideline, as illustrated in figure 1. All nurses were required to read the clinical problem scenarios without any support from the mobile or paper-based guideline to get the base line data, before they were asked to use either mobile or paper-based technology.

Figure 1
Request permissionrequest permission icon
Figure 1

Study design and procedures for severe depression and suicide risks.

Materials and measures

Clinical cases

Two clinical scenarios used in the study reflecting severe depression and suicide risk were developed by psychiatrists and primary care physicians based on what would be expected in real community clinics in Fiji. As an illustrated example, the case of severe depression is presented in figure 2.

Figure 2
Request permissionrequest permission icon
Figure 2

Clinical case of severe depression in the Pacific Islands.

Clinical guidelines

Pre-existing paper-based guidelines—the ‘Center for Epidemiological Studies Depression Scale’ (CES-D).19 for depression, and the ‘Suicide Behaviors Questionnaire-Revised’ (SBQ-R)20 for suicide—were selected for the study. CES-D is a screening tool for depression developed by the CES and validated across various populations including Fiji.21 22 The guidelines are given in online supplemental appendices A and B. The questions (20-item for depression and four items for suicide risk) were converted for use by nurses (from self-evaluation) evaluated for cultural sensitivity and contextual appropriateness with the support of local clinicians, and updated via an iterative design process.

These guidelines were programmed into a mobile-phone application, Application for Suicide Risk and Depression Assessment (ASRaDA), and updated using a number of iterations with usability evaluation provided by CHNs. The left hand side of figure 3 shows the suicide guideline screen in the mobile phone, together with a nurse’s response.

Figure 3
Request permissionrequest permission icon
Figure 3

SBQ-R Screen in ASRaDA showing guideline questionnaire and nurse’s response, and System Usability Scale.

The nurses read the questionnaire items aloud to a member of the research team who acted as the patient, and who provided a standard scripted response (typical of responses in Fiji) to the nurses. For example, the patient would respond to an item #5, ‘How often have you had trouble keeping your mind on what you were doing?’ with, ‘It has been really difficult to do my work or my usual chores’. This narrative response had to be translated into a quantitative number on a symptom frequency scale, from zero (rarely or none of the time; less than 1 day) to three (most or all of the time; 5–7 days). Total scores were added manually in the paper-based condition, but they were automatically calculated in the mobile condition.

Determining the criticality of guideline information

Two experts scored information in each of the two clinical cases as critical, important or additional supporting information for making the diagnoses. These items were then checked against the guideline items for alignment with the clinical aspects of the patient problem to capture the appropriate constructs, as intended. The criticality of guideline information is given in figure 4.

Figure 4
Request permissionrequest permission icon
Figure 4

Expert classification of level of criticality of information in clinical text and corresponding guideline(s) for depression and suicide risks.

Mobile technology and development

The application development system, invented initially at NASA Johnson Space Center that enabled the development of media-rich training and clinical practice guideline applications without programming,22 23 was used to design and test a mobile phone based clinical guidance in rural health providers in India and in Colombia.18 24 25 In our current study, we leveraged concepts and experiences from this tested system to develop the suicide risk and depression assessment mHealth tool (ASRaDA) on an Android smartphone application, and loaded on Samsung J3 smartphones. Several versions of ASRaDA were tested in a controlled setting with a small cohort of CHNs, and refined iteratively into a final version, including responding to the 10-item System Usability Survey (SUS).26 27

CHNs logged with the patient’s name into the application, displaying guideline selection (see online supplemental appendix C, for an example of the mobile ASRaDA application guideline). Each participant was given an identification number for privacy, where the mapping to the actual name was encrypted for security.

Participants

Forty-eight CHNs were recruited from an available pool of 5000 participants with the assistance of the Fiji Ministry of Health (MoH), according to the inclusion and exclusion criteria (race representation, relative fluency in English language, ages and years of experience). Since more women traditionally train as nurses, the possible recruitments were all women. The detailed eligibility criteria included: practicing or medically trained CHNs, who has satisfied requirements of Fiji’s MoH, fluent in English (national language of Fiji), capable of using smartphone applications, between 25 and 55 years-old, and at least 5 years of nursing-related experience. Approximately 60% of the CHNs were local Fijians and 40% were Fijians of Indian origin, representing the per cent distribution of nurses in the community clinics. These percentages also align with statistics from prior research studies.28 29

Only women traditionally train as nurses in the Pacific Islands. Representing the Fiji demographics, the recruits were approximately 65% i-Taukei (natives of Fiji) and 35% of Indo–Fijian origin. These percentages align with statistics from prior research studies.28 29 The community nurses had varied training and included nurses with general nursing (54%), mental health (23%) with WHO mhGAP30 and other specialised training (79%) such as maternity or dermatology.

Methods of analysis

The data were collected and analysed using mixed methods approach that closely aligned with the tasks and the corresponding cognitive measures, as given in figure 5.

Figure 5
Request permissionrequest permission icon
Figure 5

Mixed methods: tasks and measures.

Audio recordings on the mobile phones were transcribed using ‘Express Scribe Transcription Software’ V.6.10 from NCH31 automated software and checked for accuracy by the local experts. Using thematic qualitative approaches with inductive and deductive coding, as well as specific theme development,32 33 the generated ‘think-aloud’ protocols were analysed. Initial emerging themes were used to code the rest of the transcripts to create descriptions of all emerging themes that accurately reflected the data. Nurses’ reasoning patterns were identified using method in previous research on guideline-assisted reasoning.12 13 The response protocol segmentation processes identified both the concepts used by the nurses and the relations among these concepts in their reasoning. Semantic networks were then constructed, where concepts and their relations are organised into graphical structures that represent the steps in their reasoning. These semantic networks convey two types of information: conceptual information (ie, the concepts used to solve a problem) and structural information (ie, how the concepts are related to each other) about how the subjects constructed their diagnoses and recommendations

The diagnostic accuracy coding was adapted from Patel and Groen34 and comprised three categories: Correct (ie, completely accurate), partially correct (ie, some accurate diagnostic components) or incorrect (ie, not accurate). These categories were evaluated as a function of the levels of nurses’ MH training.

The summarisation protocols were analysed using formal methods of natural language representation, more specifically, concept and semantic analysis.34 Next, the expert template for information criticality was used to code CHNs summary protocols. An illustrated example of the coding of these three summary narratives is given on figure 6.

Figure 6
Request permissionrequest permission icon
Figure 6

An example of analysis of summaries and recommendations as a function of guideline use by a community health nurses.

An example of the first step in clinical text segmentation and coding of CHNs protocols for generated inferences using the depression scenario is given in online supplemental appendix D.

The times for task completion were automatically calculated on the smart phone recorders.

Results

Time on task: task efficiency

Task efficiency is the time required to complete the task. The benchmark was the time on task with no guideline support (mobile or paper).

Figure 7 depicts time for task completion with paper and mobile-based guidelines for depression and suicide risks scenarios.

Figure 7
Request permissionrequest permission icon
Figure 7

Time for task completion with paper and mobile based guidelines for depression and suicide risk. CHNs, community health nurses.

The average times for completion of the depression and suicide risk problems were the shortest when using the mobile technology, followed by the paper-based and the no guideline support condition. The use of mHealth-based guidelines was most efficient in supporting quick identification of suicide and depression risk patients.

Time on task: process efficiency

Process efficiency is the processing of patient information for depression and suicide risks as identified by concept density and accuracy of patients description in the problem summaries provided by the nurses.

Summarisation tasks were analysed for knowledge organisation among the control and the experimental groups. There were fewer concepts identified in the summaries provided by CHNs in the guideline experimental conditions for depression (paper, M=6.44 (6.34); mobile, M=11.11 (3.52)) and suicide (paper, M=3.69 (7.04); mobile, M=3.10 (1.90)) problems than in the summaries under control condition (depression: M=15.94 (7.85) and suicide: M=12.69 (5.79)) for both depression and suicide risk cases (see figure 8). Most of the information in summaries with guideline support were important and critical only, whereas in the control condition, irrelevant information was also included.

Figure 8
Request permissionrequest permission icon
Figure 8

Mean number of concepts in summarization task using paper, mobile, and no guideline conditions. CES-D, Center for Epidemiological Studies Depression Scale; SBQ-R, Suicide Behaviors Questionnaire-Revised.

Reasoning about the patient problems

Knowledge organisation drives the development of reasoning strategies for diagnoses. Thus, it is expected that the CHNs will show different reasoning patterns in the control and in both the experimental conditions. As reported earlier, the transcripts were analysed using a formal method of natural language analysis, the semantic networks.13

Figure 9A presents an illustrated example of a community nurse’s think-aloud protocol (nurse #1 with MH training) in the control group, working with the depression case. The figure shows asemantic network structure, where nodes are patient information given in the clinical text. The links are the directionality of inferences generated during the think-aloud process.18 She identified the key concepts associated with the scenario but attributed the financial crisis to the patient’s ‘bad’ behaviour, which lead to depression, and she recommended that the problem may go away if the patient stopped spending money on drinking kava and smoking suki.

Figure 9
Request permissionrequest permission icon
Figure 9

(A) Patient problem explanation prior to reading the mHealth guideline for depression. Key: rectangles represent concepts identified and the links represent connections between the concepts as inferred by the community health nurses. The arrows represent the directionality of reasoning. (B): patient problem explanation AFTER reading the mHealth depression guideline.

Figure 9B presents an illustrated example of a think-aloud protocol of another community nurse (nurse #2 with MH training) with an equal level of training in MH when primed by the depression guideline. The nurse first considered severe depression, a significant factor as a possible diagnosis, and looked for confirming evidence in the patient scenario. Drinking more kava and smoking suki was considered a result of depression rather than the cause. She recommended that Ravi seek professional help.

With the use of paper-based guidelines, a similar pattern was found. This shift in reasoning was identified in 35% and 30% of the nurses working with depression scenarios in mobile and paper-based support, respectively, showing a specific positive trend in behaviour change. These patients were recommended for illness management according to the guidelines, which otherwise would have been missed. The rest of the nurses either did not generate rich enough data to conduct the analysis of natural language (NLP), as with the rest of the data, or did not show any change in their reasoning patterns. The suicide risk scenario guideline was too short, resulting in sparse think-aloud data, and was not amenable to formal analysis for reasoning patterns.

Errors of calculation

Errors were found in manually adding the total number of scores with the paper-based guideline for depression (correct score: 36.55, CHNs: 30.00±2.6), and suicide risk (correct score:11.15, CHNs 12.15±1.3). In this particular situation, the errors did not affect the recommended treatment modalities since a wide range of the scores for each treatment modality resulted in the same treatment recommendations. However, in some other situations, where the score range relates for treatment recommendation is much smaller, these errors will be of concern. There were no calculation errors when using mobile-based guidelines since the system automatically made the calculations and suggested the appropriate recommendations based on the total scores.

Diagnostic accuracy

Table 1 presents CHNs final diagnostic decision patterns after going through the patient problems.

Table 1
|
Final diagnosis provided by the community health nurses

In the case of severe depression, the majority of CHNs provided a partially correct final diagnosis. The four nurses, who provided the correct diagnosis, had recently received or were currently undergoing training in MH (WHO-based mhGAP programme.30 However, two of these four nurses provided only partially correct recommendations for treatment. Five CHNs who provided incorrect diagnoses had not received MH training. Training in mental health programme appears to play an important role in diagnostic accuracy for identification of depression and suicide risk.

Errors of translation: From qualitative to quantitative information.

During our analysis, we discovered other form of errors when coding nurses’ responses to the guideline questions (qualitative) into quantitative scales. These errors were generated during cognitive processing of patient information for translating them (interpreting) into paper and mobile-based quantitative scales to determine the treatment recommendations.

Not all the nurses provided detailed ‘think-aloud’ data when computing this translation. Six nurses (total: 9), using paper-based guidelines and five (total: 7) using mobile-based depression guidelines, generated three to four errors in translation. An illustrated example of two such errors is given in table 2.

Table 2
|
Errors in translating narrative responses to the questions into quantity

Suicide risk guideline was too short to capture such errors. Limitations in data collection restricted our ability to provide a more comprehensive information on these errors, except that such errors are pervasive due to the complexity of patient responses, which do not fit neatly into a set of quantities.

Usability

Sixteen nurses who used the ASRaDA application, 7 participants for CES-D and 14 for SBQ-R completed the questionnaire. From a maximum score of 100 on SUS usability scale, ASRaDA was scored at 85.36. The details of the usability study and its limitations are reported elsewhere.29 ,35 The left hand side of figure 3 shows the suicide guideline screen on the mobile phone, together with a nurses’ response to the usability test.

Discussion

Clinical guideline acts as a primer to help focus on fewer and more critical/important information, filtering out most irrelevant information34 36 It constrains a broad memory search, making it easier and quicker to get to the target, due to reduced memory load. The ability to separate relevant information from irrelevant information is recognised as a hallmark of developing expertise, where the guidelines provide such support.

The conceptual shift in reasoning assists the nurses to narrow the relevant information search space and, quickly arrive at the diagnosis.37 This change in the directionality of reasoning suggests a critical behaviour change from blaming the patient to attributing the problem to the nature of the illness. This pattern of behaviour has also been reported in other studies.5

The nature of patient information processing is different for nurses with and without mHealth-supported guidelines. The narrative nature of paper-based guidelines takes longer to process, but requires fewer inferences to be generated with fewer chances of any significant errors. The discrete nature of mobile-based guidelines helps focus on critical information faster. However, the user is required to generate more inferences in order to relate the discrete symptoms, to generating some form of patient narrative, resulting in a greater chance of generating errors of interpretation.38

These results are shown to be true in our small sample of participants. With detailed analysis of fewer subjects, the results predict with some measure of confidence that community nurses with vastly different backgrounds, working in different settings, will generally experience similar problems when dealing with cognitive and technical performance pressures of varied characteristics. This detailed study enhances the reliability and rigour in generalising the study findings. The next step is to test them in the broader sample of nurses and implement and evaluate the tool in the clinical setting. The results are generalisable to other PICs (which share a relatively similar culture) and to other LMICs, as long as the tool is ecologically validated and tested for cultural sensitivity in the new context.

Impact of this research: Currently, treatment of those in PICs with mental health problems occurs when the disease is at an acute stage, for example, an unsuccessful suicide attempt, that necessitates hospitalisation. There are very few means or methods in the PICs to identify and treat at-risk persons at an early stage of the disease when treatment can be most effective. A significant benefit of our research is (a) increased task-sharing and empowerment of CHNs as the front-line care providers in mental health care and (b) likelihood of better mobile tool acceptability and sustainability in the clinical environment. The primary strength of the research is a novel application of cognitive, decision-making and safety aspects in mHealth design and user-centred testing process.

The targeted clinical problem, assessment of the risk of suicide in patients with depression, is an active public health issue everywhere in the world, with relevance to a variety of populations (for instance, war veterans and active military in the USA) and in virtually every context (outpatient or inpatient settings, primary care or rural environments).

Conclusions

The evidence from our exploratory study supports the hypothesis that culturally appropriate smartphone technology assists in the process of task-sharing with community health nurses, thereby reducing the burden on a limited number of available mental health professionals. The mHealth supports efficient and effective screening of patients with severe depression and suicide risk, with minor safety concerns. Furthermore, the results from the cognitive analysis show that culturally appropriate clinical guidelines for mental health risks trigger a positive trend in behaviour change.

Limitations and challenges

There are always challenges when intervening in any complex and culturally diverse environment. This situation is exacerbated in resource-limited conditions such as in LMICs. One such challenge is about implementation in the clinical practices. When implementing ASRaDA into the community clinics in the PICs, these challenges will be salient, and will need to be carefully monitored for any unintended consequences.

One of our experimental tasks required the nurses to ‘think-aloud’ while reading the clinical cases. As in several other cultures, people do not often read aloud in a classroom in the PICs. This resulted in the nurses processing more information at the level of text syntax rather than the semantics, making it difficult to gauge more accurately, their understanding of the clinical problem. Training the nurses to think aloud while doing their tasks was not trivial, leading to some sparse and missing information during data collection, limiting our ability to collect the complete data set.

Article metrics
Altmetricopen-url
Dimensionsopen-url