Abstract
BACKGROUND:
In Fiji and other South Pacific island countries, depression and suicide are of great concern. There is a pressing need to rapidly identify those at risk and provide treatment as soon as possible.
OBJECTIVE:
Design, develop and test a mobile health tool that enables CHNs to easily and rapidly identify individuals at risk for suicide and depression and provide guidelines for their treatment.
METHODS:
Using Android Studio, a native app called ASRaDA was developed that encoded two validated scales: Center for Epidemiological Studies-Depression (CES-D), and Suicide Behavior Questionnaire-Revised (SBQ-R). The usability of the app was measured using the System Usability Scale by community health nurses in Fiji.
RESULTS:
Out of a maximim possible of 100 on SUS, ASRaDA was scored at 86.79.
CONCLUSION:
Mobile tools with high usability can be designed to aid community health nurses in Fiji and Pacific island counties rapidly identify those at risk for depression and suicide.
Introduction
Mental illness is a pertinent risk factor related to suicide. Around 80% of suicide attempters have a diagnosis of mental illness [1]. For young people, the aggregate lifetime and 12-month prevalence of suicide attempts were 6% and 4.5% respectively [2]. Significant differences were found in precipitants for suicide attempts across the lifespan. Middle-aged adults had relatively more financial [3] and medical problems [4]. Elderly without a past history of suicidal behavior were more likely to have a pre-suicidal plan for the fatal suicide act [5]. Ethnic differences were found in risk and protective factors, and perceived lethality of suicide attempts [6]. Each ethnic group should develop culturally-sensitive suicide prevention programs to target at vulnerable population and strengthen protective factors [7].
Depression and suicide are of great concern in Pacific island countries, including Fiji [8]. There are multiple reasons for this including separation from friends and families as people move to the cities for economic reasons, intergenerational tensions due to changes in lifestyles caused by economic and societal pressures, changing roles of men and women causing stresses on roles and relationships [8]. All these factors can lead to mental stresses of various knds including depression, of which suicide can be a consequence.
The Fiji islands archipelago presents unique challenges in the management and control of population health. The population of about 900,000 is scattered across nearly 110 islands in the South Pacific, over an area of about 18,300 sq. km. Mental health poses a particular challenge due to the serious lack of physicians specializing in mental and behavioral health. The Fiji Ministry of Health and Medical Services runs community health clinics throughout the islands staffed by physicians and community health nurses (CHNs). These nurses have a three-year bachelors degree in nursing and are certified as registered nurses. They must be licensed and registered with the Fiji Nursing Council. Upon completion of a one-year internship post-qualification at one of the divisional hospitals, they are then eligible to be posted to the periphery as community (Public) health nurses. While CHNs in Fiji are fully trained nurses, mental health care is not a major focus of their clinical training although recently some have begun to receive supplementary training in mental health care. Clearly, early identification of depression and suicide risk can play a major role in addressing the suicide crisis in Fiji. With support from the US National Institute of Mental Health, we have been studying the role of mobile health technology (mHealth) to enable task-shifting to CHNs for screening individuals who are at risk for depression and suicde. In recent years, health-related apps on mobile phones (mobile health [mHealth] apps) appear to be a feasible approach for disease management [9] including postnatal depression [10], bipolar disorder [11], dementia [12], and addiction [13]. Furthermore, mHealth can provide inter-professional education [14]. Mental health professionals can be involved in mobile app development [15] by offering cost-effective cognitive behaviour therapy [16] and mindfulness therapy [17]. Developing mHealth apps or interventions towards suicide prevention could be feasible to proliferate the benefits of such application [18]. Our prior studies have shown that mHealth tools can provide community health workers with focused point-of-care information and clinical decision support tools to improve care [19, 20, 21]. While the major focus of our study is to understand cognitive mechanisms underlying use of mHealth tools by CHNs, tool usability is an important consideration in succesful adoption and use. In this paper we describe development and rapid usability evaluation of the mHealth tool developed to aid CHNs in identification of those at risk for depression and suicide.
Methods
The mHealth tool we developed is called ASRaDA (Application for Suicide Risk and Depression Assessment). ASRaDA is a native app so that once downloaded onto the user’s smart phone it can execute even without connectivity of any kind. It was developed for the Android operating system and designed for optimal use on a Samsung J3 smartphone. This phone has dimensions 142.3
The ASRaDA app encodes two scales. CES-D (Center for Epidemiological Studies-Depression) [24] and Suicidal Behavior Questionnaire-Revised [25]. Each response on these scales is assigned a numerical value and a formula is applied in order to get a numerical value for the depression severity (CES-D) and suicide risk (SBQ-R). Depending on the numerical vlaue the patient is assigned to low, medium, or high risk. The ASRaDA app enables the user, typically a community health nurse, to enter these responses into the scales. It automatically computes the scale formula, provides the severity level, and also presents recommended guidelines for patient management based on the computed scale value. Figures 1–3 illustrate the SBQ-R running on the ASRaDA app. In addition, ASRaDA also encodes the System Usability Scale [26, 27, 28] (Fig. 4).
SBQ-R screen in ASRaDA.
SBQ-R screen showing questionnaire completion.
Results of completion of SBQ-R.
System Usability Scale implemented in ASRaDA.
This study received ethical approval (no. IRB2017-0374D) from the Institutional Review Board, Coordinator Tamara Britt, of Texas A&M University. Approval by Texas A&M University was granted after obtaining approvals from New York Academy of Medicine and Fiji National University.
Target users
The target users of ASRaDA are community health nurses practicing in health clinics in Fiji. Most of the CHNs were familiar with the use of Android smartphones and apps on these phones. The official language of Fiji, widely understood by all CHNs, is English. CHNs participating in our study were located in the main island of Viti Levu. Our study was laboratory-based and included 48 CHNs who were presented with two cases on paper, one presenting an individual with depression and the other describing a person at high risk for suicide. These cases were developed with the active assistance of medical mental health experts in Fiji based on their actual experiences, together with consulting psyciatrists. The nurses were divided into three study groups, with 16 subjects enrolled in each group, in which they screened the cases as follows: Group A: usual care with no tools, Group B: paper-based tools consisting of CES-D and SBQ-R, and Group C: the same tools encoded into the ASRaDA app. Accordingly, this paper is concerned with usability results gleaned from the group of 16 CHNs that used the ASRaDA app. More information about the testing procedure is in [29].
Usability considerations
The CES-D and SBQ-R scales are complex in different ways. CES-D is lengthy, consisting of twenty questions, each of which has one of four possible responses. In our case, we used a version of CES-D with wording designed to be easily understood by our users. SBQ-R has four questions. The number of responses vary according to question, from three to seven possible responses. Acordingly, the design and usability challenge lies in fitting these questionnaires into a small screen so as to enable the user to fill in the responses with minimal strain and effor, without generating errors.
The app was developed iteratively, with each version being evaluated by colleagues and target users in Fiji and with the next version incorprating their feedback. The following design and usability considerations were taken into account.
Readability: Typeface size should allow easy readability without eyestrain; Minimize scrolling: The scales are lengthy and screen size is small. Need to minimize scrolling to prevent user fatigue and increase ease of navigation; User should be given feedback as to how far they have progressed in the scale; To eliminate incomplete data collection, all items must be completed before user can proceed to the next step of the app; App should minimize errors and user fatigue by automatically computing the scale value; The app should automatically provide interpretation of the scale value and corresponding guideline for care; The app should work seamlessly and function even when there is no connectivity. Wireless data connectivity is not of unifrom quality across the Fiji islands. To handle this consideration, we developed ASRaDA as a native app using the Android studio app development environment, to maximize speed. Once downloaded on the phone it can function stand-alone without data or wifi connectivity.
Figures 1 and 3 depict the implementation of ASRaDA based on these considerations. The screenshots have been annotated to emphasize specifc usability features.
To assess the usability of ASRaDA we used the System Usability Scale (SUS) [26, 27, 28]. SUS has been validated in numerous contexts ranging from large format computer screens to mobile phones and also in healthcare contexts. In our case, the SUS was implemented as part of ASRaDA (Fig. 4). The users (CHNs) answered the questions in SUS after completion of each of CES-D or SBQ-R. These scores were averaged to obtain an aggregated SUS value for both scales.
Results
Out of 16 subjects who used the ASRaDA app, we obtained SUS scores from seven participants for CES-D and 14 for SBQ-R. Although all subjects were requested to provide responses to the SUS questionnaire after completing each of CES-D and SBQ-R, several did not follow these instructions. Table 1 shows SUS scores for each of CES-D and SBQ-R scales in the ASRaDA app. The maximum possible value of SUS is 100. SUS score averaged across both SBQ-R and CES-D components of ASRaDA is 85.36.
System Usability Scores for CES-D and SBQ-R scales in ASRaDA app. Higher scores indicate better usability. Maximum possible value for SUS is 100
System Usability Scores for CES-D and SBQ-R scales in ASRaDA app. Higher scores indicate better usability. Maximum possible value for SUS is 100
Percentage of responses to SUS items, CES-D, N
Percentages of responses to SUS items, SBQ-R, N
Average percentage responses across CES-D and SBQ-R. For even-numbered questions smaller (1, 2) values of the response variable indicate better usability. For odd-numbered questions greater values (4, 5) indicate better usability
Tables 2–4 present the percentages of study participants responding to each of the five possible responses to the ten questions comprising the System Usability Score. Table 2 pertains to CES-D, Table 3 to SBQ-R, and Table 4 is the average across the two for each participant. Sample sizes for CES-D and SBQ-R participants were 7 and 14 respectively. In Table 4 if the same participant did not respond to CES-D his/her response for SBQ-R was used. It should be noted that in SUS for even-numbered items smaller values (1, 2) of the response variable indicate better usability while for odd-numbered items greater values (4, 5) indicate better usability.
Tables 2 and 3 display the percentage of participants responding to each of the items of SUS for CES-D and SBQ-R respectively. Table 4 shows the average percentages across both scales. Tables 2 (CES-D) and 3 (SBQ-R) show some differences. For example, 57.1% responded “Strongly agree” in Table 2 (CES-D) to item 1: “I think I would like to use the system frequently” whereas in Table 3, 93.3% responded “Strongly agree” to the same item for SBQ-R. Average (See Table 4) is 75.24%. Similarly, for item 3 “I though the system was easy to use”, 57.1% strongly agreed in Table 2 (CES-D) while 86.7% responded in like manner in Table 3 (SBQ-R). In Table 2 (CES-D) no participants agreed or strongly agreed with item 2 “I found the system unnecessarily complex” while for SBQ-R, 26.6% did. Similarly, for item b “I thought there was too much inconsistency in the system”, none of the participants agreed or strongly agreed in CES-D while for SBQ-R, 13.3% strongly agreed. The responses were similar for item 10 “I needed to learn a lot of things before I could get going with the system”. For question 9, 100% of participants either agreed or strongly agreed that they “felt very confident using the system”, while 93.33% felt this way for SBQ-R. Item 8 “I found the system very cumbersome to use” displayed substantial differences between the responses across CES-D and SBQ-R scale. Here, 71.4% in Table 2 (CES-D) strongly disagreed while only 53.33% strongly disagreed in Table 3 (SBQ-R).
The ubiquity of mobile phones in general, and the high penetration of Android smart phones in the Fiji islands indicate great potential for mHealth applications. Due to the island geography of the Fiji archipelago, mHealth technologies based on native apps can be an ideal method to disseminate support using App-based clinical practice guidleines, and other medical information. Such mHealth applications can play an important role in addressing mental health as well as other chronic health conditions such as diabetes, obesity, and hypertension. In the South Pacific region, task shifting to front line workers, such as community nurses, is important due to the scarcity of fully qualified mental health specialists in these countries. This is also true for developing countries in general [30].
The average SUS score for our app was 86.79 out of a maximum possible value of 100 and informal remarks from the study participants indicated they were very satisfied with the capability of the system to provide correct numerical values for CES-D and SBQ-R instantaneously. The study by Bangor, Kortum, and Miller [28] describes this score as falling in the “Excellent” range.
Summarizing, while the participants responded favorably to the usability of both CES-D and SBQ-R, some differences among them need to be addressed. Responses to item 4 indicate, in particular, that a sizable group (13.4%) of participants felt that they needed support from a technical person to use the SBQ-R scale. The updated version of the ASRaDA will address these issues to improve usability further.
Although we used a very simple SUS usability assessment tool to quickly develop an iteratively designed tool for mental health screening, we recognize that this is also a limitation. In our next step in the laboratory-based usability studies we hope to include a video-based capture of actual user actions to text the App usability, based on suggestions by Kushniruk and Patel [31, 32]. Furthermore, we recognize that once this app is used in the real community setting, it is likely to behave differently.
Limitations
The study has several limitations. First, the ASRaDA app encoded CES-D and SBQ-R. These scales mainly assessed mood and suicidal behavior but not cognitive aspect of suicide [33]. Second it did not measure other factors of suicide including habitual poor coping [34], medical problem and recent surgery [35], suffering from pain [36] and unemployment [37]. Third, the suicide method (e.g. drug overdose) [38] and lethality of proposed suicide method was not assessed [39]. Finally, the apps did not assess for protective factors for suicide such as resolution of precipitant [37].
Conclusions and future work
Mobile tools designed to help community health nurses screen for depression and suicide can play an important role in identifying at-risk individuals and directing them early towards treatment and further health management. App usability using simple SUS assessment tool can move the development process forward quickly using iterative cycle design, recognizing that detailed evaluation of the app will still be necessary.
The version of the ASRaDA app described in this paper has been designed only for laboratory use. The project has received considerable encouragement and publicity from news media in Fiji and efforts are now under way to develop the next version of the app suitable for stand-alone field use by community health nurses.
Footnotes
Acknowledgments
Research reported in this publication was supported by the National Institute of Mental Health of the National Institutes of Health under award number R21MH114621. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Conflict of interest
None to report.
