Psychiatric stressor recognition from clinical notes to reveal association with suicide

Abstract

Suicide takes the lives of nearly a million people each year and it is a tremendous economic burden globally. One important type of suicide risk factor is psychiatric stress. Prior studies mainly use survey data to investigate the association between suicide and stressors. Very few studies have investigated stressor data in electronic health records, mostly due to the data being recorded in narrative text. This study takes the initiative to automatically extract and classify psychiatric stressors from clinical text using natural language processing–based methods. Suicidal behaviors were also identified by keywords. Then, a statistical association analysis between suicide ideations/attempts and stressors extracted from a clinical corpus is conducted. Experimental results show that our natural language processing method could recognize stressor entities with an F-measure of 89.01 percent. Mentions of suicidal behaviors were identified with an F-measure of 97.3 percent. The top three significant stressors associated with suicide are health, pressure, and death, which are similar to previous studies. This study demonstrates the feasibility of using natural language processing approaches to unlock information from psychiatric notes in electronic health record, to facilitate large-scale studies about associations between suicide and psychiatric stressors.

Keywords

electronic health records natural language processing psychiatric stressor extraction statistical association suicide behavior

Introduction

As the worst acute outcome in psychiatry, suicide takes the lives of nearly a million people globally each year.¹ In addition to death by suicide, the rate of suicide attempts is also very high. Suicidal behaviors have an estimated economic impact of over US$40 billion in the United States. However, due to the low occurrence of, and heterogeneous risk factors for suicidal behaviors, clinicians and researchers face great challenges to detect and address such behaviors. Until now, it has been well established that suicidal ideation, non-fatal and fatal attempts often occur in the context of underlying mental health and substance use disorders.² Despite the tremendous efforts spent on prevention and intervention, the suicide rate in the United States has increased from 10 deaths per 100,000 persons in 1955 to 13 per 100,000 in 2014 in the United States, indicating the need for more effective approaches to understand suicide.³

Current approaches in suicide prevention are often based on suicide risk assessment tables or interviews. However, many patients who had suicide death denied suicidal ideation when assessed, even just 0–7 days prior to their death.⁴ Dredze et al.⁵ found that participants (or patients) tend to respond to surveys and interviews in ways that they believe are expected (“social desirability bias”) of them, especially for sensitive topics and topics that require patient cooperation. Therefore, asking people directly about suicide ideation or plan may yield less accurate results than observing their behavior.⁵

Previous studies have proposed various signs of suicidal behaviors from multiple perspectives. Based on transcripts of cognitive therapy sessions from 35 patients, Adler et al.⁶ proposed a set of cognitive warning signs for suicide attempts such as state hopelessness and focus on escape, and so on. World Health Organization⁷ proposed that the use of mixed methods was needed for suicide research, in which three key topics were highlighted: risk factors, efficacy of suicide prevention, and cultural factors. Recently, efforts were also made for suicidal behavior analysis and detection from social media data. Wongkoblap et al.⁸ conducted a systematic review of mental health research via social media and found that about 17 percent of the research was focused on using text analysis methods for suicidal behaviors. Cohan et al.⁹ used a supervised classifier with lexical, psycholinguistic, and topic modeling features to detect posts of self-harm from mental health forum. Similarly, Burnap et al.¹⁰ also used machine learning–based approaches to identify suicide ideation from tweets.

Recently, the implementation of electronic health records (EHR) systems has made large amounts of clinical data available digitally, which has facilitated applications of computational or statistical methods to detect risk factors for suicide ideation/attempts from observational data.¹¹ Researchers are investigating potential risk factors from various dimensions, including both structured, coded EHR data (e.g. demographic information, mental disorders, and physical disorders) and information (e.g. frequent words, sentiment polarity) extracted from clinical narrative using natural language processing (NLP)–based methods. By analyzing the full spectrum of data in EHR, computerized risk screening approaches may enhance prediction beyond individual analysis by clinicians.¹

One type of essential risk factor of suicide consists of psychiatric stressors,^12–14 which are defined as psychosocial or environmental factors (e.g. loss of a loved one, job issues) that can profoundly impact cognition, emotion, and behavior of patients.^15,16 Moreover, there is a major category of trauma and stressor-related mental disorders, including adjustment disorders, acute stress disorder, posttraumatic stress disorder, dissociative disorders, and so on.¹⁵ Current association analysis between stressors and suicidal behaviors is mainly dependent on surveys and interviews. Although suicide ideations/attempts and stressors are often recorded in psychiatric notes (see examples in Figure 1), few studies have utilized this type of data, probably due to that psychiatric stressors are often recorded in narrative text in EHR systems and are not directly available for analysis.

Figure 1.

Examples of sentences with stressors in psychiatric notes. The mentions of stressors are blue colored; and the mentions of suicide behaviors are red colored.

Therefore, to enable large-scale quantitative analysis of psychiatric stressors and suicidal behavior using EHR data, it is critical to develop automated approaches to extract and structure stressor information from clinical text. However, as illustrated by the examples in Figure 1, psychiatric stressors have several distinctive attributes, making automatic extraction of the information from text extremely challenging. First, stressors are highly unique to the individuals and come from a broad range of psychosocial environments, leading to very sparse distribution of stressors across different patients and clinical notes. In addition, the identification of stressors is greatly dependent on and constrained by contextual information.

This study takes the initiative to automatically extract both mentions of suicidal behaviors and psychiatric stressors from clinical notes using NLP-based methods. Mentions of stressors are further grouped into 15 types and modifiers (such as negation and subject) of each mention are recognized. Subsequently, a statistical association analysis between positive suicide ideations/attempts and stressors of the patients is carried out using the chi-square test.¹⁷ Our evaluation using a clinical corpus shows that the NLP-based methods are capable of extracting both mentions of suicidal behaviors and psychiatric stressors with practical performance. Furthermore, association signals between suicide and psychiatric stressors are similar to previous findings.^18–21 To the best of our knowledge, this is one of the first studies for association analysis between stressors and suicides based on automatically extracted information from clinical text using novel NLP approaches.

Materials and methods

System overview

Figure 2 shows an overview of the workflow for detecting stressor signals of suicide. First, the mentions of suicide ideation/attempt and stressors are annotated automatically in psychiatric notes using NLP-based methods. Mentions of stressors are further classified into 15 types to reduce the data sparseness. Then, the modifiers, such as negation, and subject of each mention are recognized. Only positive suicide ideation/attempt and stressor mentions of the patients are kept for further processing. A statistical association analysis between suicide ideations/attempts and stressors is then carried out using the chi-square test.¹⁷ The key components of the systems are presented in the following sections with details.

Figure 2.

Study design for detecting stressor signals of suicide.

Dataset and annotation

The psychiatric notes from the CEGS N-GRID (Centers of Excellence in Genomic Science Neuropsychiatric Genome-Scale and RDOC Individualized Domains) 2016 challenge organizers are used in this study.²² The 2016 CEGS N-GRID challenge aims to extract the symptom severity of patients from neuropsychiatric clinical records, which consists of three NLP tracks: Track 1 was de-identification of psychiatric text, Track 2 was the RDOC classification, which is to determine the symptom severity for patients, based on information included in their initial psychiatric evaluation. Finally, Track 3 was for novel use of the dataset: The data released for this 2016 challenge are the first set of mental health records released to the research community. These data can be used for mental health-related research questions that go beyond what is posed by the challenge organizers. The access of the dataset needs to be applied from the i2b2 data portal (https://www.i2b2.org/NLP/DataSets/). As the first corpus of mental health records released to the NLP research community, it contains 909 initial psychiatric evaluation records. Initial psychiatric evaluation records are produced by psychiatrists to document psychiatric signs and symptoms, disorders, and other medical conditions in order to decide the course of treatment.²³ All of the records are de-identified.

The original training set of the CEGS N-GRID 2016 challenge contains two subsets, one with gold-standard annotations of the severity of mental disorders and another is unlabeled. For the convenience of later applications, 409 psychiatric notes were selected from the labeled subset to build the gold-standard corpus of stressors, which was used to generate the automatic tools for suicide recognition, stressor recognition, and classification. An annotation guideline was developed and two annotators were recruited who manually annotated all the psychiatric suicide and stressor mentions in each note by following the guideline. In total, 327 psychiatric notes contained at least one stressor annotation. A total of 151 suicide behaviors were annotated, with a kappa value of 0.90 between the two annotators; 2194 stressors were annotated, with a kappa value of 0.68 between the two annotators.

Suicide behavior recognition

First, clinical records with mentions of suicide ideations/attempts need to be identified. According to the previous study of Anderson et al.,²⁴ suicidal ideation expressions extracted using keywords and patterns from clinical text only have an overlap of 3 percent with International Classification of Diseases—ninth revision (ICD-9) codes, and suicide attempt expressions only have an overlap of 19 percent with ICD-9 codes.²⁵ Therefore, it is necessary to extract such expressions from clinical text using NLP-based methods; a list of keywords such as “suicidal,” “suicide attempts,” “suicidal ideation/thoughts,” and “suicidality” are used to match mentions of suicide ideation/attempt from psychiatric notes.

Another challenge of collecting positive suicide ideation/attempt expressions of patients is the modifiers of such expressions. As illustrated in Table 1, the modifier could be negation, conditional, and uncertain, and the subject of the suicide mention could be another person instead of the patient. We use the machine learning–based modifier recognition module in the Clinical Language Annotation, Modeling, and Processing Toolkit (CLAMP)²⁶ to label the modifiers of suicide-related expressions automatically. In addition to suicide mentions in the clinical narrative, whether the patient has a personal history or family history of suicide behavior is also recorded in two structured fields (“Hx of Suicidal Behavior” and “Family History of Suicidal Behavior”) of the psychiatric notes. The values of these two fields (yes/no) are used to verify and adjust the modifiers identified by CLAMP. Finally, the positive suicide ideation/attempt mentions of patients (including both current and patient history) are kept for later association analysis with stressors.

Table 1.

Different types of modifiers for suicide mentions in the context.

Modifier	Examples
Negation	but did not feel guilty or have any thoughts of self harm or suicide
Non-patient subject	At elevated risk of harm to self given family h/o suicide. Worried about a close friend here who is suicidal …
Conditional	If you have increased thoughts of suicide while taking escitalopram, discontinue the medication, contact doctor
Uncertain	Later that year broke leg again (unclear if it was a suicide attempt)
Positive	as she does not see worth of life and has chronic suicidal ideation

Stressor recognition and classification

A hybrid approach for stressor recognition

The identification of stressor mentions can be addressed as a typical named entity recognition (NER) task. In a machine learning–based NER task, the problem is converted into a sequence labeling task by representing each word using specific labels.²⁷ The BIO labels are commonly used to represent named entities, where “B,” “I,” and “O” denote the beginning, inside, and outside of an entity, respectively. Therefore, the stressor recognition problem was converted into a sequence labeling task, assigning one of the three labels to each word. Figure 3 shows an example of the BIO representation, where the stressor entity “father’s death” is represented as “father/B ’/I s/I death/I” after tokenization.

Figure 3.

An example of BIO annotation for psychiatric stressors.

In our previous work on stressor recognition,²⁸ we mainly used Conditional Random Fields (CRF)-based machine learning methods and a set of features combining basic NLP features, domain knowledge features, and word representation features:

Basic NLP features. The most common NER features including bag-of-words, orthographic information (word patterns, prefixes, and suffixes), syntactic information (POS (part of speech) tags), as well as n-grams of words, POS tags, and their combinations (unigrams, bigrams, and trigrams).²⁹

Domain knowledge features. Features collected from domain knowledge bases related to various aspects of stressors, including lexicons of stressors, mental disorders, common disorders (i.e. non-mental disorders), negative words (e.g. “sad,” “against”), psychosocial environments (e.g. family members, social relations), and cues of discourse relations (e.g. “in setting of,” “in the context of”). Please find a complete list of terms used in this study in Appendix 1.

Unsupervised word representation features. Word representation features were generated from a corpus of unlabeled clinical documents. Specifically, we used word embeddings that produce a distributional word representation for each word in an unlabeled corpus as a real-valued vector using neural networks.^30–32 We used the binarized word embedding feature proposed in 2014 by Guo et al.³³ The intuition of the binarized embedding feature is to discretize the original real-valued matrix of Word embeddings and omit the insignificant dimensions. Specifically, to convert the real values in the original Word embedding matrix M_V×D to discrete symbolic values in [+,−, 0], the positive mean MEAN(j)⁺ and negative mean MEAN(j)⁻ for the jth dimension (column) of M_V×D are first calculated as follows

MEAN {(j)}^{+} = \frac{1}{N_{j}^{+}} \sum_{i = 0}^{V} M_{i, j}, M_{i, j} > 0

(1)

MEAN {(j)}^{-} = \frac{1}{N_{j}^{-}} \sum_{i = 0}^{V} M_{i, j}, M_{i, j} < 0

(2)

where N_j⁺ is the total number of rows with jth column M.j > 0, and N_j⁻ is the total number of rows with jth column M.j < 0. Then the discrete-valued matrix M^*_V×D can be derived by the following projection

M_{i, j}^{*} = {\begin{cases} +, if M_{i, j} > MEAN {(j)}^{+} \\ -, if M_{i, j} < MEAN {(j)}^{-} \\ 0, otherwise \end{cases}

(3)

Values in the $M_{i, j}^{*}$ row of the corresponding word will be used as its word embedding features.

The optimal configuration in Guo et al.³³ was used in our study. Specifically, the Skip-gram model is adopted for the original word embedding generation. The negative sampling method is used for optimization, and the asynchronous stochastic gradient descent algorithm (Asynchronous SGD) is applied for parallel weight updating. Since the real values of word embeddings are binarized, only three labels remained for the word embeddings, where “POS” stands for positive, “NEU” stands for zero, and “NEG” stands for negative.

However, the previous stressor recognition systems suffered from a very low recall of 34.7 percent for exact match and 65.5 percent for inexact match, mainly due to the sparseness, long boundaries and complex structures of stressor expressions, conjunctive structures formed by multiple stressor expressions, and the highly unbalanced corpus with a 1:12 ratio between sentences with stressors and the whole set of sentences. In addition to expanding the annotation corpus from 246 psychiatric notes to 327, this study uses the following strategies to address the low recall problem:

Minimal boundary of stressor annotation. To alleviate the sparseness and boundary ambiguity problems of stressors, the most informative text with minimal span was annotated as stressors. For example, in the sentence “Loss of father to cardiac event in December 2083,” “Loss of father” was annotated as the stressor without any further detailed description.

Examples of basic NLP features, domain knowledge features, and unsupervised word embedding features are listed in Table 2. In total, 36 basic NLP features are used for each token, including n-gram features, orthographic features, and so on. The dictionary of domain knowledge–based terms contains 4310 entries in total, including keywords of context, psychiatric symptoms, stressors, other common disorders, and positive and negative words. In addition, the word embedding feature is a binary feature with a value of “POS” or “NEU” for each token.

Table 2.

Examples of basic, domain knowledge, and unsupervised word embedding features for stressor recognition, taking the sentence “His depression is worsen after the break up” as an instance.

Feature type	Feature values
N-gram feature	…, TRIGRAM0 = [the + break + up], …, BIGRAM-1 = [break + up], …, BIGRAM0 = [the + break], …, BIGRAM2 = [after + the], …
Sentence feature	SentFeaLen = [8+], SEN_STARTWITH_ = [TRUE], …
Prefix–suffix feature	Prefix1 = [b], Prefix2 = [br], Prefix3 = [bre], …, Suffix1 = [k]
Section feature	Section = [Formulation]
Orthographic feature	RegCAPSMIX = [FALSE], RegEND_PUNCTATION = [FALSE], RegHAS_CAP = [FALSE], RegIS_DASH = [FALSE], …
Domain knowledge feature	DictFeaUNI-1 = [TK], DictFeaUNI-0 = [1], DictFeaUNI + 1 = [1], …
Word embedding feature	EB_0 = [POS], EB_1 = [NEU], EB_2 = [NEU], EB_3 = [POS], EB_4 = [POS], EB_5 = [NEU], EB_6 = [NEU], EB_7 = [POS]…

Re-weighting features of stressor sentences. The weights of features for each token inside the sentences with stressors are enhanced to 12 based on the proportion between sentences with stressors and the whole set of sentences.

Pattern-based stressor recognition. Considering that multiple stressors are frequently present in conjunctive structures, rules consisting of context patterns and conjunctive structure patterns are employed to label stressors. The output stressors are used as one additional type of feature for the machine learning–based method, to reduce the false positive errors produced by the pure pattern-based method. Some examples of patterns for stressor recognition are listed in Table 3.

Rule-based post-processing. We also used some simple post-processing rules to fix a number of obvious errors by the machine learning–based classifier: (1) conduct a dictionary lookup by exact match in the abstract, using the recognized entities and keywords of stressors as a lexicon. If there is a string that matches the recognized entity or a stressor keyword, then label the string as a new entity. (2) If there is any recognized named entity in a conjunctive structure, the other strings in parallel with the recognized entity will be labeled as new entities.

Table 3.

Examples of patterns for stressor recognition.

Patterns for stressor recognition
Stressor/concern/worry/issue/symptom … when/from/due to/because of Stressors
Psychiatric symptom increased/worsen … in context of/after/when/by Stressors
Stressor/pressure/risk factor/challenges ( Stressors )
stressful with Stressors
traumatic events … including Stressors
stress/stressor of Stressors

The position of candidate stressors is shown in bold.

Stressor classification

The extracted stressor mentions need to be normalized to different categories, before analyzing their association with suicide ideations/attempts. Based on manual analysis of stressors in the corpus of psychiatric notes used in this study, we summarized 14 different types of stressors, as listed in Table 4. Frequent keywords and patterns of each type of stressors are collected, based on which the stressors are categorized (Table 4).

Table 4.

Types of stressors and examples.

Type	Definition	Keywords	Examples
Abuse	Physical, physiological, verbal, sexual abuse, bully, violence	Abuse, abusive, bullying	Alleged maternal physical abuse; attempted sexual assault; she was bullied
Emotional	Marital events and emotional events with boy/girl friends	Divorce, break up	Patient’s parents divorced; intrusive memories about infidelity by ex-husband; a tumultuous relationship with her boyfriend
Relation	Conflicts in interpersonal relations	Conflict, interpersonal, relational	Severe conflict with general contractor; unfairly treated at his work
Trauma	Traumatic events, trauma	Trauma, traumatic	… has a significant trauma HX
Financial	Financial crisis	Bills, estate	financial strain; increased bills to pay
Death	Death of people with social relations	Died, death	Mother’s death
Health	Health issues, diseases, treatment, and so on	Surgery, cancer, diagnosis, health	Recurrence of her cancer; her older son has cerebral palsy
Caring_for	Taking care of children, elderly parents, and so on	Caretaker, custody	Challenges of raising twin boys; she serves as the primary caretaker for her elderly parents
Baby_birth	Baby delivery, pregnancy, miscarriage, and so on	Pregnant, miscarriage, delivery	The initial delivery; low chance of pregnancy; she had two miscarriages
Homeless	Homeless	Homeless, out of house	Out of house; patient was evicted
Pressure	Pressure from work, school, and so on	Performance, uncertainty, grades, difficulty	School performance, stressful working environment
Transition	Status, position, location change	Transition	Change, move
War	Experiences related to war	Deployment, attack	A deployment; combat and rocket attacks
Other	Rare stressors and un-specified stressors	Stressor, stressful	Getting her medications at the pharmacy; making appointments on time; family stressors

Association signal detection for suicide

To identify the salient signals between stressors and suicide ideations/attempts, contingency tables based on the co-occurrence between suicide mentions and stressors within the same psychiatric notes are generated, and the statistical association between suicide ideations/attempts and stressors is calculated using the chi-square test. Multiple mentions of suicide and stressors in different sections of one psychiatric note are only counted as one time. Furthermore, contingency tables are dropped without association calculation if the value of any entry in this table is lower than 8.

Experiments and evaluation

For experimental evaluation, the 327 clinical notes were separated into a training set of 215 clinical notes and a test set of 112 clinical notes. The optimal configuration of CRF was obtained using 10-fold cross validation on the training set, which were “-a lbfgs –p c2 = 0.9.” Word embedding features were derived from the set of unstructured notes in the Multiparameter Intelligent Monitoring in Intensive Care II (MIMIC II)³⁴ corpus. To generate word embedding features, we implemented the ranking-based deep neural network algorithm according to the paper from Collobert and Weston³⁰ using Java.

For stressor recognition, we started with a baseline system that implemented common NLP features. Then, we evaluated the effects of domain knowledge features, unsupervised word representation features, feature re-weighting, pattern-based features, and post-processing by adding each of them incrementally to the baseline system. The systems for automated suicide extraction, stressor recognition, and classification were trained from the training set and the performance of each system on the test set was evaluated using precision, recall, and F-measure, which is the harmonic mean of precision and recall as $2 \times p r e c i s i o n \times r e c a l l / p r e c i s i o n + r e c a l l$ . Both the exact and inexact match performance was reported. Exact match means that the recognized entity should be exactly the same with the gold-standard annotation, with the same string and the same offsets in the text; inexact match only requires that the recognized entity and the gold standard have an overlap with each other.

Results

Performance of suicide recognition

The suicide expressions are relatively straightforward to recognize in psychiatric notes. Only two phrases of “suicidal tendencies” and “suicidal gestures” in the test corpus were not exactly covered by the list of suicide behavior expressions collected from the training set (Table 5).

Table 5.

Performance of suicide recognition. Both the performance of exact matching and inexact matching are reported (%).

	Precision	Recall	F-measure
Exact	100	94.87	97.37
Inexact	100	100	100

Performance of stressor recognition

As shown in Table 6, the overall performance of exact matching is relatively poor. Adding domain knowledge features enhanced both the precision and recall and also yielded an F-measure of 66.70 percent. The word embedding features further enhanced the recall (60.12% vs 59.34%) and increased the F-measure to 67.63 percent. The performance increased consistently due to the addition of the feature re-weight strategy, pattern-based features, and post-processing, yielding the optimal F-measure of 73.91 percent.

Table 6.

Experimental results of CRF-based psychiatric stressor extraction systems with different types of features.

		Precision	Recall	F-measure
Baseline	Exact	59.85	46.82	52.54
Baseline	Inexact	88.30	55.40	68.08
+Knowledge	Exact	76.15	59.34	66.70
+Knowledge	Inexact	92.90	71.37	80.72
+Word embeddings	Exact	75.81	60.12	67.63
+Word embeddings	Inexact	92.56	72.87	81.54
+Feature_reweight	Exact	78.43	62.78	69.73
+Feature_reweight	Inexact	93.95	75.63	83.80
+Pattern_feature	Exact	73.64	71.23	72.41
+Pattern_feature	Inexact	91.4	84.58	87.86
+Post_processing	Exact	72.44	75.5	73.91
+Post_processing	Inexact	90.8	87.3	89.01

Both the performance of exact matching and inexact matching are reported (%).

As for the results of inexact matching, the baseline CRF model yielded high precision, while the recall was still very low (precision: 88.30%, recall: 55.40%). Integrating domain knowledge into CRF significantly enhanced the recall (55.40% vs 71.37%). The word embedding features further improved the recall (71.37% vs 72.87%), with a slight drop in the precision (92.90% vs 92.56%). Both precision and recall were increased by re-weighting the features (93.95% vs 92.56%, 75.63% vs 72.87%). Notably, the pattern-based features and post-processing enhanced the recall most significantly (84.58% vs 75.63%; 87.30% vs 84.58%), with a sacrifice of precision. Overall, the system incorporating all features achieved the optimal F-measure of 89.01 percent.

Performance of stressor classification and distribution of stressors

As illustrated in Table 7, the F-measures for classifying Abuse, Trauma, Death, Career, and War are above 90 percent because the keywords of these types are relatively limited and straightforward. The types of Health and Other yielded the lowest F-measure of 80.06 percent and 77.71 percent, respectively. The potential reason for this is that stressors of these two types are more diverse and sparse, and they are not covered comprehensively by the current lexicons and patterns.

Table 7.

Classification performance of each type of stressors (%).

Type	P	R	F-measure
Abuse	97.84	92.65	95.17
Emotional	81.49	90.14	85.60
Relation	87.46	88.25	87.85
Trauma	90.82	100.0	95.19
Financial	95.00	73.07	82.60
Death	96.53	94.91	95.71
Health	89.73	72.27	80.06
Caring_for	78.20	85.76	81.81
Baby_birth	100.0	77.04	87.03
Homeless	83.87	93.55	88.45
Pressure	88.97	83.68	86.24
Transition	74.00	88.46	80.59
War	97.44	92.68	95.00
Other	82.05	73.80	77.71

In total, we collected 4794 stressors from 669 psychiatric notes among the whole corpus of 909 notes (327 notes with manual annotation, 342 notes with automatic annotation). Figure 4 illustrates the distribution of each type of stressors. The most frequent stressors are related to issues of emotions, abuse, health, family, and death.

Figure 4.

Distribution of different stressor types.

Association between stressors with suicide

Table 8 listed the stressor types of significant associations with suicide behaviors (p < 0.05). Among them, health issues are the most significant signal for suicide behaviors, following by pressure from job or schools, death of family members or people with other social relations, taking care of children or elderly people, and abusive behavior.

Table 8.

Statistically significant association between suicide ideations/attempts and stressors (p < 0.05).

Stressor type	p value
Health	0
Pressure	0.006855
Death	0.013101
Caring_for	0.016949
Abuse	0.045616

Discussion

The wide implementation of EHR systems provides a novel opportunity to psychiatric research and practice to embrace the “big data” era with rapidly accumulating psychiatric notes available digitally.²² By analyzing the full spectrum of observational and phenotypic data in EHR, computational and statistical approaches may yield findings beyond individual research.¹¹ Particularly, as an emerging novel technique to unlock information in psychiatric text in EHRs, NLP-based methods have been adopted for various applications recently, such as negative symptom recognition of schizophrenia,³⁵ cannabis use identification,³⁶ and predicting early psychiatric readmission.³⁷ Researchers are also attempting to differentiate suicidal patients with the others using information extracted automatically by NLP-based methods, including frequent words,³⁸ sentiment polarity,³⁹ and expressions in conversations.⁴⁰ However, few studies have been carried out on associations between suicide and stressors by leveraging information in psychiatric text. To the best of our knowledge, this is the first study to automatically extract and classify psychiatric stressors using NLP-based methods and conduct statistical association analysis between suicide ideations/attempts and stressors based on clinical narrative. Experimental results show that the employed NLP-based methods can yield promising performance for automatic suicide behavior and stressor extraction.

Existing studies of associations between stressors and suicide behavior mainly rely on survey data collected from patients and people in the same social community where patients at present. Different sets of stressors and related questions are designed for the survey, dependent on the specific cohorts (e.g. youth,¹⁸ adolescents,¹³ soldiers¹⁹) investigated in the studies. Common significant suicide signals found from previous works include health problems,^20,21 abusive experience of patients (particularly from their childhood),¹⁹ and pressure from the social environment.¹⁸ Similar findings are obtained from this study, demonstrating the feasibility of conducting statistical association analysis between suicide and psychiatric stressors using narrative text in EHR.

Although our system showed reasonable performance using inexact matching, challenges remain for stressor information extraction, mainly due to complex syntactic structures and poor formatting in clinical notes. For example, some stressors have multiple modifiers in conjunctive structures (e.g. school and job stressors), with each modifier indicating a different stressor. Special pattern-based rules need to be designed for the identification and classification of such stressors. Indeed, clinical text may be ill-formed due to dependence on the writing habit of physicians. For example, some sentences consist of multiple clauses without any punctuations to connect them syntactically, such as in “Since he and his most recent girlfriend ended their relationship he has been homeless but recently he began receiving housing resources and is in the process of trying to find more permanent housing.” Such ill-formed text may cause boundary errors and false negative errors for stressors. Furthermore, a majority of false negative errors are caused by stressors of rare and complex patterns, which cannot be covered either by machine learning–based methods or by rules. Examples of rare patterns are sentences such as “Recently, there were questions whether the medications would be covered.” and “She and her husband are currently planning to sue husband’ s family over parents’ estate.” Another type of false negative errors is caused by co-reference, as in the sentences of “Her colleague was forced out and there was concern that she might be in a similar situation.” and “I’m a Manager in Public Administration and it’s been stressful.” To resolve such errors, co-reference resolution should be conducted first.

The performance of suicidal behavior recognition reported in our study is much higher than the performance reported in Hammond and Laundry¹⁹ (F-measure: 97.3% vs 83.1%). There are two possible reasons for this comparison: (1) different datasets were used in these two studies. Initial psychiatry evaluation records were used in our study, with relatively straightforward mentions of suicidal behaviors. In contrast, a wide range of clinical documents were used in the study of Hammond and Laundry.¹⁹ Their candidate documents were retrieved using a keyword-based search engine, and the relevance of the documents with suicidal behaviors was not guaranteed. (2) Since a modest dataset of 327 notes was used in this study, the performance may suffer from an overfitting problem. We tried to alleviate this problem using 10-fold cross validation for feature and parameter tuning on the training set and reported the final performance on the test set. More thorough evaluation will be conducted in the future on large-scale datasets with different types of clinical documents like in Hammond and Laundry¹⁹ to examine our method of suicidal behavior identification. Further work should also be conducted in the future to validate the method on multiple datasets such as MIMIC II. Moreover, the domain expert could manually review the stressor extraction and classification results in a random sample of notes to check the feasibility of the method for practical applications.

One limitation of this study is the small sized data of 909 psychiatric notes used for statistical association analysis between suicide and stressors. Despite that similar findings were obtained from this study with previous works using survey data, the limited data size hinders the definition of stressor types with finer granularity and in-depth association analysis focusing on specific cohorts with different demographic information. Another limitation of this work is that we mainly used the CRF algorithm for psychiatric stressor recognition. The algorithm was chosen because it was widely applied with a strong performance of NER. We will compare the CRF algorithm and other machine learning–based methods including the deep learning–based methods for stressor recognition in the next step. Besides, the longitudinal development paths of stressors are not considered in our current analysis. Acute and chronic stressors may have different roles in the formation of suicide ideations/attempts.⁴¹ Temporal modifiers of suicide behaviors and stressors will also be recognized using NLP-based methods in the near future to stratify stressors of different association with suicide behaviors. In addition, except for statistical associations, NLP-based systems could also be developed to extract the direct cause–effect relation between suicide behaviors and stressors from clinical narrative, so that more precise stressor signals of suicide behaviors can be obtained.

Conclusion

This study takes the initiative to automatically extract and classify psychiatric stressors from clinical text using NLP-based methods and conduct a statistical association analysis with suicide behaviors. Experimental results demonstrate the feasibility of using NLP approaches to unlock information from psychiatric notes in EHR, in order to facilitate large-scale studies about associations between suicide and psychiatric stressors.

Footnotes

Appendix 1 Acknowledgements

We thank the organizers of the CEGS N-GRID 2016 challenge for providing the corpus.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: H.X. is a founder of Melax Technologies, Inc., which licenses the CLAMP software.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Library of Medicine (grant number 2R01LM010681-05).

ORCID iD

Salih Selek

References

Grunebaum

. Suicidology meets “big data.” J Clin Psychiatry 2015; 76(3): e383–e384.

Center for Substance Abuse Treatment. Addressing Suicidal Thoughts and Behaviors in Substance Abuse Treatment. Treatment Improvement Protocol (TIP) Series 50. HHS Publication No. (SMA) 09-4381. Rockville, MD: Substance Abuse and Mental Health Services Administration, 2009.

Curtin

Warner

Hedegaard

Increase in suicide in the United States 1999–2014. NCHS Data Brief 2016; 241: 1–8.

Smith

Kim

Ganoczy

, et al. Suicide risk assessment received prior to suicide death by Veterans Health Administration patients with a history of depression. J Clin Psychiatry 2013; 74(3): 226–232.

Dredze

Broniatowski

Smith

, et al. Understanding vaccine refusal: why we need social media now. Am J Prev Med 2016; 50(4): 550–552.

Adler

Bush

Barg

, et al. A mixed methods approach to identify cognitive warning signs for suicide attempts. Arch Suicide Res 2016; 20: 528–538.

World Health Organization. Towards evidence-based suicide prevention programmes.

Wongkoblap

Vadillo

Curcin

Researching mental health disorders in the era of social media: systematic review. J Med Internet Res 2017; 19: e228.

Cohan

Young

Goharian

Triaging mental health forum posts. In: Proceedings of the 3rd workshop on computational linguistics and clinical psychology: from linguistic signal to clinical reality, San Diego, CA, 16 June 2016, pp. 143–147. Stroudsburg, PA: The Association for Computational Linguistics.

10.

Burnap

Colombo

Scourfield

. Machine classification and analysis of suicide-related communication on twitter. In: Proceedings of the 26th ACM conference on hypertext & social media, Morphou, Northern Cyprus, 1–4 September 2015, pp. 75–84. New York: ACM.

11.

Stewart

Davis

“Big data” in mental health research: current status and emerging possibilities. Soc Psychiatry Psychiatr Epidemiol 2016; 51(8): 1055–1072.

12.

Wilcox

Fawcett

Stress, trauma, and risk for attempted and completed suicide. Psychiat Ann 2012; 42(3): 85–87.

13.

Kim

Baek

Han

, et al. Psychosocial-environmental risk factors for suicide attempts in adolescents with suicidal ideation: findings from a sample of 73,238 adolescents. Suicide Life Threat Behav 2015; 45(4): 477–487.

14.

Gould

Greenberg

Velting

, et al. Youth suicide risk and preventive interventions: a review of the past 10 years. J Am Acad Child Adolesc Psychiatry 2003; 42(4): 386–405.

15.

Friedman

Resick

Bryant

, et al. Classification of trauma and stressor-related disorders in DSM-5. Depress Anxiety 2011; 28(9): 737–749.

16.

World Health Organization. Prevention of mental disorders: effective interventions and policy options: summary report. Geneva: World Health Organization, 2004.

17.

McHugh

ML.

The chi-square test of independence. Biochemia Medica 2013; 23(2): 143–149.

18.

You

Chen

Yang

, et al. Childhood adversity, recent life stressors and suicidal behavior in Chinese college students. PLoS ONE 2014; 9(3): e86672.

19.

Hammond

Laundry

. Application of a hybrid text mining approach to the study of suicidal behavior in a large population. In: Proceedings of the 47th Hawaii international conference on system sciences (HICSS), Waikoloa, HI, 6–9 January 2014, pp. 2555–2561. Piscataway, NJ: IEEE.

20.

Rich

Warsradt

GM.

Suicide, stressors, and the life cycle. Am J Psychiatry 1991; 148(4): 524–527.

21.

Shakya

Common stressors among suicide attempters as revealed in a psychiatric service of Eastern Nepal. J Trauma Stress Disor Treat 2014; 3: 3.

22.

Monteith

Glenn

Geddes

, et al. Big data are coming to psychiatry: a general introduction. Int J Bipolar Disord 2015; 3(1): 21.

23.

Zhang

, et al. Psychiatric symptom recognition without labeled data using distributional representations of phrases and on-line knowledge. J Biomed Inform 2017; 75(Suppl.): S129–S137.

24.

Anderson

Pace

Brandt

, et al. Monitoring suicidal patients in primary care using electronic health records. J Am Board Fam Med 2015; 28(1): 65–71.

25.

ICD—9—International classification of diseases, ninth revision. https://www.cdc.gov/nchs/icd/icd9.htm

26.

Liakata

Rebholz-Schuhmann

Biological network extraction from scientific literature: state of the art and challenges. Brief Bioinform 2013; 15: 856–877.

27.

Cho

H-C

Okazaki

Miwa

, et al. Named entity recognition with multiple segment representations. Inform Process Manag 2013; 49(4): 954–965.

28.

Zhang

, et al. Interweaving domain knowledge and unsupervised learning for psychiatric stressor extraction from clinical notes. In: Proceedings of the international conference on industrial, engineering and other applications of applied intelligent systems, Arras, 27–30 June 2017, pp. 396–406. Cham: Springer.

29.

Tang

Feng

Wang

, et al. A comparison of conditional random fields and structured support vector machines for chemical entity recognition in biomedical literature. J Cheminformatics 2015; 7(Suppl. 1): S8.

30.

Collobert

Weston

. A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th international conference on machine learning, Helsinki, 5–9 July 2008, pp. 160–167. New York: ACM.

31.

Mnih

Hinton

GE.

A scalable hierarchical distributed language model. Adv Neur In 2009; 21: 1081–1088.

32.

Mikolov

Chen

Corrado

, et al. Efficient estimation of word representations in vector space. arXiv. Epub ahead of print 7 September 2013. DOI: 1301.3781v3.

33.

Guo

Che

Wang

, et al. Revisiting embedding features for simple semi-supervised learning. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Doha, Qatar, 25–29 October, pp. 110–120. Stroudsburg, PA: The Association for Computational Linguistics.

34.

Saeed

Villarroel

Reisner

, et al. Multiparameter Intelligent Monitoring in Intensive Care II (MIMIC-II): a public-access intensive care unit database. Crit Care Med 2011; 39(5): 952–960.

35.

Gorrell

Jackson

Roberts

, et al. Finding negative symptoms of schizophrenia in patient records. In: Proceedings of the workshop on NLP for medicine and biology associated with RANLP 2013, Hissar, 13 September, pp. 9–17. Shumen: INCOMA Ltd.

36.

Patel

Wilson

Jackson

, et al. Cannabis use and treatment resistance in first episode psychosis: a natural language processing study. Lancet 2015; 385: S79.

37.

Rumshisky

Ghassemi

Naumann

, et al. Predicting early psychiatric readmission with natural language processing of narrative discharge summaries. Transl Psychiatry 2016; 6(10): e921.

38.

Poulin

Shiner

Thompson

, et al. Predicting the risk of suicide by analyzing the text of clinical notes. PLoS ONE 2014; 9(1): e85733.

39.

McCoy

Castro

Roberson

, et al. Improving prediction of suicide and accidental death after discharge from general hospitals with natural language processing. JAMA Psychiatry 2016; 73(10): 1064–1071.

40.

Pestian

Grupp-Phelan

Bretonnel Cohen

, et al. A controlled trial using natural language processing to examine the language of suicidal adolescents in the emergency department. Suicide Life Threat Behav 2016; 46: 154–159.

41.

Bryan

Clemans

Leeson

, et al Acute vs. chronic stressors, multiple suicide attempts, and persistent suicide ideation in US soldiers. J Nerv Ment Dis 2015; 203(1): 48–53.