Using sentiment analysis to detect affect in children’s and adolescents’ poetry

Abstract

Sentiment analysis is a computational method that automatically analyzes the valence of massive quantities of text. Basic sentiment analysis involves extracting and counting emotionally-laden keywords from passages of text (e.g., hate, love, happy, sad). This study describes using sentiment analysis to explore changes in emotion expression in a developmental context. A sample of n = 8,688 poems published online by children and adolescents from Grade 4 to Grade 12 was analyzed. Sentiment analysis coded words as positive or negative and these were averaged within each poem to obtain its relative percentage of positive and negative sentiment. Polynomial regressions explored linear and nonlinear trends in sentiment scores by grade. Among the results, negative sentiment demonstrated an upward curvilinear trend, increasing sharply from Grade 6 to Grade 11 and then decreasing afterward. Positive sentiment demonstrated a sinusoidal pattern throughout development. Overall, these findings are consistent with previous research on the progressions of emotion expression in childhood and adolescence. Despite some limitations, sentiment analysis presents an opportunity for researchers in developmental psychology to explore basic questions in emotional development using large quantities of data.

Keywords

Sentiment analysis emotion child adolescence development opinion mining methodology

The study of emotional development is largely concerned with assessing emotions at different stages of life. This is often achieved using self-reports, behavioral observations, and indices of psychophysiology across multiple cohorts or longitudinally. Beyond these approaches, the ubiquity of publicly available textual data online represents untapped potential for researchers to explore central questions about emotional development. Children and adolescents use the internet to publicly share thoughts, opinions, and feelings with the online community. The content of this text can reveal, among other things, patterns in how emotion expression changes across development. However, the sheer quantity of available data makes manually coding passages of text an insurmountable task. Fortunately, computational techniques that once required an extensive knowledge of coding on the part of the analyst are becoming more accessible and easier to implement.

Sentiment analysis (sometimes also called opinion mining) refers to the use of algorithms or machine learning to detect the emotional content of text (Bing, 2015). One clear advantage of sentiment analysis is that it can be applied to huge corpuses (bodies of literature) on the order of millions of passages of text. This in turn provides insights into the overall emotional content of the corpus and what factors may elicit changes in sentiment or opinions. Most commonly, sentiment analysis derives positive and negative valence from passages of text. Typical examples include instant messages, product reviews, books, and tweets.

In its simplest form, this procedure identifies and counts sentiment words in passages of text. These sentiment words come from large lists, or lexicons, of frequently used words that have been assigned sentiment value (e.g., positive, neutral, negative). Many older lexicons were developed manually, but contemporary lexicons are derived using either machine learning approaches or (as with the lexicon used in this study) thousands of human annotators through Amazon’s Mechanical Turk (Mohammad & Turney, 2010). It is worth noting that lexicons exist in many flavors and can be used for diverse purposes beyond assessing valence (e.g., WordNet lexicon to examine hierarchical nesting of terms within categories; Miller, 1995). However, lexicons for sentiment analysis are among the most widely applicable for exploring research questions in psychology.

In the era of Big Data, sentiment analysis can provide novel insights into psychological phenomena on a larger scale. For example, researchers used sentiment analysis on news articles and found that people reported greater physical and depressive symptoms following weeks where they were exposed to more negative media content (Wormwood, Devlin, Lin, Barrett, & Quigley, 2018). Sentiment analysis has also been explored as a means of assisting in identifying depression through social media posts (Jung, Park, & Song, 2017). The advantages of sentiment analysis extend to more basic research as well, in that it can explore and re-examine big theoretical questions with large quantities of data. In particular, the approach lends itself to investigating subtle trends, such as how attitudes change over time or coincide with socio-political events (e.g., Rill, Reinel, Scheidt, & Zicari, 2014). Given that sentiment analysis can identify affective properties of text efficiently and with relative accuracy, sentiment analysis may also be leveraged to investigate developmental changes in affect.

Developmental Change in Affect

During adolescence, social, cognitive, biological, and experiential factors contribute to heightened stress and negative emotions (Elkind, 1967; Forbes & Dahl, 2010; Hollenstein & Lougheed, 2013; Somerville, 2013). As a brief overview, the early to middle childhood period (4-9 years of age) is characterized by inflated self-esteem and positive affect. This gives way to early adolescence (10-14 years) and late adolescence (15-18 years), wherein social-evaluative pressures dampen one’s self-image (Barnes, Hoffman, Welte, Farrell, & Dintcheff, 2007; Roeser, Eccles, & Sameroff, 2000). Romantic relationships become more salient, heralding unfamiliar and sometimes undesirable emotions (Ha, Dishion, Overbeek, Burk, & Engels, 2014). Finally, mood disorders such as depression and anxiety also become more prevalent, resulting in an increased risk of self-harm and suicidal ideation (Garber, Keiley, & Martin, 2002; Moran et al., 2012). Taken together, the literature points to the childhood-adolescence transition as one of intensifying negative affect.

Developmental changes in affect during childhood and adolescence can be understood in terms of changes in valence (i.e., positive or negative). For example, although positive affect is generally more common than negative affect throughout development, positive affect tends to decrease over time, particularly during early and middle adolescence (Larson, Moneta, Richards, & Wilson, 2002; Weinstein et al., 2007). Researchers have alluded to adolescents experiencing a pile-up of disruptive life changes that has an adverse effect on mood and perceived well-being (Simmons, Burgeson, Carlton-Ford, & Blyth, 1987). However, during late adolescence, most youths adjust to these stressors and positive affect rebounds. Overall, these findings suggest that average affect, indeed, changes throughout development. The goal of this study was to demonstrate how sentiment analysis can be applied to this phenomenon using a corpus of over 8,000 online poems published by children and adolescents.

The Current Study

Online textual data is freely available in massive quantities (e.g., Project Gutenberg). There are numerous online platforms designed specifically for children and adolescents to anonymously publish their creative writing. One such website allows children to publish poetry to be viewed by the online community. Poetry is optimal for sentiment analysis because it is relatively short, yet popular literary medium for self-expression. Indeed, poet Robert Frost once penned that “Poetry is when an emotion has found its thought and the thought has found words”. Given that poems are often laden with emotional content (Belfi, Vessel, & Starr, 2018), it was postulated that mining the affective content of online poems could offer insights into changes in emotions over time on a grand scale. Not only does this approach greatly expand the sample size compared to research using self-reports, but it also spans a wider age range.

Drawing upon the extant literature (Larson et al., 2002; Weinstein et al., 2007), it was hypothesized that negatively valenced words would generally appear more frequently with older children and adolescents, but that the trend would be nonlinear such that negative sentiment would be strongest during the early to middle adolescence period. An inversed trend for positively valenced words was anticipated, with positive sentiment decreasing dramatically during early- to middle-adolescence, then rebounding slightly into late adolescence.

Method

Corpus

A corpus of n = 8,688 poems published online by children in Grades 4 to 12 (spanning approximately 8 to 18 years of age) was compiled. The average poem length was 71.44 words (SD = 74.35) and ranged from one word in length to 2,516 words. Although the average represents an ideal length for sentiment analysis, one poem was excluded because it was considerably longer than the others, resulting in a range of one to 1,255 words. In subsequent analyses, poems were also excluded that did not include a minimum number of words, which is described in detail in proceeding sections.

Poems were extracted at the outset of 2018 from a website where school-age children can submit poetry online which becomes openly accessible for the online community. Based on the web domain, it was presumed that a significant portion of poems were submitted for class assignments, and under the submission guidelines it is clear that the intent was for freely accessible online publication. Permissions were obtained from the domain host to extract and analyze the poems.

Online submissions identified the first name and last initial of the writer, their school grade, and the state/country in which they live; however, only the grade of the writer and the poem itself were recorded in the corpus. Grade is a suitable proxy for age given that the majority of poems appear to have been submitted from North America which largely follows a standardized grading system. For each grade, a sample of 1,000 poems was analyzed¹, except for Grade 11 where there were only 688 poems available. The specific date range of these submissions is not clearly stated, though judging by the content of some poems (e.g., references to 9/11 terrorist attacks) it is assumed that they cover the early 2000s to the present day.

Analytic Procedure

All analyses were conducted using R statistical software (v. 3.5.1, R Core Team, 2018) and followed procedures outlined by Silge and Robinson (2017) for basic text analysis. Regressions were also conducted in R using the base lm function. R code used in this analysis is available as Supplementary Material. Figure 1 summarizes the standard preliminary steps for tidying and analyzing raw text: (1) Tokenization, (2) Lemmatization, and (3) Stop word removal.

Figure 1.
Preprocessing of raw text to sentiment analysis (example depicted is written by a Grade 10 student). Sentiment scoring of this example is .73 negative (11 negative tokens / 15 non stop words) and .13 positive (2 positive tokens / 11 non stop words).

Tokenization

The term token refers to the unit of analysis, be it individual words (unigrams), series of words (n-grams), or entire sentences; though most sentiment analyses are concerned with individual words. In the current analysis, a unigram token is any string of characters separated by a space (i.e., a single word). The process of tokenization involves breaking down a document of text into individual words that can be analyzed. At the same time, tokenization removes punctuation and casing making it easier to identify frequently used words. There are a number of software packages and programs available for tokenizing text, but this analysis used the tidytext package in R (v. 0.2.0; Silge & Robinson, 2018). In this analysis, only unigrams were analyzed, which means that words are entirely stripped of context. For instance, in the sentence “I am not happy” the word happy is indexed as positive even though the context refers to an absence of happiness.

Lemmatization

Lemma is a linguistics term that refers to the canonical form of a word. Most nouns and verbs in the English language have varying permutations depending on pluralization and tense. Thus, the word write can be expressed differently as writes, writing, and wrote. Sentiment lexicons do not index all permutations of certain words because it is time- and resource-intensive to annotate these multifarious forms. For instance, the words betray and betrayal are indexed as negative, but semantically identical words betrays and betrayed are not indexed and would be neglected in subsequent analyses. Currently, no lemmatization functions or packages are available in R, but software developed by Schmid (TreeTagger; 1994–2007) uses decision tree algorithms that generate probability likelihoods of words pertaining to particular lemmas with 96.36% accuracy. This technique does not lemmatize all terms, though, as is evident in the example depicted in Figure 1. This approach was used to lemmatize the corpus of poems to better capture the frequency with which children use sentiment words.

Stop word removal

Basic unigram text analytics involves identifying words with semantic meaning. In sentiment analysis, we are concerned with sentiment words, but other types of text analysis may focus on words that convey a specific attitude (e.g., for or against abortion). Stop words, such as it, because, the, and, convey very little or no semantic meaning independent of other words and are mainly used for sentence structure and grammar. It is customary to remove stop words as they are not particularly helpful in building text analytic models. The stop_words function in tidytext package (Silge & Robinson, 2018) contains a list of 1,149 stop words obtained from three machine learning-derived lexicons (e.g., Lewis, Yang, Rose, & Li, 2004). Stop words were removed from the analysis resulting in greater overall ratios of sentiment words to total meaningful words.

Sentiment analysis

Sentiment analysis first involves aligning the tidied text with a sentiment lexicon. The NRC Emotion Lexicon (Mohammad & Turney, 2010) was used in this analysis, which indexes the valence (positive and negative²) of approximately 4,000 commonly used English lemmas. Mohammad and Turney developed this lexicon using Amazon’s Mechanical Turk to recruit annotators who then rated words according to their valence. In their development of the lexicon, words with high interrater agreement were indexed according to their sentiment characteristics. The NRC Emotion Lexicon has been used to identify sentiment in a variety of contexts including tweets (Mohammad, Kiritchenko, & Zhu, 2013), fairy tales, and hate mail (Mohammad, 2012); and ranks among the most widely used lexicons for sentiment analysis (Mohammad, Bravo-Marquez, Salameh, & Kiritchenko, 2018).

The final step in the pipeline involves calculating positive and negative sentiment scores for each poem. This is done by running inner_join functions from the dplyr package in R (v. 0.7.8; Wickham, 2018), which takes a data column of tokens and aligns it with a reference column of lexicon words that contain a sentiment label. Most words in the NRC lexicon are labeled as either positive (e.g., love, happy, smile) or negative (e.g., sad, cry, afraid). For each valence label (positive or negative), a word is assigned a value of 1 for pertaining to that valence or 0 for not pertaining to that valence. Neutral words, such as chair and dog, are indexed in this lexicon as well and receive a score of 0 for both positive and negative valence. Any other non-indexed words (i.e., words not in the lexicon) were assigned as neutral, provided they were not excluded as stop words. Some words, such as feeling and emotion register as ambiguous, which means that they are assigned a value of 1 for both positive and negative valence. The results presented included these ambiguous terms, but models were also run with these terms excluded, which revealed similar findings.

The total positive and negative valence for each poem was calculated by adding the counts of positive and negative words. However, given that older children tend to write longer poems, it was important to account for the increased likelihood of observing sentiment words in later grades due to higher word count. Ratios were computed by dividing the number of positive words by the number of non stop words within each poem, and likewise for negative words (scores ranging from 0 neutral to 1 highest). The resulting scores can be represented as a percentage of positive or negative words relative to other meaningful words within a poem. It is worth mentioning that sentiment analysis of a large corpus of text is typically less concerned with accurately scoring individual pieces of text (poems) and more concerned with capturing the overall distribution of sentiment of a corpus. In the current analysis, the objective is thus to get a sense of the distributions of positive and negative valence within each grade and how they change over time.

Because basic sentiment analysis is based on word counts, it is difficult to assess the sentiment of very brief poems. Poems containing very few words are prone to overestimation of sentiment even if only one sentiment word appears³ (Bing, 2015). Thus, as a final data cleaning step, poems containing fewer than five meaningful words (e.g., non stop words)⁴ were omitted from analyses. This removed 312 poems from the sentiment analysis, resulting in the following breakdown by grade: Grade 4 = 955, Grade 5 = 953, Grade 6 = 957, Grade 7 = 970, Grade 8 = 972, Grade 9 = 975, Grade 10 = 966, Grade 11 = 673, Grade 12 = 954.

Results

Common Words

To get a sense of the how the content of the poems changes with age, the top ten frequently used meaningful words were analyzed within three age groups: Grades 4-6, 7-9, and 10-12 (see Figure 2). The word love was used the most across all age groups and words such as day, time, people, and friend were consistently prevalent. There were notable differences between the top ten words used by the youngest and the middle group. Namely, for younger children (Grades 4-6), the object words dog and cat were among the top ten words, and several positive terms such as eat, play, and fun appeared in this list. Among the middle and older age groups (Grade 7-9, 10-12), reflective words such as life, world, and heart were among the top ten words. Older adolescents (Grade 10-12) also had leave among their top ten words.

Figure 2.
Top ten frequently used non stop words by age group. Graphic produced using ggplot2 (v. 3.1.0, Wickham, 2018).

Sentiment Analysis

Descriptive statistics are presented in Table 1. On average, poems contained approximately 20% positive words (SD = .15) and 14% negative words (SD = .12). Nevertheless, the high standard deviations of these estimates point to the substantial variability in sentiment scoring. Positive sentiment was significantly and negatively correlated with negative sentiment (r = -.24, p < .001), which provides moderate support for assessing these terms separately. However, the correlation was small, suggesting that there is some inconsistency in the pattern of positive and negative sentiment across poems.

Table 1.
Means and standard deviations for positive and negative affect per grade.

Positive Negative

Grade M SD M SD

4 .191 .164 .118 .129

5 .194 .157 .119 .129

6 .208 .160 .122 .118

7 .199 .146 .132 .124

8 .195 .138 .141 .122

9 .205 .140 .152 .117

10 .196 .126 .151 .117

11 .204 .132 .155 .115

12 .206 .132 .146 .116

Total .201 .153 .136 .127

Note. Value is the proportion of emotion words to non-emotion words (ranging from 0 to 1) for positive and negative sentiment, respectively. N = 8,375

Linear and nonlinear associations were tested between grade and positive and negative affect using 2nd (quadratic) and 3rd (cubic) order polynomials within the lm function in R (see Table 2). Separate regression equations were calculated for positive and negative affect using grade as a predictor. The cubic regression term significantly predicted negative sentiment above the quadratic and linear models, although the overall variance explained amounted to only 1.2%. The trend shows negative sentiment increasing sharply from Grade 6 to Grade 11 then dropping down somewhat in Grade 12. Looking at the means for each grade (see Figure 3a), negative sentiment increased by roughly 4% from Grade 4 to its peak at Grade 11. Linear and nonlinear models for positive sentiment were not significant.

Table 2.
Linear and nonlinear regressions predicting negative and positive affect from grade.

Model R² adj df Estimate 95% CI t p

Negative Affect .012*** 3, 8371

Linear (Grade) -.089 [-.201, .024] -1.545 .122

Quadratic (Grade²) .037 [.011, .062] 2.844 .004

Cubic (Grade³) -.003 [-.004, -.001] -3.231 .001

Positive Affect .000^† 3, 8371

Linear (Grade) .106 [-.007, .219] 1.835 .067

Quadratic (Grade²) -.022 [-.048, .003] -1.698 .090

Cubic (Grade³) .001 [-.001, .003] 1.666 .096

Positive Affect ‘love omitted’ .001* 3, 8371

Linear (Grade) .166 [.053, .279] 2.883 .004

Quadratic (Grade²) -.041 [-.066, -.015] -3.117 .002

Cubic (Grade³) .003 [.001, .004] 3.246 .001

Note. ^†p < .10, p < .05, p < .01, p < .001.

Estimates are based off standardized variables.

N = 3,875

Figure 3.
Developmental trend in negative affect (A) and positive affect with ‘love’ words removed (B). Affect scale ranges from 0 (neutral) to 1 (high in positive/negative affect). Bold lines represent fitted cubic regression lines and grey area represents 95% confidence interval around fitted values. Black dots represent means for each grade, and black error bars represent standard errors. Graphic produced using ggplot2 (v. 3.1.0, Wickham, 2018).

As an additional exploratory analysis, the model for positive sentiment was reanalyzed with the word love omitted from the sentiment analysis. This was done by temporarily labeling love as a stop word. Love was the most frequently used non stop word across all grades, occurring 3,923 times throughout the corpus, but increasing in relative frequency with grade (r = .13, p < .001). Love is indexed as a positive sentiment, though may be used more frequently in negative contexts among adolescents as they explore the trials and tribulations of romantic relationships, which include heartbreak and rejection. When these words were removed, the cubic regression for positive sentiment was significant above the quadratic and linear models; but again, the variance explained was very small (< 1%). The resulting model shows a similar peak in positivity at Grade 6, but then drops into Grade 8 followed by a more uniform upward trend (see Figure 3b).

Discussion

The current study provides preliminary evidence that sentiment analysis can detect developmental differences in the written expression of affect from middle childhood to late adolescence. On average, children’s poems contained 20% positive words and 14% negative words. Although there are no established guidelines of interpretation for these percentages, similar ratios have been observed in sentiment analysis of novels (26% positive, 17% negative; Mohammad, 2012), and the current results are consistent with previous research which found that positive affect is experienced more frequently than negative affect (Larson et al., 2002; Weinstein, 2007). More importantly, however, were the changes in the frequency of positive and negative affect over different grades. Overall, these trends mirrored those in previous research using self-report measures, providing initial support for sentiment analysis as a useful tool for developmental researchers.

Developmental Trends in Sentiment

One way to examine developmental differences with this approach is to look at which words are used most frequently within different age groups. Analysis of the top ten frequently used words demonstrated that children in Grades 4-6 used more concrete terms, such as dog and cat, whereas both early and late adolescents used more abstract, affect-laden terms, such as feel and heart. These findings underlie the broader trend of change in affect but are also indicative of changes in social-cognitive and linguistic abilities (Tousignant, Sirois, Achim, Massicotte, & Jackson, 2017). Children become increasingly skilled at understanding their own emotions and inferring emotion states in others (Pons & Harris, 2005), and these competencies appear to manifest in their writing.

Of greater interest, was how the use of sentiment words changed across developmental periods. The relative frequency of negative sentiment words (e.g., pain, hurt, hate, die) increased by an average of 4% from childhood to adolescence. This coincides with early to middle adolescence as a period of heightened stress and negative mood (Steinberg & Morris, 2001). As children transition into adolescence they are exposed to new and stressful events that elicit intense negative emotions. For instance, parent-child conflict reaches peak intensity during early and middle adolescence (Laursen, Coy, & Collins, 1998), and adolescents are generally more at-risk for depressive and anxious symptomology (Garber et al., 2002). Cumulatively, these stressors result in normative increases in negativity during adolescence.

The pattern of results for positive affect are more difficult to interpret. In contrast with the hypothesis, positive sentiment words (e.g., happy, smile, sweet, hope) fluctuated over this same period. Exploratory analysis with the word love omitted displayed a sinusoidal pattern of positive sentiment rising, falling, and rising again. Contrary to the current findings, Weinstein et al. (2007) found that positive affect decreased during adolescence, whereas negative affect did not change. Poetry may be a less accurate representation of children’s positive expression compared to negative expression, which would explain why positive sentiment was lowest in middle childhood (Grade 4) when it is assumed to be at its peak (Larson et al., 2002). In this regard, the current finding may be more indicative of changes in young children’s cognitive-linguistic abilities. Young children write more about concrete objects rather than thoughts and feelings (Borensztajn, Zuidema, & Bod, 2009). It is possible that the observed increase in positive sentiment during Grade 6 represents an increase in the use of abstract positive terms such as beautiful and heart.

Taken together, it is premature to conclude that these results signify developmental differences in emotion expression. For instance, the increased frequency of negative words may reflect children’s growing vocabulary, increased self-reflection, or progressions in symbolic and abstract thinking about emotions. The analysis of poetry may have also influenced these findings because poetry is a form of artistic expression that makes extensive use of figurative and hyperbolic language, which may impact children’s use of emotion words (Jack, 2018). There is evidence to suggest that older children are more sophisticated in their understanding and implementation of figurative language, thus offering an alternative explanation for the pattern of results (Demorest, Silberstein, Gardner, & Winner, 1983). Moreover, writing emotionally-laden content does not necessarily imply that the writer is experiencing those emotions. For instance, older children and adolescents may be effusive in their writing on pressing global issues (e.g., climate change awareness; Devine-Wright, Devine-Wright, & Fleming, 2004), but this does not imply that they are experiencing these emotions regularly.

Evaluating the Utility of Sentiment Analysis in Developmental Research

Sentiment analysis appears to hold initial promise as a method in developmental research as the current findings fit with the broader pattern of emotional development. However, there are notable limitations in the current demonstration and with sentiment analysis as a method. First, the polynomial regressions should be interpreted with caution because grade explained no greater than 1% of the variance in negative affect (and < 1% in positive affect). Although there is a trend, its interpretability is seriously dampened by noise, suggesting that statistical significance is mostly attributable to a large sample size. This lack of predictive power is not surprising given that only grade was included as a predictor, despite there being myriad factors involved in emotional development. More problematic was that affect was calculated by simply counting sentiment words outside of their context. Indeed, children’s writing is greater than the sum of its words; thus, although a broken heart hurts more than a broken chair, sentiment analysis based on counts ignores this nuance. Nevertheless, it is noteworthy that the pattern of findings largely mirrors developmental progressions in emotional development (e.g., Larson et al., 2002).

Depending on the analyst’s goals, one might choose to implement more sophisticated analytical approaches to overcome these issues. For instance, other lexicons, such as the Valence Arousal Dominance lexicon (Mohammad, 2018), use a continuous scoring approach rather than a dichotomous one, which allows for gradations in scoring as well as multiple dimensions of emotion. Additionally, the current analysis was limited in that it did not distinguish between the phrases such as “I am happy” and “I am not happy”, because the word happy triggers positive sentiment regardless of the preceding negator. A bigram approach can be easily implemented in the tidytext package to identify and reverse-score tokens that are preceded by a negator (Silge & Robinson, 2018). This was less of an issue for the current analysis as the goal was more so to ascertain the overall distribution of emotion words and not to accurately label passages of text as positive or negative. Nevertheless, more sophisticated computational approaches, such as deep learning algorithms and neural networks, would help to overcome issues pertaining to context (e.g., Kalchbrenner, Grefenstette, & Blunsom, 2014).

Finally, the web domain that hosted these poems was an educational platform, thus a substantial portion of poems may have been submitted for class assignments. Teachers may have imposed limitations on the structure and content of poems to focus on topics covered in class. It is also possible that students published their poems for class assignments and were reluctant to divulge intimate thoughts and feelings. Moreover, older grades were underrepresented in this corpus. It may be that only those adolescents who were intrinsically motivated to write poetry made the effort to submit their work online. Differing motivations across development could bias the sentiment of poems toward greater emotional content by adolescents, as they may have been more compelled to share their feelings about stressful life events. Despite these drawbacks, one advantage of using this sample is that there is a reduced risk of self-selection bias relative to other online publishing domains if students were encouraged to submit their writing for classwork.

Conclusion

The current study was intended as an initial proof of concept of the potential utility for sentiment analysis in developmental science. As with the current study, sentiment analysis could be used to examine “big picture” questions in psychology from a different perspective. Beyond analyzing changes in emotion, recent developments in software and programming are making it possible to use text analytics to answer myriad research questions in developmental psychology. The coding and computation are not specific to studying sentiment, thus they may be harnessed to explore content-specific research questions. For instance, researchers could explore developmental trends in media literacy by examining children’s usage of terms such as computer and television. Moreover, other lexicons like the MRC Psycholinguistic Database can assess features of text such as concrete vs. abstract text, meaningfulness, and familiarity (Coltheart, 1981). Cognitive developmentalists could use these techniques to explore how children’s writing shifts from the concrete to more abstract over time. Thus, in the era of Big Data, online information that is freely available offers a wealth of empirical insight for those who know how to use it.

Supplemental material

JBD830248_supplemental_material - Using sentiment analysis to detect affect in children’s and adolescents’ poetry

JBD830248_supplemental_material for Using sentiment analysis to detect affect in children’s and adolescents’ poetry by William M. Bukowski, David G. Perry, Melisa Castellanos and Will E. Hipson in International Journal of Behavioral Development

	Positive	Negative
4	.191	.164	.118	.129
5	.194	.157	.119	.129
6	.208	.160	.122	.118
7	.199	.146	.132	.124
8	.195	.138	.141	.122
9	.205	.140	.152	.117
10	.196	.126	.151	.117
11	.204	.132	.155	.115
12	.206	.132	.146	.116
Total	.201	.153	.136	.127

Model	R² adj	df	Estimate	95% CI	t	p
Negative Affect	.012***	3, 8371
Linear (Grade)			-.089	[-.201, .024]	-1.545	.122
Quadratic (Grade²)			.037	[.011, .062]	2.844	.004
Cubic (Grade³)			-.003	[-.004, -.001]	-3.231	.001
Positive Affect	.000^†	3, 8371
Linear (Grade)			.106	[-.007, .219]	1.835	.067
Quadratic (Grade²)			-.022	[-.048, .003]	-1.698	.090
Cubic (Grade³)			.001	[-.001, .003]	1.666	.096
Positive Affect ‘love omitted’	.001*	3, 8371
Linear (Grade)			.166	[.053, .279]	2.883	.004
Quadratic (Grade²)			-.041	[-.066, -.015]	-3.117	.002
Cubic (Grade³)			.003	[.001, .004]	3.246	.001

Footnotes

Acknowledgements

I would like to acknowledge Robert Coplan for providing insightful critiques and recommendations on drafts of this manuscript. I would also like to acknowledge Saif Mohammad for his suggestions on the analyses.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Will E. Hipson

Notes

Supplemental material

Supplemental material for this article is available online.

References

Arnett

J. J.

(1999). Adolescent storm and stress, reconsidered. American Psychologist, 54, 317–326.

Barnes

G. M.

Hoffman

J. H.

Welte

J. W.

Farrell

M. P.

Dintcheff

B. A.

(2007). Adolescents’ time use: Effects of substance use, delinquency, and sexual activity. Journal of Youth and Adolescence, 36, 697–710. doi: https://doi.org/10.1007/s10964-006-9075-0.

Belfi

A. M.

Vessel

E. A.

Starr

G. G.

(2018). Individual ratings of vividness predict aesthetic appeal in poetry. Psychology of Aesthetics, Creativity, and the Arts, 12, 341–350. doi: https://dx-doi-org.web.bisu.edu.cn/10.1037/aca0000153.

Bing

(2015). Sentiment analysis. Cambridge, UK: Cambridge University Press.

Borensztajn

Zuidema

Bod

(2009). Children’s grammars grow more abstract with age – evidence from an automatic procedure for identifying the productive units of language. Topics in Cognitive Science, 1, 175–188. doi:10.1111/j.1756-8765.2008.01009.x.

Coltheart

(1981). The MRC psycholinguistic database. Quarterly Journal of Experimental Psychology, 33, 497–505.

Demorest

Silberstein

Gardner

Winner

(1983). Telling it as it isn’t: Children’s understanding of figurative language. British Journal of Developmental Psychology, 1, 121–134.

Devine-Wright

Fleming

(2004). Situational influences upon children’s beliefs about global warming and energy. Environmental Education Research, 10, 493–506. doi:10.1080/1350462042000291029.

Elkind

(1967). Egocentrism in adolescence. Child Development, 38, 1025–1034.

10.

Forbes

E. E.

Dahl

R. E.

(2010). Pubertal development and behavior: Hormonal activation of social and motivational tendencies. Brain and Cognition, 72, 66–72. doi:10.1016/j.bandc.2009.10.007.

11.

Garber

Keiley

M. K.

Martin

N. C.

(2002). Developmental trajectories of adolescents’ depressive symptoms: Predictors of change. Journal of Consulting and Clinical Psychology, 70, 79–95. doi: https://dx-doi-org.web.bisu.edu.cn/10.1037/0022-006X.70.1.79

12.

Dishion

T. J.

Overbeek

Burk

W. J.

Engels

R. C. M. E.

(2014). The blues of adolescent romance: Observed affective interactions in adolescent romantic relationships associated with depressive symptoms. Journal of Abnormal Child Psychology, 42, 551–562. doi:10.1007/s10802-013-9808-y.

13.

Hollenstein

Lougheed

J. P.

(2013). Beyond storm and stress: Typicality, transactions, timing, and temperament to account for adolescent change. American Psychologist, 68, 444–454. doi:10.1037/a0033586.

14.

Jack

(2018). Poetry and emotion. The Lancet: Perspectives, 391, 732–733. doi: https://doi.org/10.1016/S0140-6736(18)30265-4

15.

Jung

Park

H. A.

Song

T. M.

(2017). Ontology-based approach to social data sentiment analysis: Detection of adolescent depression signals. Journal of Medical Internet Research, 19, e259. doi:10.2196/jmir.7452.

16.

Kalchbrenner

Grefenstette

Blunsom

(2014). A convolutional neural network for modelling sentences. Proceedings of ACL 2014.

17.

Larson

R. W.

Moneta

Richards

M. H.

Wilson

(2002). Continuity, stability, and change in daily emotional experience across adolescence. Child Development, 73, 1151–1165. doi: https://doi.org/10.1111/1467-8624.00464.

18.

Laursen

Coy

K. C.

Collins

W. A.

(1998). Reconsidering changes in parent-child conflict across adolescence: A meta-analysis. Child Development, 69, 817–832.

19.

Lewis

D. D.

Yang

Rose

T. G.

(2004). RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5, 361–397.

20.

Miller

G. A.

(1995). WordNet: A lexical database for English. Communications of the ACM, 38, 39–41.

21.

Mohammad

S. M.

(2012). From once upon a time to happily ever after: Tracking emotions in mail and books. Decision Support Systems, 53, 730–741. doi:10.1016/j.dss.2012.05.030.

22.

Mohammad

S. M.

(2018). Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words. Proceedings of the Annual Conference for the Association for Computational Linguistics, Melbourne, Australia.

23.

Mohammad

S. M.

Bravo-Marquez

Salameh

Kiritchenko

(2018). SemEval-2018 Task 1: Affect in Tweets. Proceedings of International Workshop on Semantic Evaluation, New Orleans, LA.

24.

Mohammad

S. M.

Kiritchenko

Zhu

(2013). NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets. Proceedings of the Seventh International Workshop on Semantic Evaluation, SemEval’13, Atlanta, GA, pp. 321–327.

25.

Mohammad

S. M.

Turney

P. D.

(2010). Emotions evoked by common words and phrases: Using mechanical turk to create an emotion lexicon. Proceedings of the NAACL-HLT Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, Los Angeles, CA.

26.

Moran

Coffey

Romaniuk

Olsson

Borschmann

Carlin

J. B.

Patton

G. C.

(2012). The natural history of self-harm from adolescence to young adulthood: A population-based cohort study. Lancet, 379, 236–243. doi: https://doi.org/10.1016/S0140-6736(11)61141-0

27.

Pons

Harris

(2005). Longitudinal change and longitudinal stability of individual differences in children’s emotion understanding. Cognition and Emotion, 19, 1158–1174. doi:10.1080/02699930500282108

28.

R Core Team. (2018). R: A language and environment for statistical computing. Vienna, Australia: R Foundation for Statistical Computing.

29.

Roeser

R. W.

Eccles

J. S.

Sameroff

A. J.

(2000). School as a context of early adolescents’ academic and social-emotional development: A summary of research findings. Elementary School Journal, 100, 443–471.

30.

Rill

Reinel

Scheidt

Zicari

R. V.

(2014). PoliTwi: Early detection of emerging political topics on twitter and the impact on concept-level sentiment analysis. Knowledge-Based Systems, 69, 24–33. doi: https://doi.org/10.1016/j.knosys.2014.05.008.

31.

Schmid

(1994). Probabilistic part-of-speech tagging using decision trees. Proceedings of International Conference on New Methods in Language Processing, Manchester, UK.

32.

Schmid

. (2007). TreeTagger software - a part-of-speech tagger for many languages. http://www.cis.uni-muenchen.de/∼schmid/tools/TreeTagger/.

33.

Silge

Robinson

(2017). Text mining with R: A tidy approach. Sebastopol, CA: O’Reilly Media.

34.

Silge

Robinson

(2018). tidytext: Text mining for word processing and sentiment analysis using ‘dplyr’, ‘ggplot2’ and other tidy tools (R package version 0.2.0).

35.

Simmons

R. G.

Burgeson

Carlton-Ford

Blyth

D. A.

(1987). The impact of cumulative change in early adolescence. Child Development, 58, 1220–1234.

36.

Somerville

L. H.

(2013). The teenage brain: Sensitivity to social evaluation. Current Directions in Psychological Science, 22, 129–135. doi:10.1177/0963721413476512

37.

Steinberg

Morris

A. S.

(2001). Adolescent development. Annual Review of Psychology, 52, 83–110.

38.

Tousignant

Sirois

Achim

A. M.

Massicotte

Jackson

P. L.

(2017). A comprehensive assessment of social cognition from adolescence to adulthood. Cognitive Development, 43, 214–223. doi: https://doi.org/10.1016/j.cogdev.2017.05.001

39.

Weinstein

S. M.

Mermelstein

R. J.

Hankin

B. L.

Hedeker

Flay

B. R.

(2007). Longitudinal patterns of daily affect and global mood during adolescence. Journal of Research in Adolescence, 17, 587–600. doi:10.1111/j.1532-7795.2007.00536.x

40.

Wickham

(2018). ggplot2: Create elegant data visualizations using the grammar of graphics (R package version 3.1.0).

41.

Wickham

(2018). dplyr: A grammar of data manipulation (R package version 0.7.8).

42.

Wormwood

J. B.

Devlin

Lin

Y.-R.

Barrett

L. F.

Quigley

K. S.

(2018). When words hurt: Affective word use in daily news coverage impacts mental health. Frontiers in Psychology, 9, 1–10. doi:10.3389/fpsyg.2018.01333

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.25 MB

	Positive		Negative
Grade	M	SD	M	SD
4	.191	.164	.118	.129
5	.194	.157	.119	.129
6	.208	.160	.122	.118
7	.199	.146	.132	.124
8	.195	.138	.141	.122
9	.205	.140	.152	.117
10	.196	.126	.151	.117
11	.204	.132	.155	.115
12	.206	.132	.146	.116
Total	.201	.153	.136	.127