Make No Mistake? Exploring Cognitive and Perceptual Effects of Grammatical Errors in News Articles

Abstract

Using four between-subjects experiments (N₁ = 106, N₂ = 166, N₃ = 159, and N₄ = 164), this project tests the ways audiences process grammatical errors in news articles. In all, results suggest that readers perceive stories with grammatical errors to be lower in quality, credibility, and informativeness, but the number of errors needed is relatively large. Analysis shows amplified effects for people who report concern about grammar, and, to a lesser degree, people with knowledge of grammar rules. Given these results, the findings suggest a nonlinear, nonuniversal effect of grammatical errors on readers of news articles.

Keywords

journalism studies media effects quantitative methods experiments

Do grammatical errors bother modern news audiences? For years, journalists have assumed that they do. Print publications in the United States have hired copy editors to fix mistakes since at least in the 1890s (Solomon, 1985, 1989), and British publications hired “sub-editors” with the same responsibilities as early as the 1840s (Keith, 2009). Journalism professionals have discussed the connection between journalistic grammar and quality for decades (see Walsh, 2008), but this has not been extensively studied in academic research. Exceptions include Appelman and Bolls (2011), which found that reader perceptions are negatively affected by grammatical errors, and Vultee (2015), which found that readers distinguish between edited and unedited news articles. This project seeks to replicate and better understand these findings.

This is a timely subject within journalism education, where there are concerns about time spent on traditional editing skills, such as prescriptive grammar rules, as compared with online skills. It could be that grammatical accuracy has become obsolete in the current media landscape. On the contrary, grammatical accuracy might help readers distinguish traditional, professional journalism outlets from other user-generated content. Perhaps grammatical accuracy serves as a proxy for other perceptions; for example, if readers sense that an article has been edited for grammar, then maybe they also believe it has been carefully reported and fact-checked. Academic research has yet to thoroughly address such concerns.

Through a series of between-subjects experiments, this study looks at the ways modern audiences process grammatical errors. By doing so, it tests common assumptions that are at the root of media education and practice, strives to replicate the limited research on this topic, and links media practice to theories of information processing and news credibility.

Literature Review

Grammatical Errors in the News Media

“Grammar” refers to formal standards of usage, spelling, and punctuation. This study focuses on those English standards enforced by journalists in their canonic literature (e.g., Brooks, Pinson, & Wilson, 2006; O’Conner, 2003; Truss, 2006; Webster’s New World College Dictionary, 2005). In other words, “grammar” is used here as an overall label for a number of copy-editing issues, including those—such as spelling—that are not technically “grammatical.”

“Grammatical errors” refer to violations of these journalistic writing standards. Here, we use the term to refer to what Johnson and VanBrackle (2012) called “surface writing errors,” or “non-standard writing (marked) features that do not interfere with communication but would be noted by most raters as non-conformity to the conventions of standard English” (p. 9). Johnson and VanBrackle included improper spelling, verb choice, punctuation, and syntax as examples.

Determining what constitutes a grammatical error has proven challenging to researchers. There is an inherent difficulty in establishing a universal, narrowly defined list of grammatical errors, largely because language changes and evolves, so what once might have been considered an error is now considered correct. In an article chronicling formal errors in college writing assignments, Connors and Lunsford (1988) noted this struggle: “Looking into the history of this kind of research showed us clearly how teachers’ ideas about error definition and classification have always been absolute products of their times and cultures” (p. 399). To address this issue, their list of common errors came from consultation with current experts; the researchers noted what they thought to be grammatical errors, established a top 20 list, and then had others (50 college teaching assistants, instructors, and professors) search the writing for those error patterns.

Similarly, the grammatical errors addressed in the following studies came from consultations with others. In the first study, the list was based on the mistakes students needed to correct as part of a journalism school’s grammar entrance exam: incorrect spelling, incorrect subject–verb agreement, confused homonyms, and double negatives (Appelman, 2009). In the other three studies, the list was based on the mistakes ranked as most problematic by a group of journalists (n = 39): incorrect subject–verb agreement (e.g., “they has” instead of “they have”), incorrect spelling of proper nouns, confused homophones (e.g., “whether” instead of “weather”), confusion of possessive and plural cases of nouns, incorrect spelling of common nouns, and incorrect pronoun–antecedent agreement.

Grammatical Errors and Reader Judgments

Significant time and energy is spent on teaching grammatical accuracy in journalism classrooms. Journalism departments have stand-alone editing classes that focus on these rules, and introductory reporting classes employ grammar quizzes as part of their curricula. Journalism schools teach copyediting to students; copy editors formed their own professional society, ACES: The Society for Editing (formerly the American Copy Editors Society); and professional journalism publications analyze each other’s errors (e.g., the Columbia Journalism Review’s Language Corner).

Despite the focus in professional training, insufficient research demonstrates that audiences view grammatically “correct” content as superior, or otherwise process and respond to such copy differently from content that contains consistent or notable grammatical errors. The attention paid by journalists may reflect professional norms and practices that impress other journalists and instructors, but this does not ensure that the average reader notices or cares.

Research in other fields, however, suggests that readers might harshly judge news media with grammatical errors. Studies in psychology, for example, have examined perceptual effects of spelling errors. Kreiner, Schnakenberg, Green, Costello, and McClin (2002), for example, found that readers judge writers for spelling errors, particularly in terms of perceived writing ability. Figueredo and Varnhagen (2005) found this to be true even when readers are told authors used spell-check. In a linguistics study, Queen and Boland (2015) found that readers negatively judged writers of error-filled emails. In this study, the emails were presented as responses to a housemate ad, and they included typographic and grammatical errors.

Several business studies suggest negative effects of grammatical errors on employment decisions. Schramm and Neil Dortch (1991), for example, found negative college recruiter perceptions of grammatically incorrect resumes. Charney and Rayman (1989) and Charney, Rayman, and Ferreira-Buckley (1992) conducted similar experimental studies where recruiters, again, negatively judged resumes with grammatical errors. Martin-Lacroux and Lacroux (2016) conducted a more recent study on the relative effects of spelling errors on application forms as compared with content. They found spelling errors had the same negative effect on hiring as lack of professional experience and that this effect was moderated by recruiters’ spelling skills. Related studies show these negative perceptions can lead to what is known as “linguistic profiling” or discrimination (Baugh, 2016; Hughes & Mamiseishvili, 2014).

Despite the research in other areas and the professional norms that emphasize editing in journalism, academic research in the news context is limited. Exceptions include Appelman and Bolls (2011), which found that reader perceptions are negatively affected by grammatical errors, and Vultee (2015), which found that readers distinguish between edited and unedited news articles. This study seeks to corroborate these effects and determine their boundaries. Both sets of experiments used college-student samples, so one reason for replication is to determine whether effects are generalizable. Perhaps the participants in those studies noticed grammatical errors and cared about them more than the average person. Undergraduate journalism or communication students—like those surveyed in Appelman and Bolls—would arguably be more aware of grammar rules than the average reader, so perhaps that study overestimated effects. Vultee considered students majoring in communication as well as other fields, but it still raises the question of generalizability beyond college students.

Another reason for replication is to determine whether such effects are linear. In Appelman and Bolls (2011), each participant read three articles with no errors and three articles with 10 errors. Similarly, in Vultee (2015), each participant read four edited articles and four unedited articles. It is unclear in both instances whether an article with fewer mistakes or one that was slightly edited would have elicited the observed effects.

A final reason for replication is to address and extend the core outcome variables. Appelman and Bolls (2011) found negative effects on perceived credibility, and Vultee (2015) found negative effects on perceived quality, so both are included here for replication. In Study 1, perceived expertise was measured as a proxy for credibility (Chung, Nam, & Stefanone, 2012); the others use a validated scale of message credibility, “an individual’s judgment of the veracity of the content of communication” (Appelman & Sundar, 2016, p. 63). These studies refer to quality as the “degree or level of overall excellence of a news story” (Sundar, 1999, p. 381).

The other outcome variables measured in these previous studies are reading time, recall (Appelman & Bolls, 2011), and whether participants thought the content was worth paying for (Vultee, 2015). Here, all are addressed under the umbrella concept of informativeness, which refers to a perception of the article’s ability to communicate information or knowledge (Mutz & Reeves, 2005).

Based on industry-based suggestions and available academic research, this study predicts negative effects of grammatical errors on all of the aforementioned reader perceptions. Here, the prediction is that grammatical errors serve as a proxy for professionalism or confidence in the truth of published information. This leads to the prediction addressed in the following studies:

H1: There will be an effect of grammatical errors on reader perceptions, such that articles with errors will be perceived more negatively than articles without in terms of (H1a) perceived credibility, (H1b) perceived quality, and (H1c) perceived informativeness.

Methodological Approach

To evaluate this hypothesis, a series of studies were conducted to replicate and extend previous research addressing errors. The first two studies test the main effects of grammatical errors on audience perceptions. The second two studies amplify the manipulation, test main effects, and test potential moderating effects of grammar knowledge and grammar concern. In each set, one study uses a student sample and one an online sample. Individual study results are discussed in their respective sections, and comprehensive results and implications are discussed at the end. All studies were approved by the applicable Institutional Review Board.

Studies 1 and 2: Baseline Effects of Grammatical Errors on Perceptions

The first two studies test the main effects of grammatical errors on audience perceptions, once with a student sample and once with a sample from Amazon Mechanical Turk (MTurk).

Method

Study designs

Two between-subjects experiments were conducted online, where participants read a news article with either zero or 10 grammatical errors. The news article for the first study discussed polio eradication (modified from Hensley, 2012). To account for potential idiosyncrasies in content, multiple articles were selected for the second study, all of which discussed environmental and science issues. Four news articles were initially modified (Alvarez, 2014; Chang, 2014; Gorman, 2014; Rabin, 2014), though only results from Alvarez (2014) and Chang (2014) were used in the main study. Both were manipulated to be 12 paragraphs and 440 to 460 words. These studies were conducted in 2012 and 2014, respectively.

Participants

For Study 1, participants (N = 106) were students from undergraduate communications courses at a large public university in the United States. Ages ranged from 17 to 42 (M = 21.39, SD = 4.00), and there were more women (n = 70) than men (n = 36). Most were native English speakers (n = 94).¹ For Study 2, participants (N = 166) from the United States were recruited from MTurk, an online forum, or “crowdsourcing web service,” that connects researchers with participants (Paolacci, Chandler, & Ipeirotis, 2010, p. 411; for more, see “Amazon Mechanical Turk FAQ,” n.d.; Paolacci & Chandler, 2014). Ages ranged from 19 to 68 (M = 35.20, SD = 11.56), and there were more men (n = 98) than women (n = 68). Most participants were White (n = 139), though others identified as Hispanic (n = 13), Asian (n = 10), Black (n = 7), Native American (n = 1), and Other (n = 2). Geographic diversity was more pronounced than racial diversity, as participants were located throughout the United States (South, n = 71; Midwest, n = 34; Northeast, n = 33; and West, n = 28).²

Manipulation

For both studies, the researchers looked for baseline effects by comparing two error conditions: Participants read a news article with or without errors. In Study 1, the error condition had up to 10 errors; in Study 2, the error condition had 10 errors.³ Participants were randomly assigned to read an article in one of the grammar conditions for Study 1 (no errors: n = 37; errors: n = 85) and for Study 2 (no errors: n = 84; errors: n = 82).

The errors for Study 1 included incorrect spelling, incorrect subject–verb agreement, confused homonyms, and double negatives. These are the elements that defy proper modern English standards and “the types of mistakes copy editors are trained to eliminate from print publications . . . limited to those recognizable by a mass audience” (Appelman, 2009, pp. 32-33).

The errors used in Study 2 (and again in the next set of studies) were determined based on a pilot test. Journalists (n = 39) were recruited and asked to rank errors. They were given 25 grammatical errors with examples and asked to select the five most problematic. They chose six grammatical errors more than the others, so articles in the grammatical error condition were each given 10 errors from the following list, with each present at least once: incorrect subject–verb agreement, incorrect spelling of proper nouns, confused homophones, confusion of possessive and plural cases of nouns, incorrect spelling of common nouns, and incorrect pronoun–antecedent agreement.

Manipulation checks for Study 1 and Study 2 both were successful, which confirmed that participants noticed the grammatical errors.⁴

Measures

Quality

In Study 1, participants were asked to indicate how well four adjectives described the article, from 1 = describes very poorly to 7 = describes very well: coherent, clear, concise, well-written (Sundar, 1999). The quality variable was constructed as an average of the scores on these four items (Cronbach’s α = .897, M = 4.50, SD = 1.38). In Study 2, the same adjectives were used, but the prompt and anchors were rewritten to be consistent with the study’s other measures: “The content of the article was . . . ,” from 1 = strongly disagree to 7 = strongly agree. Quality was constructed as an average (Cronbach’s α = .83, M = 5.54, SD = 0.94).

Credibility

In Study 1, participants were asked to indicate how well three adjectives described the article, from 1 = describes very poorly to 7 = describes very well: professional, in-depth, written by professional journalists (Chung et al., 2012). The expertise variable (used here as a proxy for credibility) was constructed as the average of the scores on these three items (Cronbach’s α = .72, M = 3.75, SD = 1.22). In Study 2, participants were given the prompt “The content of the article was . . .” and asked to indicate how well each of three adjectives completed the sentence, from 1 = strongly disagree to 7 = strongly agree: accurate, believable, authentic (Appelman & Sundar, 2016). The message credibility variable was constructed as an average of the scores on these three items (Cronbach’s α = .82, M = 5.48, SD = 0.91). This was different than the measure used in Study 1; it more directly addresses message credibility, and it was developed and published after Study 1 was conducted.

Informativeness

Participants were asked to indicate agreement with six statements from 1 = strongly disagree to 7 = strongly agree: “In general, I found the article to be informative”; “I learned new things about public issues from this article”; “This article gave me food for thought”; “If I needed information about public issues, I would visit this website”; “I felt like I got to know about the public issue by reading this article”; and “As a result of reading this article, I am more comfortable talking to friends about this issue” (Mutz & Reeves, 2005). The informativeness variable was constructed as the average of the scores on these six items for Study 1 (Cronbach’s α = .84, M = 4.52, SD = 0.98) and for Study 2 (Cronbach’s α = .89, M = 5.33, SD = 0.99).

Results

H1 predicted a main effect of grammatical errors on perceived quality, perceived credibility, and perceived informativeness. Here, t tests were conducted to test for these relationships.

In Study 1, analysis showed a significant effect of grammatical errors on perceived quality, such that articles with errors were perceived to be of lower quality than articles with no errors, t(104) = 2.34, p = .02, d = .48. Similarly, analysis showed a pattern linking the presence of errors to perceptions of expertise, but this was shy of traditional significance levels, such that articles with errors were perceived to be of lower expertise than articles with no errors, t(104) = 1.91, p = .06, d = .40. As noted above, expertise here was used as a proxy for perceived credibility. Similar but nonsignificant patterns were seen for perceived informativeness.

In Study 2, analysis showed no significant main effect of grammatical errors on perceived quality, perceived credibility, or perceived informativeness. In addition, no discernable patterns arose.

Regarding H1, then, the student sample finds support, but the MTurk sample does not. Table 1 shows t-test results for Study 1 and Study 2.

Table 1.

Mean Differences in Article Perceptions as Affected by Grammatical Errors.

Perception	No errors	Errors	t	p	d
Study 1: Student sample
Quality	4.95 (1.42)	4.29 (1.32)	2.34*	.02	.48
Credibility	4.08 (1.16)	3.60 (1.23)	1.91	.06	.40
Informativeness	4.71 (0.85)	4.44 (1.03)	1.30	.20	.29
Study 2: MTurk sample
Quality	5.57 (0.87)	5.51 (1.01)	0.45	.66	.06
Credibility	5.41 (0.85)	5.54 (0.97)	0.90	.37	.14
Informativeness	5.31 (1.02)	5.35 (0.97)	0.27	.79	.04

Note. N₁ = 106, df = 104; N₂ = 166, df = 164. Perceptions measured 1 to 7 and presented as M (SD).p = exact p value of the t statistic; d = Cohen’s d, a measure of effect size for comparisons of means. MTurk = Amazon Mechanical Turk.

p < .05.

These results are suggestive but not conclusive. Why did the student sample suggest findings not seen in the MTurk sample? It could be that errors in the second study were not prominent enough to elicit the effect. In addition, perhaps there is a nonlinear effect. In Appelman and Bolls (2011), for example, where grammar-based differences were significant, the average article length was 135 words, compared with Study 2 here, where the average was 445; it could be that additional errors need to be added to compensate for the additional article length. By amplifying the grammar manipulation, the next set of studies addresses these questions.

Studies 3 and 4: Refinement With Amplified Grammar Manipulation and Moderators

The second two studies amplify the manipulation from the previous studies to again test main effects, once with a student sample and once with a sample from MTurk. In addition, Study 4 seeks to provide an explanation for the findings by testing the moderating effects of grammar knowledge and grammar concern.

If grammatical errors affect perceptions, then why is that the case? For relatively minor errors, like those included in these studies, the most likely explanation is that such mistakes provide cues about the professionalism and quality of the reporting and outlet. The types of errors included in these studies do not necessarily affect comprehension, so we would not expect their effect to be based on inhibiting the ability of readers to follow a story.

The Heuristic-Systematic Processing Model (HSM; Chaiken, 1980; Chaiken, Liberman, & Eagly, 1989) refers to this as heuristic information processing, or surface-level processing based on “activation and application of judgmental rules or ‘heuristics’” (Chen & Chaiken, 1999, p. 74). Research suggests that people are engaged in heuristic processing while consuming news media. In fact, Metzger, Flanagin, and Medders (2010) specifically noted grammar as an example of the expectancy-violation heuristic that media consumers use when judging credibility: “Bad grammar and typographical errors were mentioned across all of the focus group sites as a quick and easy way to determine a site’s credibility” (p. 431). This supports the argument that grammatical errors could serve as cues.

To determine whether this is the case, we test whether all individuals respond equally to errors, or whether effects are stronger for those high in grammar concern (people who indicate they value “proper” grammar) or those high in grammar knowledge (people most familiar with grammar “rules”). This is consistent with the argument of Bellur and Sundar (2014), who proposed that measuring endorsement of heuristics and treating that as a moderating factor can establish whether assumed heuristics are at work.

In this way, researchers argue they can find support for a cue- or distraction-based explanation based on whether individual differences moderate the main effects of grammatical errors. This leads to the second prediction tested here.

H2: The effect of errors on reader perceptions will be moderated by (H2a) grammar knowledge and (H2b) grammar concern.

Method

Study designs

Two additional between-subjects experiments were conducted online. Participants here read a news article with zero, 10, or 30 grammatical errors. The stimulus article was the environmental article modified from Alvarez (2014) because it had the most successful manipulation check in Study 2. The purpose was to amplify the manipulation, test main effects, and also test potential moderating effects of grammar knowledge and grammar concern. These rounds of data collection for Study 3 and Study 4 were both conducted in 2015.

Participants

For Study 3, participants (N = 159) from the United States recruited from MTurk ranged in age from 19 to 67 (M = 33.48, SD = 11.37); there were more men (n = 95) than women (n = 64). Most participants were White (n = 134), though others identified as Asian (n = 12), Black (n = 11), Hispanic (n = 5), and Native American (n = 3). Geographic diversity was more pronounced than racial diversity, as participants were located throughout the United States (South, n = 54; Midwest, n = 40; Northeast, n = 35; and West, n = 30). For Study 4, participants (N = 164) were students from a medium-sized public university in the United States from undergraduate courses in multiple departments. (This was a different university than that of the previous student sample.) Ages ranged from 18 to 48 (M = 22.20, SD = 4.15), and there were more women (n = 89) than men (n = 72) (three did not respond to the question). Most grew up in the United States (n = 149).

Manipulation

For both studies, participants read a news article with either zero errors, 10 errors (some errors), or 30 errors (many errors), using the types of grammatical errors established in Study 2. Each participant was randomly assigned one of the grammar conditions for Study 3 (none: n = 50; some: n = 49; and many: n = 60) and for Study 4 (none: n = 59; some: n = 52; and many: n = 53). Manipulation checks for Study 3 and Study 4 both were successful, which confirmed that participants noticed the grammatical errors.

Measures

Both studies assessed perceptions, but only the student sample also assessed grammar knowledge and grammar concern. There is an inherent difficulty in asking knowledge questions in MTurk because of shared discussion groups among users, discussed in the “Limitations and Future Research” section.

Quality

Both studies used the quality measure described in Study 2 (Study 3: Cronbach’s α = .92, M = 4.39, SD = 1.56; Study 4: Cronbach’s α = .89, M = 3.98, SD = 1.46).

Credibility

Both studies used the credibility measure described in Study 2 (Study 3: Cronbach’s α = .85, M = 4.80, SD = 1.24; Study 4: Cronbach’s α = .83, M = 4.08, SD = 1.28).

Informativeness

Both studies used the informativeness measure described in Study 1 (Study 3: Cronbach’s α = .93, M = 4.49, SD = 1.32; Study 4: Cronbach’s α = .91,M = 3.73, SD = 1.41).

Grammar knowledge

This variable represents familiarity with the “rules” of grammar.⁵ In Study 4, participants were given six fill-in-the-blank statements written by the researcher to include specific errors used in the stimuli (e.g., “They is/are/did/does finished with their homework”; “He asked me whether/weather I would like to go to a movie this weekend, but I said no because I heard the whether/weather would be bad”; “The magician made the rabbit go away, or disapear/dissapear/disappear/dissappear”). The grammar knowledge variable was constructed as the number of correct responses, so the highest possible score was 6 (M = 5.13, SD = 1.09).

Grammar concern

This variable represents whether participants value “correct” copy. In Study 4, participants were asked to indicate agreement with four statements (“Grammar errors bother me,” “I get annoyed when I see grammar errors,” “Proper grammar is important,” “I notice grammar errors”) from 1 = strongly disagree to 7 = strongly agree. The grammar concern variable was constructed as an average of the scores on these four items (Cronbach’s α = .87, M = 5.44, SD = 1.50).

Results

H1 predicted a main effect of grammatical errors on perceived quality, perceived credibility, and perceived informativeness. Here, omnibus one-way ANOVA tests were conducted, followed by post hoc tests using Tukey’s honestly significant difference (HSD) test for extension and clarification.

In Study 3, the omnibus ANOVA tests showed significant main effects of grammatical errors on perceived quality, F(2, 158) = 26.02, p < .001, $η_{p}^{2}$ = .25; perceived credibility, F(2, 158) = 10.48, p < .001, $η_{p}^{2}$ = .12; and perceived informativeness, F(2, 158) = 11.69, p < .001, $η_{p}^{2}$ = .13. This suggests support for H1. Post hoc tests for perceived quality show significant differences between the no errors, some errors, and many errors conditions. Post hoc tests for the other two perceptions show significant differences only between the many errors condition and the some errors condition, and between the many errors condition and the no errors condition.

In Study 4, the omnibus ANOVA tests again showed significant main effects of grammatical errors on perceived quality, F(2, 161) = 18.48, p < .001, $η_{p}^{2}$ = .19; perceived credibility, F(2, 161) = 13.13, p < .001, $η_{p}^{2}$ = .14; and perceived informativeness, F(2, 160) = 9.30, p < .001, $η_{p}^{2}$ = .10. This suggests support for H1. Post hoc tests for all three perceptions show significant differences only between the many errors condition and the some errors condition, and between the many errors condition and the no errors condition.

Regarding H1, then, both studies find support. Omnibus tests show main effects of grammatical errors on perceptions. The post hoc tests, however, show this is only true when the errors are numerous. The general pattern revealed that the many errors condition differed from the others, with consistently worse perceptions from those in that condition. Table 2 shows ANOVA results for Study 3 and Study 4.

Table 2.

ANOVA in Article Perceptions as Affected by Grammatical Errors.

Perception	No errors	Some errors	Many errors	F	$η_{p}^{2}$
Study 3: MTurk sample
Quality	5.30 (1.07)_a	4.60 (1.27)_b	3.45 (1.63)_c	26.02***	.25
Credibility	5.24 (1.11)_a	5.00 (0.96)_a	4.27 (1.36)_b	10.48***	.12
Informativeness	5.07 (0.98)_a	4.58 (1.12)_a	3.93 (1.49)_b	11.69***	.13
Study 4: Student sample
Quality	4.63 (0.99)_a	4.11 (1.38)_a	3.12 (1.57)_b	18.48***	.19
Credibility	4.51 (0.92)_a	4.30 (1.24)_a	3.40 (1.41)_b	13.13***	.14
Informativeness	4.08 (1.20)_a	3.99 (1.55)_a	3.08 (1.41)_b	9.30***	.10

Note. N₃ = 159, N₄ = 164. Perceptions measured 1 to 7 and presented as M (SD). Means with differing subscripts within rows are significantly different at p < .05 based on post hoc analysis using Tukey’s HSD. $η_{p}^{2}$ = measure of effect size that reflects the proportion of total variance in the dependent variable associated with the independent variable, without the effects of other variables. MTurk = Amazon Mechanical Turk; HSD = honestly significant difference.

***

p < .001.

H2 predicted a moderating effect of grammar knowledge and grammar concern in the relationship between grammatical errors and article perceptions. Here, regression analyses were conducted. In these analyses, two dummy-coded variables were created to compare the some errors and the many errors conditions to the excluded condition of no errors. Our focus is on significant interactions between these terms and either grammar knowledge or grammar concern (tested separately), which would indicate the results noted above are conditional upon those moderating variables. These analyses were only conducted in Study 4.

The regression analysis showed that grammar knowledge significantly moderated the effect of grammar condition on quality (see Table 3). This interaction was significant when comparing either error condition with the control condition. Analysis also showed that grammar knowledge significantly moderated the effects of grammar condition on credibility, but only for the comparison of the many errors condition to the control condition. Informativeness showed a similar pattern, but the interaction was significant only when comparing the some errors condition to the control condition.

Table 3.

Moderation Tests From Regression Models Predicting Moderating Effects of Grammar Knowledge and Grammar Concern in the Relationship Between Grammatical Errors and Article Perceptions.

	Quality		Credibility		Informativeness
	B (SE)	β	B (SE)	β	B (SE)	β
Knowledge
Some errors	−0.50 (.22)	−0.85*	−0.31 (.20)	−.60	−0.73 (.29)	−1.30*
Many errors	−0.69 (.22)	−1.20**	−0.43 (.21)	−.85*	−0.39 (.23)	−0.70
Constant	4.67 (.70)		4.45 (.66)		3.94 (.74)
R² change		.05**		.03		.04*
Concern
Some errors	−0.55 (.16)	−1.01**	−0.22 (.16)	−.47	−0.21 (.18)	−0.40
Many errors	−0.86 (.18)	−1.74***	−0.57 (.17)	−1.31**	−0.59 (.20)	−1.23**
Constant	3.60 (.65)		3.97 (.63)		3.33 (.71)
R² change		.10**		.06**		.05*

Note. N₄ = 164.

Coefficients reflect the interaction of grammar knowledge or concern (as indicated) and the dummy-coded condition variable, obtained from a model also containing the main predictors of concern or knowledge along with the dummy codes.

Reference group for dummy variables is the no error condition. B = unstandardized coefficient. β = standardized coefficient.

p < .05. **p < .01. ***p < .001.

There were similar interactions between grammar concern and condition on the perceptions, which makes sense as grammar knowledge was positively correlated with grammar concern, r(161) = .26, p = .001. Analysis showed that grammar concern significantly moderated the effect of grammar condition on quality. This interaction was significant when comparing either error condition with the control condition. Analysis also showed that grammar concern significantly moderated the effects of grammar condition on credibility and informativeness, but only with many errors. In both cases, the interaction was not significant when comparing some errors with no errors, but it was significant when considering many errors. In all cases, the results show that grammar concern magnifies the negative effect of errors on perceptions—Those who score lowest on grammar concern actually favor stories with more errors, but as concern increases, evaluations become increasingly negative.

Thus, regarding H2, the student sample suggests that, in most cases, those familiar with and concerned with grammar “rules” are most bothered by their violation. Thus, this study finds support for H2a and H2b.

Summary and Concluding Discussion

Through a series of between-subjects experiments, this project tests the ways readers process grammatical errors in news articles. This project found effects of grammar on perceptions, provided the manipulation was strong enough. In other words, the data suggest that readers perceive stories with grammatical errors to be lower in quality, credibility, and informativeness, but the number of errors needed is relatively large.

In addition, grammatical errors did not affect audiences equally. The effect was strongest for people high in grammar concern, and to a lesser degree grammar knowledge. In fact, people with the lowest levels of grammar concern perceived stories with more errors as equal to or even more positive than the stories without errors, which suggests errors might be considered a marker of “authenticity” for people who reject grammar conventions. In the framework of the HSM, the amplified effect for people who say they care about grammar could suggest heuristic processing. Because motivation was not directly manipulated, the researchers cannot definitively rule out the possibility that grammar judgments are systematic in nature; however, the fact that people had to say they cared about grammar for it to matter suggests errors act as a heuristic judgment and not a universal barrier to understanding. This study’s findings suggest that grammatical errors are a heuristic cue, but they are a heuristic cue that many people do not employ. This validates the idea that some heuristics are not universal and that future HSM studies should measure the likelihood of a person applying a particular heuristic. Future tests that manipulate involvement, motivation, or ability to process could further test this possibility.

The conditional heuristic effects explored here serve as a plausible explanation for the effects of grammatical errors, but it could be that processing depends on the type or nature of the grammatical error. These studies grouped several types of errors in each experimental condition, so it is unclear whether some errors had more pronounced effects than others. In some cases, grammar errors might be a barrier to systematic processing rather than merely a heuristic cue for those not engaged in such processing. Research shows complex language inhibits comprehension (e.g., Reilly & Richey, 2011; van Weert et al., 2011; Wolf et al., 2007). Some error-laden text may have similar effects. Future studies could test the possibility that distracting grammatical errors are heuristically processed, conditional upon grammar concern, while confusing grammatical errors are systematically processed. Clarity could be measured by Flesch–Kincaid readability tests or by other techniques used in the studies of language complexity (e.g., Miestamo, Sinemmäki, & Karlsson, 2008; Sampson, Gil, & Trudgill, 2009; Vulanović, 2007). Future tests that parse out these results could direct practitioners’ and educators’ attention toward specific types of errors.

Practically, this study suggests the occasional grammatical error is unlikely to undermine perceptions for most readers. Journalists should still be aware of grammatical accuracy, but it could be that emphasis on factual accuracy is more important in the current online and social media news environment. Future studies could test the relative importance of factual accuracy and grammatical accuracy to news audiences, as a way of better parsing out these effects.

Limitations and Future Research

Using MTurk as a sampling method increased the diversity and generalizability of the samples but does not ensure representativeness. Research shows that demographically, MTurk samples better represent the population than do college samples or other Internet samples (Paolacci et al., 2010, p. 414), but this study suggests other potential concerns.

By nature of the sampling technique, MTurkers are tech-savvy and frequent Internet users; because they have done similar studies before, they also are likely more familiar with the format. The implications of this are seen through a quick post hoc investigation into their test-taking behavior. A search revealed that MTurk participants have online forums (e.g., Turkopticon) for sharing information about studies. In fact, three Reddit postings were found regarding this project’s Study 2. The possibility of interparticipant bias must be considered when conducting experiments with this sample. This was the reason grammar knowledge was not asked in the MTurk samples; there was concern the MTurk workers would share answers with each other, out of concern that a less-than-perfect score would mean not getting paid for participation. However, the student participants in Study 4 scored quite high on grammar knowledge (M = 5.13 out of a possible 6), so it is possible that all online studies face these concerns, regardless of sampling technique.

It also could be that MTurk workers are otherwise inattentive or efficiency focused. However, a quick analysis of reading speed suggests this is not the case. In Study 2, average article length was 445 words and average reading time was 132 seconds, so participants were reading about 200 words per minute: The average American adult can read about 250 words per minute (Thomas, n.d.), so skimming or inattentiveness are unlikely explanations for the results.

Participant age was relatively low across the four studies, regardless of the sampling technique (21, 35, 33, and 22, respectively). Because of that, moderating effects of age were unable to be determined with certainty. Preliminary analysis suggests a moderating effect of age in Study 1 regarding two of the measures, with older participants rating the articles with errors as being less expert and of lower quality: b = −1.12, t(102) = −1.98, p = .05 for expertise, and b = –.1.62, t(102) = −2.11, p = .004 for quality. However, this pattern was not replicated across the other three studies. In fact, Study 4 results show age as negatively correlated with both grammar knowledge and grammar concern, though those relationships were not significant.

Another concern is that the number of errors required to show a consistent effect is fairly large: 30 errors in a 450-word article. This does not reflect a typical news story; however, the results are still telling in that they suggest an eventual psychological effect for the general public. In addition, negative perceptions were triggered by fewer errors in some cases, especially among those concerned with grammar, and it could be that that those are the people most likely to read news articles anyway. Future studies that identify characteristics and media habits of people high in grammar concern could help better explain these findings.

In addition, these studies were set up as between-subjects designs, where participants each read one article. It could be that exposure to multiple articles would have resulted in different findings; perhaps readers are less bothered the first time they see grammatical errors in a source’s article but become more aware and more frustrated upon the second or third error-filled article. In other words, perhaps the effect depends on exposure frequency. The design was used here as a way to control for the effects of multiple articles (e.g., the differences in content, complexity, or word choice) and as a way to mimic the way news articles are seen in isolation from the original source (e.g., when shared on social media by a friend). However, future studies that control for those factors could determine the effects of repeated exposure.

Conclusion

Based on this project, grammar still matters in the digital age. Although effects were amplified for participants who say they care about grammar, baseline effects still existed regardless of individual differences. Journalists cannot completely ignore grammar conventions and expect no reaction from the audience; however, for many readers, the occasional error is unlikely to undermine audience trust or perceptions of quality.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: For Study 1, the first author was supported by a summer research grant awarded by the Donald P. Bellisario College of Communications at the Pennsylvania State University during her doctoral program. For Study 2, participant compensation was funded by ACES: The Society for Editing (formerly the American Copy Editors Society).

Supplemental Material

The online appendices are available online at

Notes

Author Biographies

Alyssa Appelman (PhD, The Pennsylvania State University) is an assistant professor in the Department of Communication in the College of Informatics at Northern Kentucky University. She teaches courses in journalism and mass communication, and her research focuses on journalism and media effects. She examines the cognitive and perceptual effects of news style, and she is particularly interested in the ways news conventions affect knowledge-gain and credibility perceptions. She also analyzes reader engagement in digital and print media.

Mike Schmierbach (PhD, University of Wisconsin–Madison) is an associate professor of media studies in the Donald P. Bellisario College of Communications at the Pennsylvania State University. His research addresses how media use and content affect perceptions of media, with a particular focus on video games and third-person research. He also specializes in research methodology and is coauthor of the textbook Applied Communication Research Methods.

References

Alvarez

(2014, June 5). Researchers race to save coral in Miami. The New York Times. Retrieved from https://www.nytimes.com/2014/06/06/us/researchers-race-to-save-coral-before-dredging-starts-in-miami.html

Amazon Mechanical Turk FAQ. (n.d.). Retrieved from https://www.mturk.com/mturk/help?helpPage=overview

Appelman

(2009). Grammar and cognitive processing of news articles: Exploring dual processing theories (Master’s thesis). University of Missouri, Columbia.

Appelman

Bolls

(2011). Article recall, credibility lower with grammar errors. Newspaper Research Journal, 32, 50-62.

Appelman

Sundar

(2016). Measuring message credibility: Scale construction and validation with news articles. Journalism & Mass Communication Quarterly, 93, 59-79.

Baugh

(2016). Linguistic profiling and discrimination. In Garcia

Flores

Spotti

(Eds.), The Oxford handbook of language and society (pp. 349-368). New York, NY: Oxford University Press.

Bellur

Sundar

S. S.

(2014). How can we tell when a heuristic has been used? Design and analysis strategies for capturing the operation of heuristics. Communication Methods and Measures, 8, 116-137.

Brooks

B. S.

Pinson

J. L.

Wilson

J. G.

(2006). Working with words: A handbook for media writers and editors (6th ed.). Boston, MA: Bedford/St. Martin’s.

Chaiken

(1980). Heuristic versus systematic information processing and the use of source versus message cues in persuasion. Journal of Personality and Social Psychology, 39, 752-766.

10.

Chaiken

Liberman

Eagly

A. H.

(1989). Heuristic and systematic information processing within and beyond the persuasion context. In Uleman

J. S.

Bargh

J. A.

(Eds.), Unintended thought (pp. 212-252). New York, NY: Guilford Press.

11.

Chang

(2014, January 8). NASA extends life of the international space station. The New York Times. Retrieved from https://www.nytimes.com/2014/01/09/science/space/nasa-extends-the-life-of-the-international-space-station.html

12.

Charney

D. H.

Rayman

(1989). The role of writing quality in effective student résumés. Journal of Business and Technical Communication, 3, 36-53.

13.

Charney

D. H.

Rayman

Ferreira-Buckley

(1992). How writing quality influences readers’ judgments of resumes in business and engineering. Journal of Business and Technical Communication, 6, 38-74.

14.

Chen

Chaiken

(1999). The Heuristic-Systematic Model in its broader context. In Chaiken

Trope

(Eds.), Dual-process theories in social psychology (pp. 73-96). New York, NY: Guilford Press.

15.

Chung

Nam

Stefanone

(2012). Exploring online news credibility: The relative influence of traditional and technological factors. Journal of Computer-Mediated Communication, 17, 171-186.

16.

Connors

Lunsford

(1988). Frequency of formal errors in current college writing, or Ma and Pa Kettle do research. College Composition and Communication, 39, 395-409.

17.

Figueredo

Varnhagen

C. K.

(2005). Didn’t you run the spell checker? Effects of type of spelling error and use of a spell checker on perceptions of the author. Reading Psychology, 26, 441-458.

18.

Gorman

(2014, June 5). N.I.H. seeks $4.5 billion to try to crack the code of how brains function. The New York Times. Retrieved from https://www.nytimes.com/2014/06/06/science/nih-seeks-4-5-billion-to-try-to-crack-the-code-of-how-brains-function.html

19.

Hensley

(2012, May 24). Call for emergency action on polio eradication. NPR. Retrieved from http://www.npr.org/blogs/health/2012/05/24/153579718/call-for-emergency-action-on-polio-eradication

20.

Hughes

Mamiseishvili

(2014). Linguistic profiling in the workforce. In Byrd

Scott

(Eds.), Diversity in the workforce: Current issues and emerging trends (pp. 249-265). New York, NY: Taylor & Francis.

21.

Johnson

VanBrackle

(2012). Linguistic discrimination in writing assessment: How raters react to African American “errors,” ESL errors, and standard English errors on a state-mandated writing exam. Assessing Writing, 17, 35-54.

22.

Keith

S. M.

(2009, September). Sinking subs and collapsing copy desks? The evolution of editing at newspapers and their web sites. Paper presented at the Second Biennial Conference on the Future of Journalism, Cardiff, Wales.

23.

Kreiner

D. S.

Schnakenberg

S. D.

Green

A. G.

Costello

M. J.

McClin

A. F.

(2002). Effects of spelling errors on the perception of writers. The Journal of General Psychology, 129, 5-17.

24.

Martin-Lacroux

Lacroux

(2016). Do employers forgive applicants’ bad spelling in résumés? Business and Professional Communication Quarterly, 80, 321-335.

25.

Metzger

M. J.

Flanagin

A. J.

Medders

R. B.

(2010). Social and heuristic approaches to credibility evaluation online. Journal of Communication, 60, 413-439.

26.

Miestamo

Sinemmäki

Karlsson

(Eds.). (2008). Language complexity: Typology, contact, change. Amsterdam, The Netherlands: John Benjamins.

27.

Mutz

Reeves

(2005). The new videomalaise: Effects of televised incivility on political trust. American Political Science Review, 99, 1-15.

28.

O’Conner

P. T.

(2003). Woe is I: The grammarphobe’s guide to better English in plain English. New York, NY: Riverhead Books.

29.

Paolacci

Chandler

(2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23, 184-188.

30.

Paolacci

Chandler

Ipeirotis

P. G.

(2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5, 411-419.

31.

Queen

Boland

J. E.

(2015). I think your going to like me: Exploring the role of errors in email messages on assessments of potential housemates. Linguistics Vanguard, 1, 283-293.

32.

Rabin

(2014, June 5). Single dose of antibiotic found effective in quelling MRSA. The New York Times. Retrieved from https://www.nytimes.com/2014/06/05/health/single-dose-of-antibiotic-found-effective-in-quelling-mrsa.html

33.

Reilly

Richey

(2011). Ballot question readability and roll-off: The impact of language complexity. Political Research Quarterly, 64, 59-67.

34.

Sampson

Gil

Trudgill

(Eds.). (2009). Language complexity as an evolving variable. Oxford, UK: Oxford University Press.

35.

Schramm

R. M.

Neil Dortch

(1991). An analysis of effective resume content, format, and appearance based on college recruiter perceptions. The Bulletin of the Association for Business Communication, 54(3), 18-23.

36.

Solomon

W. S.

(1985). Technological change in the workplace: The impact of video display terminals on newspaper copy desk work (Unpublished dissertation). University of California, Berkeley.

37.

Solomon

W. S.

(1989, August). Tasks and status: Newspaper copy editing and the division of newsroom labor. Paper presented at the Annual Meeting of the Association for Education in Journalism and Mass Communication, Washington, DC.

38.

Sundar

(1999). Exploring receivers’ criteria for perception of print and online news. Journalism & Mass Communication Quarterly, 76, 373-386.

39.

Thomas

(n.d.). What is the average reading speed the best rate of reading? Health Guidance. Retrieved from http://www.healthguidance.org/entry/13263/1/What-Is-the-Average-Reading-Speed-and-the-Best-Rate-of-Reading.html

40.

Truss

(2006). Eats, shoots & leaves: The zero tolerance approach to punctuation. New York, NY: Gotham Books.

41.

van Weert

van Noort

Bol

van Dijk

Tates

Jansen

. (2011). Tailored information for cancer patients on the Internet: Effects of visual cues and language complexity on information recall and satisfaction. Patient Education and Counseling, 84, 368-378.

42.

Vulanović

(2007). On measuring language complexity as relative to the conveyed linguistic information. SKY Journal of Linguistics, 20, 399-427.

43.

Vultee

(2015). Audience perceptions of editing quality: Assessing traditional news routines in the digital age. Digital Journalism, 3, 832-849.

44.

Walsh

(2008, February 18). The case for copy editing [Web log comment]. Retrieved from http://theslot.blogspot.com/2008/02/case-for-copy-editing.html

45.

Webster’s New World College Dictionary (4th ed.). (2005). Cleveland, OH: John Wiley.

46.

Wolf

Davis

Shrank

Rapp

Bass

Connor

. . . Parker

(2007). To err is human: Patient misinterpretations of prescription drug label instructions. Patient Education and Counseling, 67, 293-300.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.29 MB