Preaching to the choir? An analysis of newspaper readability vis-a-vis public literacy

Abstract

This study examined the interplay among readability, literacy, story type, and demographics in regard to the complexity of local newspaper content. Literacy data were derived from the US Census Bureau’s 2014 American Community Survey and readability data were derived from a content analysis of a random sample of 400 county level newspaper articles in Texas. Readability was measured dually by the Flesch–Kincaid grade reading level and the Flesch reading ease scale. On average, the news articles were written at an 11.63 grade level and a 47.78 ease level. With a 77 percent high school attainment level among residents in the population under study, these readability levels mean that the news articles were written at a level marginally comprehensible to a majority of potential readers. Additionally, business news was written at a 12.32 grade level making it even less comprehensible to a majority of potential readers. Hard news was the second most complex story type with an 11.98 grade level. Overall, residents age 65 and over had the lowest comprehension threshold, despite being the most likely to read newspapers on a daily basis. The author also discusses the implications to local news readability.

Keywords

Flesch–Kincaid literacy local news news complexity newspapers readability

‘Can newspaper content quality be measured, and if, so, what is the quality measure?’ (Bogart, 2004: 40). This statement opens an oft-cited article by media expert Leo Bogart on the conundrum that is measuring the quality of news. He discusses several quality markers that have been adopted by the industry including fairness, balance, breadth of coverage, accuracy, vivid writing, easy navigability, variety of content, and so on. He also mentions the empirical methods used to measure quality and these include circulation, financial resources and revenue, credibility, readability, and so on. Bogart concludes his discourse by stating that whatever constructs one uses to measure the quality of news, it is how well a newspaper invests in its news operation that guarantees a good return. But what better investment is there than producing stories readers can readily comprehend? This question drives the central focus of this study, which is to examine whether the readability of newspaper content aligns with the literacy level of its potential readers.

Readability is a measure of the complexity of written material and refers to ‘the ease to read and comprehend the text’ (Martin and Gottron, 2012: 238). Regardless of the similarity in terminology, readability studies differ from readership studies. The latter aim at among other things measuring the number of readers, how often they do it, and other reading habits. The main difference been the two is that while readability primarily examines news content, readership examines those who read that content and how they do it. Readability studies use a variety of indices to measure the complexity of textual material including the Dale–Chall, Simple Measure of Gobbledygook (SMOG), and Gunning Fog scales. The most widely used index, which this study also uses, is the Flesch–Kincaid grade scale and the accompanying Flesch reading ease scale. The Flesch–Kincaid grade scale measures the school grade level at which textual material is written. The Flesch reading ease scale measures how difficult or easy it is to read the material. The author discusses the computational and definitional details of these two indices in the ‘Method’ section.

Rationale

Three reasons necessitate this study. First, and as the author discusses in later sections, newspaper readability has become somewhat of a forgotten area of scholarly inquiry within journalism research. Pertinent studies date back to the 1940s. The 1970s and 1980s saw an explosion of such inquiry, but little has been done lately. This is despite the fact that the bulk of exitant newspaper readability research indicates that newspaper content is generally written at levels beyond the comprehension level of most people. Meanwhile, other fields such as linguistics, public opinion research, education, and even health communication have sustained a steady stream of scholarly output on the readability of various materials such as health pamphlets, dietary guidelines, online health information, election ballots, textbooks, and so on.

Second, this study is unique because little exists in terms of examining how readability measures up to the literacy and comprehension capabilities of target audiences. This omission is common in most fields and not just in journalism and mass communication. Scholars have examined readability and comprehension in terms of point of origin of materials, readers’ age, online versus print, the effect of translation, and so forth. Third, most journalism scholars have ignored the effect of readability on declining newspaper circulation figures. Most scholars have and still examine readership in relation to circulation and this kind of literature is plenty, current, and ongoing. However, renowned scholars such Guido Stempel have long warned of the danger of ignoring readability and its effect on circulation. Just as the author discusses this issue later, Stempel (1981) states that ‘the key to reaching more readers is making newspapers easier to read’ (p. 32).

Given the above mentioned reasons, this study seeks to find if local newspapers produce news content in alignment with the literacy levels of local readers. It also seeks to find if readability varies in terms of story type. Specifically, the study uses Texas as a case study and analyzes content from a random sample of local newspapers in relation to Census literacy data within the state. The study seeks to find if (1) News readability matches literacy levels? (2) News readability matches the literacy levels in terms of age and gender? and (3) Readability varies among different story types?

The role of local newspapers in society

Research shows that local newspapers play a unique and integral role in society. One such is the promotion of social organization. Research indicates that community newspapers promote social cohesion especially in terms of social bonds, communal beliefs, and shared values with a more profound effect on long term readers (Yamamoto, 2011). This is a role local newspapers have played for a while given that earlier research from as far back as the late 1920s suggests a relationship between newspaper circulation and the integration of local interests (Park, 1929). Other research works indicate that not only do local newspapers foster close relationships with readers, they also improve community connectivity and reinforce local identity while acting as advocates of local issues. The advantage of this localization of journalism is captured well by a local journalist in Kathryn Bowd’s (2011) article examining local newspapers in Australia: ‘I’ve had a lot of cases where people have come to me before any other news outlets, with stories … just because they know who I am’ (p. 83). Additionally, research also indicates that local news readership improves community stakeholder role in local government. Here, Hollander (2010) found that interest in government news positively correlated with reading local print news. This is a result of local newspapers’ emphasis on local government news.

The uniqueness of local newspapers is more pronounced when they are juxtaposed against national publications. For instance, Holt and Major (2010) found that local dailies humanized the 2006 Jenna Six controversy in Louisiana more than national newspapers. While the local newspapers emphasized the human interest frame in this racially charged incident, the national press emphasized morality. Differences between the two types of newspapers also occurred in the coverage of the 2007 Virginia Tech shooting. First, Holody et al. (2013) found that local newspapers covered the 2007 Virginia Tech Shooing longer than national dailies. Second, national newspapers were more likely to racialize their coverage by mentioning the shooter’s race, ethnicity, and immigration status. In a study of the coverage of the Mexican drug-related violence, Lacasse and Forster (2012) also found differences between local and national newspaper coverage. Local dailies with circulation radii within the US border region presented a more positive outlook to the conflict while emphasizing peace journalism frames. The peace journalism frames emphasized by the local newspapers included mentioning local sources, avoiding a blame game, focusing coverage on causes and consequences, providing alternatives for resolution, and avoiding demonizing language among others frames.

Research also shows the uniqueness of local newspapers when it comes to crisis reporting, reader satisfaction, economic reporting, and so on. In a study of the 2009 Red River flood in the Fargo, North Dakota area, Vevea et al. (2011) found that local newspapers used visuals in a manner that reinforced values of resilience. An example would be the portrayal of human dominance over nature via the publishing of images depicting people using machines to control the flood waters. In terms of reader satisfaction, Hansen and Hansen (2011) found that an increase in the reporting of local issues was positively related to reader satisfaction. Their study indicates that an increase in bylined stories, staff photos, and publicly submitted photos led to more positive reader attitudes toward accuracy, good coverage, and overall satisfaction with the paper. Local newspapers have also been shown to have superiority over other local media such as television when it comes to covering local business news. Riffe and Reader (2007) found this much in a survey comparing the performance of various media formats, where newspapers dominated economic reporting in terms of covering local business news, job opportunities, and economic development. Scholars have also explored the uniqueness of local newspapers on matters such as credibility (Hou and Oyedeji, 2011), comprehension (Fleming and Steffens, 2009), political scandals (Fogarty, 2013), and so on. It is for these unique functions and characteristics of local publications that this study focuses on readability and local newspapers.

Newspaper readability

Readability has a long history of scholarly inquiry within various disciplines. In linguistics and literature, such inquiry dates back to the 19th century (Martin and Gottron, 2012). Plenty of readability studies have been done in clinical research, for instance, in regard to the complexity of informed consent documents (Jhanwar and Bishnoi, 2010). In public opinion research, scholars have long examined the readability of election ballots and its effect on voting patterns (Gafke and Leuthold, 1979; Reilly and Richey, 2011; Terris, 1949). Law and policy scholars have examined readability in regard to issues such as the comprehension of jury instructions (Small et al., 2013). Finally, education scholars have studied the readability of teaching material and textbooks (Kasule, 2011; Rottensteiner, 2010; Schneider, 2011).

While similar inquiry within mass communication does not date to as far back as the 19th century as does that in linguistics and literature, scholars here have examined readability from as far back as the 1940s. Case in point would be Melvin Lostutter’s lament about the complexity of news writing in an article published in Journalism Quarterly in 1947: ‘Domestic news … presents such hurdles of hard words and abstractions … to the average reader, who has gone to school less than nine years’ (p. 307). Lostutter goes further by citing an early study of readability which led the then United Press to reduce its writing complexity from a 16.5 grade level to an 11.7 level. Such criticism of complex news writing is still alive in present day scholarship. For instance, Dalecki et al. (2009) point out that the complexity of news content has been on the rise even as the complexity of non-news content is decreasing.

Hence, the complexity of news content has raised quite a bit of scholarly inquiry. A review of literature revealed that there was an explosion of interest in newspaper readability between the 1970s and the 1980s and not much has been done since then. This adds impetus to the current study. In a 1976 examination of the Mini Page, a syndicated newspaper insert targeted at K-7 readers, Bittner and Shamo found readability levels as high as the 8th grade, with a 6.75 average grade reading level across four readability measures – Gunning’s Fog Index, Flesch, Lorge’s, and Spache’s. In another early study of readability, Fowler (1978) compared reading complexity between newspaper content and novels. This longitudinal content analysis focusing on three periods, 1904, 1933, and 1965, found statistically significant differences in complexity between the two types of reading material. Newspapers were overwhelmingly more difficult to read with an average Flesch reading ease scale of 42.8 as compared to 76.12 for novels. Measured on a scale of 1–100, the Flesch reading ease metric (also used in the current study) ranks easier-to-read material on the higher end of the scale. This means that the higher the number, the easier it is to read the material in question. Research indicates that any material with a score below 50 falls under the difficult-to-read category and would be fully comprehensible to readers with some form of college education (Dalecki et al., 2009; Flesch, 1981).

Scholars in the 1980s expanded inquiry by examining issues such as the readability of different newspaper content, readability and gratification, editors’ ability to gauge readability, and the readability of tabloid and non-tabloid content among others. While pointing out the folly of concentrating on readership studies over readability studies to explain declining circulation figures, Stempel (1981) examined the readability of six types of news content including international, national, local, family, sports news, and editorials. The study found international news to be the most complex with family and sports news the easiest to read. In a warning to the industry then, Stempel concluded that on average all newspapers sampled in his study had on average a difficult-to-read level. Stempel further suggests that while studying reader interests and habits (readership) serves newspapers well, scholars also need to examine readability as a contributing factor to low circulation figures.

Fowler and Smith’s (1982) study of Time and Newsweek magazines also deserves mention here. On average, both publications were relatively easy to read with Flesch reading ease scores of 56.85 and 56.18, respectively. The interesting finding here was that immediate gratification news stories such as sports and entertainment were written at a significantly easier reading level than hard news items such as national affairs, science, and business news. In a deeper analysis of news readability, Porter and Stephens (1989) examined the capability of newspaper editors to discern reading difficulty, a test they quickly flunked when they underestimated the complexity of sample reading material by an average of 4.2 grade levels.

More recently, scholars have examined readability in a comparative perspective and also within health communication. Within health communication, pertinent research indicates a consensus suggesting that health material is written at difficult-to-read levels. McInnes and Haglund (2011) found that on average, most online information on diseases such as cancer, alcoholism, depression, dementia, heart disease, and so forth is difficult to read. Specifically, information appearing on academic websites (with .edu suffixes) were the most difficult to read as were those from Wikipedia. Inversely, information derived from government-related sites with .gov and .nhs suffixes were the easiest to read. Similarly, health materials found in academic databases such as the widely used EBSCO were written at a difficult-to-read level (Stossel et al., 2012). Relatedly, Kondilis et al. (2010) found that health pamphlets distributed at Greek public hospitals were easier to read than those distributed at private hospitals. The source is not the only factor that affects the comprehension of health information. Friedman and Hoffman-Goetz (2007) found that textual complexity hampered comprehension among older patients in regards to cancer prevention articles. Other research also shows that health information is usually too difficult to read in regard to dietary supplement leaflets (Clauson et al., 2010), during counseling sessions for both oral and printed information (Nair and Cienkowski, 2010), informed consent documents used in clinical trials (Jhanwar and Bishnoi, 2010), and so on.

Other research has taken a comparative approach to readability. In a study that compared the overall quality of student and community newspapers, Bodle (1996) found that student newspapers were more readable both in terms of reading grade levels and Flesch reading ease. In another comparative study, Jung (2003) compared online business news from CNBC, CNNfn, and CBSMarketWatch against print news from the Wall Street Journal, the New York Times, and USA Today. In terms of reading ease, business news was more difficult to read among all media types. Additionally, business news was written at an average 11.3 grade reading level with the New York Times scoring the highest average at an 11.8 grade level. Other scholars have examined issues such as readability of in-house commercial newsletters (Sparks, 1997), the readability of different wire service stories (Catalano, 1990), readability and reader evaluation of news stories (McAdams, 1992), and readability in relation to information recall and enjoyment among young readers (Wanta and Gao, 1994).

The complexity of news content

As mentioned earlier, scholars from as far back as the 1940s such as Lostutter (1947) to more recent times have lamented the complexity of news content. Dalecki et al. (2009) attribute this complexity to several factors including better educated journalists, the contextualization of news events, and an increasingly complex world in which reporters now operate. In terms of education, Weaver and Wilhoit’s (and lately Wilnat) long running survey of the journalism industry indicates that currently 92 percent of all journalists are college graduates. This stands in comparison to 58 percent in 1958, 70 percent in 1982, and 89 percent as recently as 2002 (Willnat and Weaver, 2014).

Meanwhile, the contextualization of news, as opposed to the objective writing style, has been on the rise. Contextualization refers to a news writing style that is more interpretive than descriptive of facts. The style examines news events in wider angles and is more analytic with an aim to explain, remedy, investigate, or inspire (Fink and Schudson, 2014). According to Patterson (1994), two factors explain the rise of contextualization. One is the advent of interpretive reporting, a by-product of a shift in journalistic perspective which saw reporters adopt a more assertive and confrontational approach toward politicians and government officials. Beginning in the 1960s, journalists began to scrutinize and question politicians more and were also willing to further examine and challenge political statements. This new era in journalism was aptly surmised by famed wartime reporter turned political journalist Theodore H White: ‘The way of advancement in journalism is to attack’ (Paterson, 79). Another factor has to do with the impact of the then newer medium of television, whose interpretive and dramatic news presentation style would soon be adopted by the now more assertive print journalists. This trend is illustrated in a long-term study of reporting frameworks in election news in the New York Times, where Patterson reports that interpretive coverage increased from a mere 8 percent of front page content in 1960 to 80 percent by 1992.

Journalists currently operate in a more complex world than ever before and a majority of them now think that reporting techniques should reflect that. The aforementioned 2014 Willnat and Weaver study found this much, with 69 percent of journalists stating that analyzing complex societal problems should be a priority in news reporting. Meanwhile, there has been a long standing criticism of the lack of complexity within science journalism writing. Accompanying this complaint have been calls for more contextualization of science news reporting (Dentzer, 2009; Olvera-Lobo and López-Pérez, 2015; Secko et al., 2013). Overall, complexity is exacerbated by the fact that present day journalists have to compete with new media practitioners such as bloggers and citizen journalists, who now produce first person accounts of news events and go as far as to assume some functions of mainstream journalism practice (Compton and Benedetti, 2010; Hermida, 2010; Urban and Schweiger, 2014). In turn, this has led to a de-emphasis of the getting-information-out-quickly function in favor of more complex analysis of news events, which could also be explained by an explosion of niche media outlets tailored toward younger and more educated audiences (Willnat and Weaver, 2014). In a nutshell, it is indeed a complex world for present day journalism.

Research questions

RQ1. What is the comprehension threshold among Texas readers in regards to local newspaper content complexity?

RQ2. What is the comprehension threshold of local newspaper content among Texas readers in regards to gender and age?

RQ3. What is the readability of the five story types: hard news, sports news, entertainment news, business news, health news, and education news in local Texas newspapers?

Method

Sampling and data collection

This study used both primary and secondary data. Readability data were derived primarily via the content analysis of local newspaper articles. Literacy data were derived secondarily from the US Census Bureau’s 2014 American Community Survey education attainment database. The unit of analysis for readability was an entire news article from a local newspaper. Because this study uses Texas as a case study, the sampling frame encompassed all 254 counties in the state and the author randomly selected 15 percent of these for a total of 40 counties. The author then selected the largest newspaper in each county. Large publications with multi-county circulation were excluded. These include dailies such as The Houston Chronicle, Dallas Morning News, The Christian Science Monitor, and so on. From each local publication, the author then randomly selected 10 news articles published no earlier than 2014. This time range suitably matched the years within the Census dataset used to collect the literacy data. Because this study strictly focuses on local newspapers, articles derived from wire services such as the Associated Press were excluded. Additionally, the author took care to select articles that specifically dealt with local or county-wide issues – see Table 1 for examples of local interest headlines. This way, the sample articles accurately reflect local news writing dynamics. This resulted into a sample of 400 news articles and all were analyzed. Three trained coders were used for the content analysis, and the inter-coder agreement for the one category that needed personal judgment (story type) was Krippendorff’s alpha = 0.83.

Table 1.

Examples of local interest headlines.

Title	Newspaper
Things are looking up in downtown Stockdale	Wilson County News
County Begins Year With General Housekeeping Tasks	Mason County News
Police seek tips in robbery case	The Brownsville Herald
Ken’s Furniture demolished to make way for CVS Pharmacy	Raymondville Chronicle
El Paso firefighter accused of groping girl	El Paso Times
Only Minor Injuries After Massive Gas Plant Explosion Near Orla	Hudspeth County Herald
Girls BB: Maidens Power Past Prowlers	Seminole Sentinel
Hotel project inches forward	Wise County Messenger
Jose Gonzalez named interim superintendent for McAllen schools	The Monitor
Plainview native finalist in Super Bowl commercial contest	The (Plainview) Herald

Dependent variables

Readability

This variable was measured dually by the Flesch–Kincaid grade reading level and the Flesch reading ease level. Data for both parameters were derived automatically from Microsoft Word’s ‘Show readability statistics’ function and needed no personal judgment from the coders. The Flesch–Kincaid grade reading level, which gauges the grade at which textual material is written, derives from a combination of the word count, syllables, sentence length, and a number of formulaic numerical constants (see formula below). The Flesch reading ease level, which measures how easy or difficult textual material is to read is measured on a 1–100 scale with difficult-to-read material on the lower end of the scale. Specifically, material scoring 50 or below is generally considered difficult to read and may need some level of college education to comprehend (Dalecki et al., 2009; Flesch, 1981). The specific formulae used by Microsoft Word (2016) are:

F l e s c h g r a d e r e a d i n g s c o r e = (0.39 \times A S L) + (11.8 \times A S W) - 15.59

F l e s c h r e a d i n g e a s e l e v e l = 206.835 - (1.015 \times A S L) - (84.6 \times A S W)

where ASL is the average sentence length (the number of words divided by the number of sentences) and ASW is the average number of syllables per word (the number of syllables divided by the number of words).

Literacy

This variable was derived secondarily from the Census data. The Census Bureau presents literacy data based on the proportion of county residents who have attained education at different levels such as less than high school, high school graduate, some college, associate’s degree, bachelor’s degree or higher, and so on. Additionally, these data are segregated by age and gender, for instance, female residents 25 years and over with an associate’s degree, residents 65 years and over with a bachelor’s degree, total male residents with high school education, and so on.

Independent variables

Story type

This variable examines the readability of different story types. The categories were derived from a pilot study of the news articles analyzed and these were (a) hard news, (b) sports news, (c) entertainment news, (d) business/financial news, (e) health news, and (f) education news. As mentioned above, this was the only category that needed personal judgment from the three coders and the inter-coder reliability was Krippendorff’s alpha = 0.83.

Results

Overall, 400 articles were analyzed for this study. On average, articles were 488 words in length with a Flesch–Kincaid grade reading level of 11.63 and a Flesch reading ease score of 47.78. The least complex article was written at a 9th grade level and the most complex at a 14th grade level. In terms of Flesch reading ease, the most-difficult-to-read article was written at a 37 level and the least-difficult-to-read at 60. It is important to reiterate here that any material with a 50 or lower reading ease score is generally deemed difficult to read and needs some form of college education to comprehend (Dalecki et al., 2009; Flesch, 1981).

Research question 1 queried the percentage of Texas residents likely to comprehend the news articles. Here, the sample data show that on average, at least 77 percent of all residents in the counties examined have graduated high school and are therefore likely to comprehend the news at the reported 11.63 average grade reading level. However, these results come with three caveats. First, and as the author discusses at length in the next section, educational attainment at a certain grade level does not equal reading proficiency at the same grade level. Additionally, this reading discrepancy is quite marked in the United States. Second and relatedly, because the news articles were written at a marginally comprehensible level with a Flesch–Kincaid score that almost matches a high school equivalent of a 12th grade reading level, it leaves very little room for comprehension error. This means that most articles were written at a level barely readable by a high school graduate if at all. More so, the Flesch reading ease of 47.78 demonstrates marginal reading ease given that any material with an ease score of 50 or less is generally deemed difficult-to-read and needs some level of college attainment. Third, closer examination of Figure 1 shows that guaranteed comprehension may be as low as 50 percent given that only 35 percent of the sampled residents have some form of associate degree and only 15 percent have a bachelor’s degree or higher. Besides, 50 percent is a generous estimate if we assume that there is no overlap between those with associate’s and bachelor’s degrees, which likely is not the case.

Figure 1.

Educational attainment and likelihood of news comprehension.

In regards to gender (RQ2), the data show that female residents are slightly more likely to comprehend the news because they have a slightly higher high school achievement rate (79%) as compared to males (76%) – Figure 2. However, differences in educational attainment were not statistically significant. In regards to specific age groups (Figure 3), data show that residents aged 24–34 were the most literate in terms of high school educational attainment (81.72%) and those over 65 were the least literate (69.83%). Analysis of variance (ANOVA) analysis showed that intra-age group educational attainment differences were statistically significant and post-hoc analysis showed that the literacy level for those aged 65 and over was significantly lower than all other age groups (F = 9.33; Df = 3; p < 0.001; N = 40 counties). Taken at face value, this means that 30 percent of residents 65 years and over will have difficulty comprehending local news. Given that only 14.7 percent of residents in this age group hold a bachelor’s degree and local news is written at a nearly 12th grade reading level, the news comprehension threshold in this group could be lower. Even among the most literate group (25–34 years), college education attainment stood at 14 percent meaning that with a near 12th grade news writing complexity, guaranteed news comprehension could be much lower than the apparent 81.72 percent reported here.

Figure 2.

Gender, educational attainment, and likelihood of news comprehension.

Figure 3.

Age, educational attainment, and likelihood of news comprehension.

Research question 3 queried the readability of different story types and here the data indicate that business/financial news was the most complex (Table 2) with a 12.32 Flesch–Kincaid grade reading level. Additionally, hard news and health news were also written at a near 12th grade reading level (11.98 and 11.94, respectively). Table 2 also shows the ANOVA analysis results that returned significant differences among the different story types and the corresponding post-hoc analyses. The readability data here mean that business and financial news articles were written at a level outside the comprehensibility threshold of an entire population who did not have either some form of associate or bachelor’s degree (given that only 35% of the sample residents have attained at least an associate’s degree and 15% a bachelor’s degree).

Table 2.

Flesch grade level and reading ease of story types.

Story type	Grade level mean (SD)	Ease mean (SD)	N
1. Business news	12.32 (2.60)^a	43.63 (12.97)^b	56
2. Hard news	11.98 (2.30)	46.27 (11.44)	193
3. Health news	11.94 (2.36)	46.32 (13.64)	20
4. Education news	11.56 (2.00)	46.92 (13.10)	29
5. Arts/entertainment news	11.11 (2.54)^a	50.07 (13.14)^b	69
6. Sports news*	9.35 (1.87)	60.23 (9.04)	32

SD: standard deviation.

Means that share the same superscript differ at the p < 0.05 level in the Tukey post-hoc comparisons.

Additionally, sports news significantly differed from all other news types in both readability measures.

What stands out here is that hard news, usually the most common news type, and accounting for 48 percent of all articles analyzed here, was barely compressible to the majority of potential readers because it scored a near 12th grade readability level. Additionally, all but one story type (sports) were written at a Flesch reading ease score of 50 or below with business/financial news and hard news written at the most difficult reading ease levels (43.63 and 46.27, respectively). ANOVA analysis for reading ease also returned significant differences among the six news types with sports being significantly less complex than all other story news types. ANOVA for grade level: F = 8.61; Df = 5; p < 0.001; N = 400. ANOVA for reading ease: F = 9.36; Df = 5; p < 0.001; N = 400. Post-hoc results are shown in superscript in Table 2.

Discussion

This study takes a unique approach to a well examined area of newspaper journalism research by incorporating Census literacy data into news readability. Several key findings emerged here. One has to do with news complexity in general. Research from as far back as the forties (Lostutter, 1947) to more recent times (Dalecki et al., 2009) has shown newspaper content to have a complexity problem and this study comes to similar conclusions. On average, the articles analyzed here were written at a near 12th grade Flesch–Kincaid reading grade level (11.63). Additionally, they averaged a 47.78 Flesch reading ease score. As mentioned in the ‘Results’ section, these two numbers do not bode well for readability. For one, only 77 percent of the residents in the counties sampled have completed high school. Because attaining a certain academic grade does not necessarily translate into equivalent reading proficiency at that grade (National Center for Education Statistics, 2013), the actual news comprehension level is most likely lower than that reported here. Additionally, because any material scoring a reading ease score lower than 50 is generally considered difficult-to-read and needs some level of college education to comprehend (Flesch, 1981; Dalecki et al., 2009), the threshold of news comprehension is possibly lower.

There is plenty of evidence for such a scenario. A 2014 study by the Annie Casey Foundation found that almost two-thirds of 4th graders in the United States under-achieved in reading proficiency at their grade level. Even though reading proficiency improves over time, there still is substantive under-achievement nationally. A similar and ongoing study by the National Center for Education Statistics (2013, 2016) reports that on average, 8th and 12th grade students scored 278 out of a possible 500 in a reading competency assessment in 2013 and a lower 276 in 2015. The same study also reports that only 34 percent of 8th graders and 37 percent of 12th graders were proficient readers in 2015. Additionally, the study found that no statistically significant improvement in reading proficiency occurred between years 2011 and 2013 for 38 out of the 50 states, Texas included. In fact, of the 21 large cities and school districts listed in the report, only one, Hillsborough County, Fl., at 267 scored above the 266 national reading proficiency mean score for the 8th grade level. Between 2013 and 2015, only one state (West Virginia) scored statistically significant improvement in reading proficiency.

On a more nuanced look, the results also indicate that certain story types were written out of the comprehension range of most residents. One such was business/financial news which not only carried a 12.32 grade level score, it had an accompanying 43.63 reading ease score, the lowest of the six story types examined here. This means that this one story type will be suitable mostly for readers with some college education and only 35 percent of the sample residents had some form of associate’s degree and only 15 percent had attained a bachelor’s degree. These readability scores for business news are more difficult than those from national publications. The aforementioned Jung (2003) study found that the Wall Street Journal and the New York Times scored 10.5 and 11.8 grade levels for business news respectively. On average, the two newspapers scored 42.25 on reading ease, much comparable to the 43.63 for local business news reported here. One might rightfully argue that financial news would interest highly educated readers and thus the complexity. However, this would be an unlikely scenario given that this study analyzed local newspapers which normally target a smaller readership with deep local ties. This means that most business news would have a local angle suited for the typical local resident.

Examples of such local coverage from this study included articles on fuel and utility charges, small business Saturday, property valuation and taxation, income tax preparation, and so on. But even if such articles were targeted toward elite readers, what stood out was that hard news was written at higher than average grade and reading ease levels too, 11.98 and 46.27, respectively. Health news was also written at similar complexity in terms of grade level and reading ease (11.94 and 46.32, respectively). Examples of local interest health news articles include a speech by a local health center CEO, safety of the local water supply system, closure of a local hospital, a blindness eradication effort by a local chapter of the Lions Club International, and so on. It is noteworthy that these three story types, which made up the bulk of the articles analyzed (67%), averaged a 12.08 grade reading level with a corresponding 45 reading ease score. It would not be a stretch to conclude that most news content in local Texas newspapers is complex and probably out of the comprehension threshold of most potential readers.

Another key finding has to do with age, readability, and literacy. The results indicate that 30 percent of those in the 65 years and over category had the lowest comprehension level for news written at the 12th grade level, the lowest rate among all four age categories. Given that some story types such as business and hard news were written at higher complexity levels than the average of all articles examined here, the 30-percent incomprehensibility mentioned above might be a conservative estimate. This number has substantive meaning given that a Pew Research Center (2014) survey had this as the age group by far most likely to read newspapers on a daily basis. This age category has also held this lead position for the past 15 years. The Pew survey also indicated that as of 2014, 52 percent of Americans age 65 and above read newspapers daily as compared to 20 percent of those 25–34 and 40 percent of those between 55 and 64 years. Yet, this study finds that this is also the age group most likely to be shut out in terms of news complexity and comprehension.

Limitations

The three main limitations to this study are scope, the methodological validity of the Flesch–Kincaid scale, and the possible challenge to the use of literacy data to measure news readability. In terms of scope, this article analyzed a sample of Texas counties and the results can only be generalized within the state. This, however, does not mean that the results cannot be used to discuss national readability trends or even spur such a study. Second, the Flesch–Kincaid scale is not without its shortcomings. For one, while it is the most commonly used readability measure, it is only one of several measures with others including the Dale-Chall, Gunning Fog and SMOG indices. Additionally, these scales lack uniformity because they sometimes produce different readability results when tested on the same textual material with variations of up to two whole grade reading levels (Burke and Greenberg, 2010). Some scholars have also suggested that readability scales are not the best measures to use when testing younger readers or people with poor reading habits. Such subjects would be better served by using oral reading fluency tests. These tests measure one’s ability to read text fluently and quickly (Begeny and Greene, 2014). Others have pointed out that some readability scales such as the Flesch–Kincaid tend to underestimate textual complexity while some like the SMOG scale are sensitive to the length of sample passages under study (Burke and Greenberg, 2010). Finally, readability is just that, and it does not take into account factors such as a subject’s age, his/her reading preference, or even reading technique and strategy (Pitcher and Fang, 2007). This means that when testing the readability of textual material, an ideal analysis would include an amalgam of parameters.

Third, because this is the first study to analyze news readability vis-a-vis Census literacy data, some might challenge the relational aspect of these two variables. This is an understandable criticism but a number of factors should mitigate this concern. One is the fact that newspaper circulation figures have been declining over time and the future looks no brighter. This compels both academicians and media practitioners to examine all possible avenues to correct this downward trend. Second, given the pivotal role that newspapers play in society in general and the unique role that local newspapers play, no stone can be left unturned in the quest for a solution to sagging circulation figures. Finally, it is inarguable that the local news articles analyzed in this study were written beyond the comprehension levels of a majority of potential readers (as compared to literacy data). There definitely exists a problem and this last observation alone should set off a series of alarms.

Solutions and recommendations

This article set out by posing the question, ‘but what better investment is there than producing stories readers can readily comprehend?’ The results here indicate that local Texas newspapers might be underserving not just the plurality of potential readers with minimal education, but also their most loyal customer base, those aged 65 years and over. But it would be remiss if the author did not discuss solutions to the readability problem. Several scholars have offered a variety of solutions to combat news complexity. For instance, Dalecki et al. suggest a unique approach where journalism educators avoid using made-up stories in news writing classes. Because their study found that deceptive news pieces were more readable than authentic ones, such an approach would inure students with the capabilities to write readable stories when dealing with on the job and real world complexities rather than simulated ones in a classroom context.

They also suggest more use of direct quotations, which they found to reduce reading complexity. Another solution would be the use of shorter sentences. It is important to note that sentence length is a marker of reading ease. Research indicates that long sentences, usually 25 words or more, create material that is difficult to read (Flesch, 1981; Jung, 2003). Finally, because research shows that a simple eye test is likely to cause writers to underestimate the complexity of news content (Porter and Stephens, 1989), journalists could run computerized readability tests for all stories during the editing process. An automated readability test provides an objective and accurate method to gauge textual complexity. Readability tests are easy to run and should not be an additional onus to reporters already burdened by deadline and other pressures. For instance, by simply filling a single check box, Microsoft Word automatically produces readability statistics under its spell check function. An alternative could be one of the many free readability applications online.

These solutions come with some caveats though. Outside of the automatic readability tests mentioned above, the other solutions suggested require more effort and might be onerous to the reporters. For instance, the use of shorter sentences might lengthen the writing and editing process. It might even force some to change long held writing habits and this might produce resistance and friction in the newsroom. The use of real-life stories in news writing and reporting classes will definitely require journalism instructors to adjust their syllabi and teaching materials. This too might meet some resistance. But while such burdens might be hard to bear on the short term, the long term benefits of easily comprehensible news stories outweigh the negatives. For one, it makes little sense to produce news content that a majority of readers cannot fully comprehend. Second, it is not unreasonable to assume that a barrage of difficult-to-read news articles would be discouraging to certain readers over the long run. In turn, this might exacerbate an already bad situation in regards to declining readership and circulation figures and this hurts the journalism industry as a whole.

Footnotes

Acknowledgements

I would like to thank my student coders for their help and dedication: Andrea Torres, Valentine Treviño, and Maythe Villareal.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author biography

Ben Wasike is an Associate Professor in the Communication Department at the University of Texas Rio Grande Valley. He teaches classes in visual communication, communication theory, and social media communication among others. His research focuses on social media, political communication, visual communication, rhetorical analysis, and law and policy issues.

References

Annie Casey Foundation (2014) Early reading proficiency in the United States. Available at: http://files.eric.ed.gov/fulltext/ED544749.pdf

Begeny

Greene

(2014) Can readability formulas be used to successfully gauge difficulty of reading materials?: Readability formulas. Psychology in the Schools 51(2): 198–215.

Bittner

Shamo

(1976) Readability of the ‘mini page’. Journalism Quarterly 53(4): 740–743.

Bodle

(1996) Assessing news quality: A comparison between community and student daily newspapers. Journalism & Mass Communication Quarterly 73(3): 672–686.

Bogart

(2004) Reflections on content quality in newspapers. Newspaper Research Journal 25(1): 40–53.

Bowd

(2011) Reflecting regional life: Localness and social capital in Australian country newspapers. Pacific Journalism Review 17(2): 72–91.

Burke

Greenberg

(2010) Determining readability: How to select and apply easy-to-use readability formulas to assess the difficulty of adult literacy materials. Adult Basic Education and Literacy Journal 4(1): 34–42.

Catalano

(1990) On the wire: How six news services are exceeding readability standards. Journalism Quarterly 67(1): 97–103.

Clauson

Zeng-Treitler

Kandula

(2010) Readability of patient and health care professional targeted dietary supplement leaflets used for diabetes and chronic fatigue syndrome. Journal of Alternative & Complementary Medicine 16(1): 119–124.

10.

Compton

Benedetti

(2010) Labour, new media and the institutional restructuring of journalism. Journalism Studies 11(4): 487–499.

11.

Dalecki

Lasorsa

Lewis

(2009) The news readability problem. Journalism Practice 3(1): 1–12.

12.

Dentzer

(2009) Communicating medical news – Pitfalls of health care journalism. New England Journal of Medicine 360(1): 1–3.

13.

Fink

Schudson

(2014) The rise of contextual journalism, 1950s–2000s. Journalism 15(1): 3–20.

14.

Fleming

Steffens

(2009) Understanding readers of local newspapers and editorial journalism in small communities. Paper presented at the annual meeting of the International Communication Association, Chicago, IL, 21–25 May.

15.

Flesch

(1981) How to Write Plain English: A Book for Lawyers and Consumers. New York: Barnes & Noble.

16.

Fogarty

(2013) Scandals, news coverage, and the 2006 congressional elections. Political Communication 30(3): 419–433.

17.

Fowler

Jr (1978) The comparative readability of newspapers and novels. Journalism Quarterly 55(3): 589–592.

18.

Fowler

Jr Smith

(1982) Readability of delayed and immediate reward contend in Time and Newsweek. Journalism Quarterly 59(3): 431–460.

19.

Friedman

Hoffman-Goetz

(2007) An exploratory study of older adults’ comprehension of printed cancer information: Is readability a key factor? Journal of Health Communication 12(5): 423–437.

20.

Gafke

Leuthold

(1979) The effect on voters of misleading, confusing, and difficult ballot titles. Public Opinion Quarterly 43(3): 394–401.

21.

Hansen

(2011) Newspaper improves reader satisfaction by refocusing on local issues. Newspaper Research Journal 32(1): 98–106.

22.

Hermida

(2010) Twittering the news: The emergence of ambient journalism. Journalism Practice 4(3): 297–308.

23.

Hollander

(2010) Local government news drives print readership. Newspaper Research Journal 31(4): 6–15.

24.

Holody

Park

Zhang

(2013) Racialization of the Virginia Tech shootings. Journalism Studies 14(4): 568–583.

25.

Holt

Major

(2010) Frame and blame: An analysis of how national and local newspapers framed the Jena six controversy. Journalism & Mass Communication Quarterly 87(3–4): 582–597.

26.

Hou

Oyedeji

(2011) News credibility and user-generated content: Young readers’ perceived credibility of local newspaper’s user-generated content. Paper presented at the annual meeting of the International Communication Association, Boston, MA, 26–30.

27.

Jhanwar

Bishnoi

(2010) Comprehensibility of translated informed consent documents used in clinical research in psychiatry. Indian Journal of Psychological Medicine 32(1): 7–12.

28.

Jung

(2003) Business news web sites differ from newspapers in business content. Newspaper Research Journal 24(2): 114.

29.

Kasule

(2011) Textbook readability and ESL learners. Reading and Writing 2(1): 63–76.

30.

Kondilis

Akrivos

Sardi

, et al. (2010) Readability levels of health pamphlets distributed in hospitals and health centres in Athens, Greece. Public Health 124(10): 547–552.

31.

Lacasse

Forster

(2012) The war next door: Peace journalism in US local and distant newspapers’ coverage of Mexico. Media, War & Conflict 5(3): 223–237.

32.

Lostutter

(1947) Some critical factors of newspaper readability. Journalism Quarterly 24(4): 307.

33.

McAdams

(1992) Readability reconsidered: A study of reader reactions to fog indexes. Newspaper Research Journal 13–14(4/1): 50–59.

34.

McInnes

Haglund

(2011) Readability of online health information: Implications for health literacy. Informatics for Health and Social Care 36(4): 173–189.

35.

Martin

Gottron

(2012) Readability and the web. Future Internet 4(1): 238–252.

36.

Microsoft Word (2016) Test your document’s readability. Available at: https://support.office.com/en-us/article/Test-your-document-s-readability-0adc0e9a-b3fb-4bde-85f4-c9e88926c6aa?ui=en-US&rs=en-US&ad=US

37.

Nair

Cienkowski

(2010) The impact of health literacy on patient understanding of counseling and education materials. International Journal of Audiology 49(2): 71–75.

38.

National Center for Education Statistics (2013) Mapping State Proficiency Standards onto NAEP Scales: Results from the 2013 NAEP Reading and Mathematics Assessments. National Assessment of Educational Progress. Available at: http://nces.ed.gov/nationsreportcard/subject/publications/studies/pdf/2015046.pdf

39.

National Center for Education Statistics (2016) Reading performance. Available at: https://nces.ed.gov/programs/coe/pdf/coe_cnb.pdf

40.

Olvera-Lobo

López-Pérez

(2015) Science journalism: The standardisation of information from the press to the internet. JCOM: Journal of Science Communication 14(3): 1–12.

41.

Park

(1929) Urbanization as measured by newspaper circulation. American Journal of Sociology 35(1): 60–79.

42.

Patterson

(1994) Out of Order: An Incisive and Boldly Original Critique of the News Media’s Domination of America’s Political Process. New York: Vintage Books.

43.

Pew Research Center (2014) Daily readership by age: Percentage nationally who read any daily newspaper yesterday. Available at: http://www.journalism.org/media-indicators/newspapers-daily-readership-by-age/

44.

Pitcher

Fang

(2007) Can we trust levelled texts? An examination of their reliability and quality from a linguistic perspective. Literacy 41(1): 43–51.

45.

Porter

Stephens

(1989) Estimating readability: A study of Utah editors’ abilities. Newspaper Research Journal 10(2): 87–96.

46.

Reilly

Richey

(2011) Ballot question readability and roll-off: The impact of language complexity. Political Research Quarterly 64(1): 59–67.

47.

Riffe

Reader

(2007) Most rely on newspapers for local business news. Newspaper Research Journal 28(2): 82–98.

48.

Rottensteiner

(2010) Structure, function and readability of new textbooks in relation to comprehension. Procedia: Social and Behavioral Sciences 2(2): 3892–3898.

49.

Schneider

(2011) Assessing the readability of college textbooks in public speaking: Attending to entry level instruction. Communication Teacher 25(4): 246–255.

50.

Secko

Amend

Friday

(2013) Four models of science journalism: A synthesis and practical assessment. Journalism Practice 7(1): 62–80.

51.

Small

Platania

Cutler

(2013) Assessing the readability of capital pattern jury instructions. Jury Expert 25(1): 18–22.

52.

Sparks

(1997) Employee newsletter readability for a large public utility. Public Relations Quarterly 42(3): 37–40.

53.

Stempel

(1981) Readability of six kinds of content in newspapers. Newspaper Research Journal 3(1): 32–37.

54.

Stossel

Segar

Gliatto

, et al. (2012) Readability of patient education materials available at the point of care. JGIM: Journal of General Internal Medicine 27(9): 1165–1170.

55.

Terris

(1949) Are poll questions too difficult? Public Opinion Quarterly 13(2): 314–319.

56.

Urban

Schweiger

(2014) News quality from the recipients’ perspective. Journalism Studies 15(6): 821–840.

57.

Vevea

Littlefield

Fudge

, et al. (2011) Portrayals of dominance: Local newspaper coverage of a natural disaster. Visual Communication Quarterly 18(2): 84–99.

58.

Wanta

Gao

(1994) Young readers and the newspaper: Information recall and perceived enjoyment, readability, and attractiveness. Journalism Quarterly 71(4): 926–936.

59.

Willnat

Weaver

(2014) The American journalist in the digital age: Key findings. Available at: http://news.indiana.edu/releases/iu/2014/05/2013-american-journalist-key-findings.pdf

60.

Yamamoto

(2011) Community newspaper use promotes social cohesion. Newspaper Research Journal 32(1): 19–33.