Development and assessment of the Korean Author Recognition Test

Abstract

This research reports the development and evaluation of a Korean Author Recognition Test (KART), designed as a measure of print exposure among young adults. Based on the original, English-language version of the Author Recognition Test (ART), the KART demonstrates significant relationships with offline measures of language ability, as well as online measures of word recognition. In particular, KART scores were related to participants’ responses on the Comparative Reading Habits (CRH) checklist, suggesting that KART is a valid measure of print exposure. In addition, KART scores showed reliable correlations with offline measures of vocabulary knowledge and language comprehension. Finally, results from a lexical decision task showed that KART scores modulated the magnitude of the word familiarity effect, such that the effect was smaller for participants with higher KART scores The results suggest that the ART is a language-universal task that measures print exposure, which is useful for explaining individual differences in language comprehension abilities and word recognition processes.

Keywords

Korean Author Recognition Test print exposure language ability individual differences lexical decision

Introduction

A large literature indicates that higher levels of language exposure are associated with enhanced vocabulary knowledge, better reading comprehension skills, and greater abilities in many other language-related outcomes. This relationship between language exposure and language ability is detectable in children as young as preschoolers, for whom early print exposure is associated with not only basic language ability but also a greater interest in reading (Fletcher & Reese, 2005). As children develop more refined reading skills, they are more likely to devote their leisure time to reading. By fourth grade, children start learning new concepts or acquiring new information from books (i.e., read to learn), which continues to enhance their language abilities (Chall, 1983; Hirsch, 2003; Vellutino, Tunmer, Jaccard, & Chen, 2007). This spiral continues into adolescence and adulthood, such that increased levels of print exposure lead to better language abilities, which in turn lead to increased levels of print exposure (Paris, 2005; West, Stanovich, & Mitchell, 1993)—a pattern referred to as reciprocal causation (Mol & Bus, 2011). The robust relationship between print exposure and language outcome measures highlights the potential usefulness of developing a measure that estimates an individual’s exposure to printed language. Such a measure might play an important role in explaining individual variability in both online and offline measures of language ability.

Early measures of print exposure tended to come from self-report questionnaires (Greaney, 1980). However, this approach is highly subjective, and people tend to answer in socially desirable ways, exaggerating their reading habits (Ennis, 1965; Paulhus, 1984; Zill & Winglee, 1990). In contrast to the subjective nature of self-report measures, Stanovich and West (1989) constructed the Author Recognition Test (ART) as an objective assessment of print exposure. In the original version of the ART, the names of 50 real authors (from bestselling books) and 50 foils were included in a list, and participants were instructed to select the names that they recognised to be real authors. To prevent indiscriminate selection, scores are penalised for selecting foils. Several findings in particular are noteworthy from Stanovich and West. First, they reported a strong correlation between ART scores and word processing ability. Specifically, higher ART scores were associated with better performance on the Experimental Spelling Test (EST: Fischer, Shankweiler, & Liberman, 1985) and the Wide Range Achievement Test–spelling (WRAT: Jastak & Wilkinson, 1984). Second, ART scores were related to orthographic processing ability independent of phonological processing ability. Performance on the ART did not explain all the variance in orthographic processing; however, after removing the phonological processing factor, ART scores were a strong predictor of variance in orthographic processing. Finally, Stanovich and West demonstrated that performance on the ART was unrelated to social desirability, which represents an improvement over self-report questionnaires. In short, Stanovich and West verified the ART as an objective measure of print exposure that is related to language ability. Since then, the ART has been widely used to measure one’s variability in print exposure and has been reported as a powerful tool to predict a variety of language abilities (for a comprehensive review, see Mol & Bus, 2011).

Stanovich and West’s (1989) original ART was updated 20 years later by Acheson, Wells, and MacDonald (2008) to remove authors who were no longer familiar and to include new authors. Specifically, Acheson et al. used 15 authors from Stanovich and West’s original ART and 50 new authors for a final list that consisted of 65 real authors and 65 foils. Acheson et al. showed a reliable correlation between scores on the updated ART and scores on the ACT, a standardised achievement test for high school students. Using Acheson et al.’s version of the ART, Choi, Lowder, Ferreira, and Henderson (2015) also found correlations between ART scores and online reading measures such that mean fixation durations during sentence reading were shorter as readers’ ART score became higher (see also Lowder & Gordon, 2017; Moore & Gordon, 2015). These results indicate that print exposure, as measured by the ART, relates to not only readers’ offline reading performances but also online reading behaviours.

As previously mentioned, Mol and Bus (2011) argued that there is a reciprocal causation between print exposure and language ability. They applied a meta-analysis to examine the relation between print exposure and reading abilities including comprehension, technical reading, and spelling based on 99 studies. Given that print exposure is both a cause and an effect of enhanced language abilities, the gap in reading ability between proficient readers and less-skilled readers increases with age. For example, ART scores explained 34% of the variance in reading ability among undergraduates, compared with 12% of the variance among preschoolers. This suggests that measures of print exposure can be quite effective in explaining individual differences in reading ability among college students.

There is a substantial body of research demonstrating relationships between ART scores and English language ability across a wide range of ability measures (for a review, see Mol & Bus, 2011). However, to the best of our knowledge, only one study so far has examined the relationship between ART and language ability in Asian languages (Chen & Fang, 2015). Chen and Fang (2015) constructed a Chinese version of the Author Recognition Test (CART) in Taiwan and tested whether there were relationships between CART scores and a variety of measures of reading ability among college-level students. The results showed that CART scores were indeed correlated with offline reading measures such as vocabulary size and reading comprehension, suggesting that measures of print exposure are useful tests for predicting variability in reading ability even in a country in which English is not a dominant language.

Given that the relationship between print exposure measured by the ART and reading ability emerges in English and Chinese, we would expect that the relationship should emerge in additional languages as well. Accordingly, the main goal of this study was to develop an objective measure of print exposure in Korean (the Korean Author Recognition Test [KART]) and test it against online and offline measures of language ability. As far as we know, this is the first attempt to link print exposure and language ability measures in Korean. Our specific goals were twofold. First, we investigated whether the KART is a valid measure of print exposure by examining correlations between KART scores and other self-report measures of print exposure, as well as offline language tasks that measure language ability. Second, we investigated the relationship between KART scores and measures of online language processing.

The first specific goal was to investigate the relationship between KART scores and other self-report measures of print exposure. Previous work has demonstrated moderate correlations between scores on the ART and scores on other measures of print exposure. For example, the ART shows reliable correlations with scores on the Home Literacy Environment questionnaire, which is a self-report measure designed to assess a child’s literacy habits and the literacy habits of others in the household (Mol & Bus, 2011). Acheson et al. (2008) reported a significant correlation between scores on the ART and responses on the Comparative Reading Habits (CRH) questionnaire, which asks respondents to rate themselves on criteria such as reading speed and interest in reading. Similar correlations have been reported by Choi et al. (2015) in English, as well as Chen and Fang (2015) in Chinese.

The second goal of this study was to examine the extent to which online language processing is modulated by scores on the KART. Indeed, several previous studies have shown that aspects of online language processing are affected by individual differences in print exposure. For example, Chateau and Jared (2000) conducted multiple experiments to examine the effects of print exposure on phonological and orthographic word recognition processes. Participants in these experiments were divided into two groups based on their ART scores. The general pattern of results was that the high ART group showed shorter reaction times and more accurate responses than did the low ART group. More specifically, in a lexical decision task (LDT), the magnitude of the word frequency effect was smaller in the high than in the low ART group, indicating that participants with higher levels of print exposure tended to have more efficient word recognition processes compared with participants with lower levels of print exposure—a phenomenon referred to as the lexical entrenchment effect (Diependaele, Lemhöfer, & Brysbaert, 2013) (for similar effects using eyetracking during reading, see Moore & Gordon, 2015). Lowder and Gordon (2017) reported that effects of lexical repetition during natural reading were modulated by ART scores such that the repetition effect decreased as ART scores became higher. This result indicates that higher levels of print exposure, as measured by the ART, are associated with efficient word recognition processes during natural reading. In addition, Choi et al. (2015) tested participants on a battery of individual differences measures to examine what factors best account for variability in the size of the perceptual span during reading. They showed that the size of readers’ perceptual span was best predicted by a composite score of language ability in which ART scores were included as one of the measures, thus supporting the idea that variation in print exposure is associated with various aspects of online language processing. In contrast, Acheson et al. (2008) failed to find any reliable relationship between scores on the ART and online measures of sentence processing, as measured by a self-paced reading task, as well as offline comprehension question accuracy.

In this study, we developed the KART as an objective measure of print exposure for speakers of Korean. We then compared scores on the KART with scores on a vocabulary task and a comprehension task as offline measures of language ability, and we compared scores on the KART with performance on a LDT as an online measure of word recognition processes. If the KART is an effective measure of print exposure, we should see significant relationships between scores on this measure and scores on offline measures of language ability, as well as online measures of word recognition.

Construction of a KART

Several decisions had to be made about which authors would be eligible to appear on the KART. Before selecting author names, we agreed on several criteria. First, the authors of foreign books (translated into Korean) were eligible to be included on the list. Chen and Fang (2015) excluded authors of translated books because there was no united notation of translated names in Taiwan. As phonetic notation of foreign language is standardised in Korea, authors of translated books were included on the list with no distinction between Korean authors and foreign language authors. Second, although it would have been desirable to include authors from various genres in equal proportions, several of the authors were ambiguous between more than one genre, which made it difficult to represent all genres equally. Therefore, this was not used as a criterion for selecting authors. Finally, in Chen and Fang’s (2015) Chinese version of the Author Recognition Test (CART), they included two types of author lists: CART-popular and CART-highbrow. CART-popular was a list of authors of books that were popular choices at college libraries and were top sellers at large bookstores. CART-highbrow was a list of authors recommended by avid readers. However, Chen and Fang showed that the relationship between language ability and scores on the CART did not differ across the two lists. We therefore decided to create only one list of authors for the KART.

An initial list of 275 authors was selected by searching for popular books from two libraries and three big online bookstores:

The 100 most borrowed books from Seoul National University library in 2016;

The 100 most borrowed books from GIST (Gwangju Institute of Science and Technology) college library in 2016;

The top 100 bestselling books from Kyobobook in 2016;

The top 100 bestselling books from Aladin in 2016;

The top 100 bestselling books from Yes24 for 18 months before January 5, 2017.

Out of the original 275 authors, we excluded authors of textbooks, books for the TOEIC/TOFEL, comics, recipe books, baby books, and various kinds of self-help and reference books. This left 126 authors remaining. A pilot test was conducted to identify the authors who showed significant variance in the rate of correct responses. Specifically, we added four foils to the list (the names of four Korean idols), and the full list of 130 names was presented to 186 subjects via a Google Docs survey. The participants were instructed to mark the names they recognised as being authors. Authors with a hit rate of less than 10% or greater than 80% were excluded. Using these criteria, our final list contained 40 author names. Of these 40 authors, 17 were Korean, five were Japanese, and 18 were from western countries. In addition, 40 non-author names were added to the list as foils. These foils were created by matching nationality with the real author names. Otherwise, the names were completely made up, based on our knowledge of valid names in Korea, Japan, and western countries. The proportion of foreign names was equal across the real authors and foils. Participants who complete the KART are instructed to (a) indicate the names they recognise as real authors and (b) indicate how the name is known. For this second question, participants must choose one of the following three choices: (a) I have heard the author’s name, but have never read his or her book(s); (b) I started reading his or her book(s), but never finished; or (c) I have read his or her book(s) before. The KART is presented in Supplementary Material A.

Method

Subjects

A total of 105 undergraduate and graduate students (47 females) from GIST (Gwangju Institute of Science and Technology, Gwangju, South Korea) participated in this study. They were all native speakers of Korean, and their ages ranged from 18 to 26 years. One participant did not finish the experiment, leaving data from 104 participants for the final analysis.

Materials

For the LDT, 120 Korean words were selected from a standard Korean language dictionary (National Institute of Korean Language, 2000). The subjective familiarity of each word was assessed in a rating task. The 120 words were presented to 31 participants via Google Docs, and participants rated each word on a Likert-type scale ranging from 1 to 7 (1 = I have never seen, heard, or used this word before; 7 = I see, hear, or use this word almost every day). The mean familiarity across all words was 6.0 (standard deviation [SD]: 0.8),¹ and the mean length was 2.1 syllables (range: 1-4 syllables).

One hundred and twenty Korean nonwords were generated by changing a letter in each of the words. For example, the nonword, “징문,” was created by changing the letter “ㄹ” to “ㅇ” from the Korean word, “질문.” The mean length of the nonwords was 2.1 syllables (range: 1-4 syllables).

Individual difference tasks

Reading/writing time estimates

In this questionnaire, participants were asked to estimate how much time they typically spend reading and writing in a given week. Participants checked the option that corresponded with their estimate in 1-hr increments up to 7 hr. Specifically, we translated the “Reading Time Estimates” and “Writing Time Estimates” used in Acheson et al. (2008) into Korean. In estimating reading time, participants were told that they could include time spent reading ebooks and webtoons (webtoons are a type of online Korean cartoon that is popular among undergraduates). This measure is presented in Supplementary Material B.

CRH

CRH is a self-report questionnaire asking participants to compare their own reading habits with the habits of their peers, using a Likert-type scale ranging from 1 to 7. Higher numbers indicate better reading habits, as judged by each participant. The CRH consists of five items: amount of reading, complexity of reading material, reading enjoyment, reading speed, and reading comprehension ability. We translated the CRH used in Acheson et al. (2008) into Korean, except that the item about amount of time spent reading was edited to ask about how much more material participants estimated they read, compared with their peers. This measure is presented in Supplementary Material C.

Vocabulary task

To develop a vocabulary measure, we first ran a pilot test using 80 items with four response options. Items were designed to measure participants’ general knowledge of word meanings, as well as knowledge about spelling, proverbs, and idioms. Items were based on a previously published vocabulary test book (E. Lee et al., 2014). Nine subjects participated in the pilot test to help us identify items that were too easy or too difficult. Out of the original 80 items, 20 were excluded based on results of the pilot test, leaving a total of 60 items, also with four response options, for the vocabulary test. This measure is presented in Supplementary Material D.

Comprehension task

Five texts were chosen for the comprehension task. Two of them were selected from a middle school textbook of Korean language (Shin, 2015; Yeom, 2015), and the other three texts were extracted from three different non-fiction books (Gescheider, 1985; J. Lee, 2015; N. Lee, 2014). Each text was approximately one page long. Four comprehension questions were written for each text, for a total of 20 comprehension questions. The questions asked about factual information, but also required readers to draw inferences. Each question was presented in multiple choice format with four options.

It is important to note that comprehension questions can assess different levels of text understanding. Acheson et al. (2008) assessed sentence-level comprehension using yes/no questions. In contrast, Chen and Fang (2015) assessed more global text-level comprehension, asking about both literal and inferential aspects of the text. In this study, we also ask factual and inferential questions; however, our passages were longer than the vignettes used in Chen and Fang’s (2015) comprehension task.

Procedure

After providing informed consent, participants completed the LDT. They were instructed to decide whether each letter string presented on the screen was a word or not, pressing the “m” key on the keyboard for words and the “z” key for nonwords as rapidly as possible without sacrificing response accuracy. The LDT began with 10 practice trials, consisting of five words and five nonwords. After this practice block, the remaining 240 trials (120 words and 120 nonwords) were presented in a random order.

After completing the LDT, participants were given the KART. In the KART, participants were asked to check the names they thought were authors and to indicate how they were familiar with each name that they checked (see Supplementary Material A). After completing the KART, participants were given the CRH questionnaire, as well as the questionnaire for reading/writing time estimates. Finally, participants completed the vocabulary task and the comprehension task. The experimental session lasted approximately 45 min.

Results

Analyses

Descriptive statistics of all task measures are shown in Table 1. As shown in Table 1, KART scores were calculated in three ways. First, the conventional KART score was calculated by subtracting the number of marked foils from the number of marked real authors. A KART score reflecting primary print knowledge (KART.PPK) was obtained by calculating the number of author names for which participants indicated that they had read that author’s books and dividing this number by the total number of authors (i.e., 40). Finally, a KART score reflecting secondary print knowledge (KART.SPK) was obtained by calculating the number of author names for which participants indicated that they had not read that author’s books and dividing this number by 40. For all analyses, KART.PPK and KART.SPK were weaker predictors than the composite KART score. Accordingly, the following analyses report only the conventional KART score.

Table 1.

Means, SDs, and ranges of each measure.

	Mean (SD)	Range
KART.C	17.5 (6.7)	4–35
KART.PPK	0.21 (0.13)	0.00–0.55
KART.SPK	0.18 (0.09)	0.00–0.48
Self.R (hr)	1.6 (0.6)	0.3–2.8
Self.W (hr)	1.3 (0.7)	0.0–3.4
CRH.A	3.1 (1.5)	1–7
CRH.C	3.2 (1.5)	1–7
CRH.E	3.7 (1.7)	1–7
CRH.S	3.8 (1.8)	1–7
CRH.U	4.2 (1.3)	1–7
CRH.Ave	3.6 (1.1)	1–7
Voca.S	34.1 (3.9)	25–44
Comp.S	16.4 (2.1)	9–20
LDT.RT (ms)	705 (291)	317–6,353
LDT.ACC	0.91 (0.05)	0.77–0.99

SD: standard deviation; KART.C: Korean Author Recognition Test score (conventional score); KART.PPK: Korean Author Recognition Test–primary print knowledge; KART.SPK: Korean Author Recognition Test–secondary print knowledge; Self.R: Self-report questionnaire (reading time estimates); Self.W: self-report questionnaire (writing time estimates); CRH.A: Comparative Reading Habit (amounts of reading); CRH.C: Comparative Reading Habit (complexity of reading materials); CRH.E: Comparative Reading Habit (reading enjoyment); CRH.S: Comparative Reading Habit (reading speed); CRH.U: Comparative Reading Habit (reading comprehension ability); CRH.Ave: Comparative Reading Habit (average value); Voca.S: vocabulary test score; Comp.S: comprehension test score; LDT.RT: lexical decision task reaction time; LDT.ACC: lexical decision task accuracy.

KART.PPK and KART.SPK are presented as proportions as described in the text.

The reaction time data from the LDT were analysed using linear mixed effects (LME) models with the lmer function in the lme4 package (Bates, Maechler, & Bolker, 2012) in R (R Development Core Team, 2011), with subjects and items entered as crossed random effects. The lmerTest package (Kuznetsova, Brockhoff, & Christensen, 2017) was used to obtain all p values. To examine the relationship between KART and online processes of word recognition, fixed effects included word familiarity, KART score, and the interactions among the two factors. Both fixed effects were continuous variables. Random effects included intercepts for subjects and items. By-subject and by-item random slopes were not included in the analyses because the models including them failed to converge.

The remainder of this section is organised into two parts. First, we describe the pattern of correlations between KART and the other offline tasks. Then, we report analyses designed to investigate how KART relates to online word recognition processes.

Correlations between KART and other measures

Figure 1 shows correlation coefficients and scatter plots for KART, the offline individual difference tasks, and LDT accuracy. In addition, the diagonal of Figure 1 shows the intraclass correlation coefficients (ICC) of each task, which is a useful metric for assessing the reliability of each measure (see Bartko, 1966; Shrout & Fleiss, 1979). As indicated by the ICC values, all measures showed exceptional reliability. In particular, the KART showed reliability of .99.

Figure 1.

Correlations between KART and other measures.

As shown in Figure 1, KART scores showed modest but reliable correlations with the other offline test scores. For example, KART scores and average CRH scores were correlated at greater than .4, which is consistent with correlations between these two measures observed in previous studies (e.g., r = .30 in Chen & Fang, 2015; r = .51 in Choi et al., 2015).²

Regarding online measures of word recognition, KART scores showed a significant correlation with accuracy on the LDT, such that higher scores on the KART were associated with better word recognition performance. Given that the words used in the LDT were relatively difficult, the positive correlation between KART scores and LDT accuracy indicates that print exposure measured by the KART seems to adequately capture the difficulty of lexical access.

LDT data modulated by KART

For the LME analysis of the LDT reaction time data, incorrect trials were excluded, as well as reaction times that were shorter than 300 ms or longer than 2000 ms. In total, 10.02% of the data were excluded.

The results revealed a significant main effect of KART scores (t = −2.84, p < .005), demonstrating that higher scores on the KART were associated with faster reaction times. In addition, a significant main effect of word familiarity emerged (t = −14.73, p < .001), indicating that increases in word familiarity were associated with faster reaction times. More importantly, there was a significant interaction between KART scores and word familiarity (t = 2.83, p < .005). As shown in Figure 2, the relationship between word familiarity and reaction times was modulated by KART scores such that participants with higher KART scores showed a smaller effect of word familiarity compared with those with lower KART scores.

Figure 2.

Effects of word familiarity and KART score (split as high vs. low) on reaction time (ms) in the lexical decision task.

Because our mixed effect models analysing the effect of word familiarity and KART scores on accuracy rates failed to converge, we instead conducted a two-factor analysis of variance (ANOVA) to examine the relationship. For this analysis, KART scores and word familiarity were both divided into two levels based on a median-split method. Error variance was calculated by participants (F₁) and by items (F₂). As expected, the main effect of KART was statistically significant such that the participants with higher KART scores were more accurate on the task than the participants with lower KART scores, F₁(1, 202) = 11.39, p < .001; F₂(1, 234) = 4.06, p < .05. In addition, a main effect of word familiarity emerged in the subject (F₁) analysis and was marginally significant in the item analysis (F₂) such that the more familiar word condition showed higher accuracy rates than did the less familiar word condition, F₁(1, 202) = 64.68, p < .0001; F₂(1, 234) = 2.98, p < .09. Interestingly, there was a significant interaction between KART scores and word familiarity, F₁(1, 202) = 13.09, p < .001; F₂(1, 234) = 4.49, p < .05. The source of the interaction was such that accuracy rates for the familiar word condition were not different between the two KART groups (98.3% in the high KART group vs. 98.5% in the low KART group), whereas accuracy rates for the unfamiliar word condition were higher in the high KART group (86.6%) than in the low KART group (80%). This result indicates that the relationship between word familiarity and accuracy rates in the lexical decision depends critically on KART scores.

Discussion

In this study, we created a KART to examine the relationship between print exposure and language abilities among young adults whose native language is Korean. To this end, we examined correlations between KART scores and offline tasks measuring individual differences in language ability, as well as online measures of word recognition. The results showed that (a) there were moderate correlations between KART scores and other offline measures of language ability and (b) the effects of word familiarity in a LDT were modulated by KART scores.

With respect to the first point, KART scores showed significant correlations with vocabulary knowledge scores, as well as reading comprehension scores. These results are consistent with previous findings demonstrating moderate-to-strong correlations between ART scores and vocabulary knowledge among college students, assessed using a variety of vocabulary tests (e.g., Burt & Fury, 2000; Grant, Wilson, & Gottardo, 2007; Stanovich & Cunningham, 1992; Stanovich, West, & Harrison, 1995; West & Stanovich, 1991). Previous studies have also shown significant correlations between ART scores and scores on a variety of language comprehension measures (e.g., Burt & Fury, 2000; Grant et al., 2007; Martin-Chang & Gould, 2008; Osana, Lacroix, Tucker, Idan, & Jabbour, 2007; Stanovich & Cunningham, 1992, 1993; Stanovich & West, 1989).

Beyond these results using native speakers of English, Chen and Fang (2015) reported significant correlations between ART scores and measures of vocabulary and reading comprehension among native speakers of Chinese. Similarly, Rodrigo, McQuillan, and Krashen (1996), using a Spanish version of the ART, reported a significant correlation between print exposure and vocabulary among native speakers of Spanish. Vander Beken and Brysbaert (2018) recently developed a Dutch version of the ART, which showed significant correlations with scores on vocabulary and spelling tests among native speakers of Dutch. The results of this study, combined with these previous findings, indicate that measures of print exposure are likely to show robust relationships with measures of vocabulary and comprehension ability irrespective of the language under investigation. In addition, the excellent reliability of the KART (.99) is very similar to the high reliability estimate that was reported for the Dutch version of the ART (.97; Vander Beken & Brysbaert, 2018). Although other versions of the ART have not generally reported reliability estimates quite this high, they do tend to have reliabilities that are quite good: .84 (Stanovich & West, 1989), .76 (Chen & Fang, 2015), and .85 (Martin-Chang & Gould, 2008). One exception is the Spanish version of the ART, which had a reliability of only .61 (Rodrigo et al., 1996). One reason for the lower reliability for this version may be the relatively low number of author names that appeared on the test (i.e., 16 authors on the Spanish ART, compared with 40 or more on most other versions of the ART). Thus, the ART seems to be a reliable measure in general, with some versions such as the KART developed here demonstrating excellent levels of reliability.

It is important to note the pattern of relationships between KART and the other measures of print exposure used in this study. Specifically, KART scores were significantly correlated with CRH scores, whereas KART scores were not related to the self-report reading/writing habits measures, suggesting that these self-report estimates are perhaps unreliable as measures of print exposure (see also Acheson et al., 2008; Cunningham & Stanovich, 1990, 1991; Stanovich & West, 1989). As mentioned in section “Introduction,” self-report measures of the time one spends reading and writing are known to be unreliable because they are quite susceptible to responses that reflect social desirability. Another issue associated with these self-report measures is the accuracy of the responses. In the questionnaire used in this study, we asked respondents to estimate how much time they spend reading a variety of materials during a week. Even if a respondent is not intending to respond in a socially desirable manner, it is somewhat difficult to generate an accurate time estimate, which may lead to values that are either greatly underestimated or overestimated. The issues raised here might also be reasons that these self-report measures did not show any relationships with any of the other measures including offline tasks of vocabulary and reading comprehension, or performance on the LDT. Another important finding of this work is that KART scores modulated the relationship between word familiarity and word recognition time such that readers with higher KART scores showed smaller differences in reaction times between familiar and unfamiliar words compared with those with lower KART scores. This result is consistent with Chateau and Jared (2000), who showed a smaller word frequency effect in an LDT for their high ART group compared with their low ART group. These results indicate that readers with greater amounts of print exposure engage in quicker and more efficient word recognition processing compared with readers with lower amounts of print exposure. These similarities between English and Korean results suggest that the (K)ART is not only a good measure for estimating print exposure but also a useful measure for explaining individual differences in processes of word recognition.

Similar relationships between print exposure and online language processes have also been found by Lowder and Gordon (2017). Lowder and Gordon recorded adult readers’ eye movements while they read sentences. They showed that the word repetition effect (measured by fixation durations) was modulated by ART scores such that the size of the effect became smaller as a reader’s ART score increased. This result indicates that readers with higher ART scores tend to process words more effectively during reading compared with those with lower ART scores. However, Acheson et al. (2008) did not find significant correlations between ART and word reading time or comprehension question accuracy in a self-paced reading task. Thus, the relationship between ART scores and sentence-level online processing is a bit mixed, even in English. Further research is needed to clarify the relations between print exposure and online language comprehension at all levels of processing.

Notably, some previous work has suggested that PPK is a better predictor of language abilities than conventional ART scores (Chen & Fang, 2015; Martin-Chang & Gould, 2008), whereas conventional KART scores in this study were a substantially better measure than either PPK or SPK. There are several possible reasons for this discrepancy. First, there may be important cultural differences between the Korean sample used in this study and the Chinese and American samples used in previous experiments. Our own intuitions are that Korean students do not read many books, but rather they read other texts that contain information about authors and their books, as when they study to prepare for college entrance exams. Supporting this point, Korean participants in the current experiment indicated that they had read books by the authors only about eight times, on average, despite knowing an average of 17 authors. Thus, it seems possible that Korean participants were more likely to encounter these names by reading other types of materials than reading the books themselves—a pattern that may be different in other cultures. Second, it should be noted that PPK is not as well established a measure as the conventional ART score, and in fact many experiments using the ART do not ask participants to indicate how they know the author names. As so few experiments have separated ART scores into PPK and SPK, we cannot determine at this point whether the pattern we have observed here will turn out to be the norm, or whether PPK may turn out to be a more useful measure of print exposure than the conventional ART score. Finally, it is possible that PPK scores reflect, at least in part, some aspect of socially desirable responding, such that participants are motivated to report that they have personally read books by the author even when they have not. To the extent that this is the case, it undermines the purpose of the ART as an objective measure of print exposure.

In sum, this study is the first to develop KART. We found that native Korean-speaking college students’ levels of print exposure, as measured by the KART, were related to online processes involved in word recognition, as well as with offline reading-related tasks. Given the relationships between ART scores and language ability found in this study, along with previous studies in other languages, this measure seems to be a good indicator of an individual’s print exposure, which in turn is related to a variety of language processing skills.

Supplemental Material

QJE-STD-18-253.R2-Supplementary_Material – Supplemental material for Development and assessment of the Korean Author Recognition Test

Supplemental material, QJE-STD-18-253.R2-Supplementary_Material for Development and assessment of the Korean Author Recognition Test by Hyosun Lee, Eunjin Seong, Wonil Choi and Matthew W Lowder in Quarterly Journal of Experimental Psychology

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This research was supported in part by a grant from the National Research Foundation of Korea (NRF-2017R1C1B5014973) and by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF- 2017S1A3A2066319).

Supplemental material

The Supplemental material is available at

Notes

References

Acheson

D. J.

Wells

J. B.

MacDonald

M. C.

(2008). New and updated tests of print exposure and reading abilities in college students. Behavior Research Methods, 40, 278–289.

Bartko

J. J.

(1966). The intraclass correlation coefficient as a measure of reliability. Psychological Reports, 19, 3–11.

Bates

Maechler

Bolker

(2012). lme4: Linear mixed-effects models using S4 classes (R package version 0.999999-0). Available from https://cran.r-project.org/

Burt

J. S.

Fury

M. B.

(2000). Spelling in adults: The role of reading skills and experience. Reading and Writing, 13, 1–30.

Chall

J. S.

(1983). Stages of reading development. New York, NY: McGraw-Hill.

Chateau

Jared

(2000). Exposure to print and word recognition processes. Memory & Cognition, 28, 143–153.

Chen

S. Y.

Fang

S. P.

(2015). Developing a Chinese version of an Author Recognition Test for college students in Taiwan. Journal of Research in Reading, 38, 344–360.

Choi

Lowder

M. W.

Ferreira

Henderson

J. M.

(2015). Individual differences in the perceptual span during reading: Evidence from the moving window technique. Attention, Perception, & Psychophysics, 77, 2463–2475.

Cunningham

A. E.

Stanovich

K. E.

(1990). Assessing print exposure and orthographic processing skill in children: A quick measure of reading experience. Journal of Educational Psychology, 82, 733–740.

10.

Cunningham

A. E.

Stanovich

K. E.

(1991). Tracking the unique effects of print exposure in children: Associations with vocabulary, general knowledge, and spelling. Journal of Educational Psychology, 83, 264–274.

11.

Diependaele

Lemhöfer

Brysbaert

(2013). The word frequency effect in first-and second-language word recognition: A lexical entrenchment account. The Quarterly Journal of Experimental Psychology, 66, 843–863.

12.

Ennis

P. H.

(1965, September). Adult book reading in the United States (National Opinion Research Center Report No.105). Chicago, IL: University of Chicago Press.

13.

Fischer

F. W.

Shankweiler

Liberman

I. Y.

(1985). Spelling proficiency and sensitivity to word structure. Journal of Memory and Language, 24, 423–441.

14.

Fletcher

K. L.

Reese

(2005). Picture book reading with young children: A conceptual framework. Developmental Review, 25, 64–103.

15.

Gernsbacher

M. A.

(1984). Resolving 20 years of inconsistent interactions between lexical familiarity and orthography, concreteness, and polysemy. Journal of Experimental Psychology: General, 113, 256–281.

16.

Gescheider

G. A.

(1985). Psychophysics: Method and theory ( Lee

, Trans.). Seoul, Korea: Yangyounggak.

17.

Grant

Wilson

A. M.

Gottardo

(2007). The role of print exposure in reading skills of postsecondary students with and without reading disabilities. Exceptionality Education International, 17, 175–194.

18.

Greaney

(1980). Factors related to amount and type of leisure time reading. Reading Research Quarterly, 15, 337–357.

19.

Hirsch

E. D.

(2003). Reading comprehension requires knowledge of words and the world. American Educator, 27(1), 10–13.

20.

Jastak

Wilkinson

G. S.

(1984). The wide range achievement test—Revised. Wilmington, DE: Jastak.

21.

Kang

B. M.

Kim

H. K.

(2009). Token frequency of Korean: Analyze of 15 million words in Sejong corpus. Seoul, Korea: Hankookmunhwasa.

22.

Kuznetsova

Brockhoff

P. B.

Christensen

R. H. B.

(2017). lmerTest Package: Tests in linear mixed effects models. Journal of Statistical Software, 82(13), 1–26.

23.

Lee

Kim

Kang

Part

Yang

Cho

(2014). Korean vocabulary for middle school students 3. Seoul, Korea: Erumenb.

24.

Lee

(2015). Human rights law. Seoul, Korea: Hongmoonsa.

25.

Lee

(2014). The history of feminism and the appearance of the gender concept. In Korean Women Studies Institute (Ed.), Gender and society (pp. 213–214). Gyeonggi-do, Korea: Dongnyok.

26.

Lowder

M. W.

Gordon

P. C.

(2017). Print exposure modulates the effects of repetition priming during sentence reading. Psychonomic Bulletin & Review, 24, 1935–1942.

27.

Martin-Chang

S. L.

Gould

O. N.

(2008). Revisiting print exposure: Exploring differential links to vocabulary, comprehension and reading rate. Journal of Research in Reading, 31, 273–284.

28.

Mol

S. E.

Bus

A. G.

(2011). To read or not to read: A meta-analysis of print exposure from infancy to early adulthood. Psychological Bulletin, 137, 267–296.

29.

Moore

Gordon

P. C.

(2015). Reading ability and print exposure: Item response theory analysis of the Author Recognition Test. Behavior Research Methods, 47, 1095–1109.

30.

National Institute of Korean Language. (2000). Standard Korean language dictionary. Seoul, Korea: Doosan Dong-A Co., Ltd.

31.

Osana

H. P.

Lacroix

G. L.

Tucker

B. J.

Idan

Jabbour

G. W.

(2007). The impact of print exposure quality and inference construction on syllogistic reasoning. Journal of Educational Psychology, 99, 888–902.

32.

Paris

S. G.

(2005). Reinterpreting the development of reading skills. Reading Research Quarterly, 40, 184–202.

33.

Park

(2003). Subjective frequency estimates of Korean words and frequency effect on word recognition. Korean Journal of Experimental Psychology, 15, 349–366.

34.

Paulhus

D. L.

(1984). Two-component models of socially desirable responding. Journal of Personality and Social Psychology, 46, 598–609.

35.

R Development Core Team. (2011). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Available from http://www.r-project.org/

36.

Rodrigo

McQuillan

Krashen

(1996). Free voluntary reading and vocabulary knowledge in native speakers of Spanish. Perceptual and Motor Skills, 83, 648–650.

37.

Shin

(2015). Eagle owl, the emperor of night. In Bang

Hwang

Shin

Kim

Seo

Lee

S-J.

(Eds.), Korean language 5 (pp. 213–214). Seoul, Korea: Jihak Publishing Company.

38.

Shrout

P. E.

Fleiss

J. L.

(1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.

39.

Stanovich

K. E.

Cunningham

A. E.

(1992). Studying the consequences of literacy within a literate society: The cognitive correlates of print exposure. Memory & Cognition, 20, 51–68.

40.

Stanovich

K. E.

Cunningham

A. E.

(1993). Where does knowledge come from? Specific associations between print exposure and information acquisition. Journal of Educational Psychology, 85, 211–229.

41.

Stanovich

K. E.

West

R. F.

(1989). Exposure to print and orthographic processing. Reading Research Quarterly, 24, 402–433.

42.

Stanovich

K. E.

West

R. F.

Harrison

M. R.

(1995). Knowledge growth and maintenance across the life span: The role of print exposure. Developmental Psychology, 31, 811–826.

43.

Vander Beken

Brysbaert

. (2018). Studying texts in a second language: The importance of test type. Bilingualism: Language and Cognition, 21, 1062–1074.

44.

Vellutino

F. R.

Tunmer

W. E.

Jaccard

J. J.

Chen

(2007). Components of reading ability: Multivariate evidence for a convergent skills model of reading development. Scientific Studies of Reading, 11, 3–32.

45.

West

R. F.

Stanovich

K. E.

(1991). The incidental acquisition of information from reading. Psychological Science, 2, 325–330.

46.

West

R. F.

Stanovich

K. E.

Mitchell

H. R.

(1993). Reading in the real world and its correlates. Reading Research Quarterly, 28, 35–50.

47.

Yeom

(2015). “Guernica,” the masterpiece of Picasso. In Bang

Hwang

Shin

Kim

Seo

Lee

S-J.

(Eds.), Korean language 5 (pp. 202–203). Seoul, Korea: Jihak Publishing Company.

48.

Zill

Winglee

(1990). Who reads literature? The future of the United States as a nation of readers. Cabin John, MD: Seven Locks Press.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.05 MB