Critical language awareness for the heritage context: Development and validation of a measurement questionnaire

Abstract

Critical language awareness (CLA) is increasingly identified as a central component of the Spanish heritage language (SHL) classroom (Leeman, 2005; Martínez, 2003; among others). As a minority language, SHL is subject to sociopolitical, cultural, and economic forces that devalue its status. It is devalued in the eyes of the public, as a legitimate U.S. language, and as something worthy of being maintained. It is essential that students receive instruction not only in the heritage language, but also on the contextual factors that affect the Spanish-English sociopolitical relationship in the United States. Such instruction will enable learners to begin resisting heritage language loss by questioning language ideologies that promote English monolingualism and standard, monolingual Spanish as the ideal norm. The aim of this study was to develop a questionnaire with adequate psychometric properties to measure CLA in the SHL context. The respondents were 301 students enrolled in SHL courses in four U.S. universities. The questionnaire results were submitted to a series of statistical analysis to investigate how well the instrument meets the criteria of reliability and validity specified in this study. The final 19-item instrument had adequate psychometric properties and detected change in the CLA of students in a class where CLA was taught.

Keywords

Critical language awareness heritage language assessment heritage language instruction language ideologies Spanish as a heritage language

The goal of maintaining and promoting heritage languages is more critical than ever in educational environments in the United States. A recent report from the Commission on Language Learning (2017) identified the education of heritage language (HL) learners as one of the five key goals for language instruction in the United States in the twenty-first century, in recognition of the prospective benefits not only for heritage learners themselves, but also for building the multilingual capacity of the nation as a whole. At the same time, however, because heritage languages are minority languages in the United States, their maintenance is compromised by dominant societal language ideologies that favor English monolingualism over multilingualism, monolingual varieties over bilingual ones, and educated varieties over vernacular ones. These ideologies have been historically reproduced through education and other mainstream avenues and, as a consequence, students of these languages may enter a HL course with deep-seated beliefs about the inferiority of their language, their linguistic variety, and their level of proficiency (Beaudrie & Ducar, 2005; Bernal-Henríquez & Hernández-Chávez, 2003; Martínez, 2003).

In recent years, HL educators have responded with a call to reconfigure the content of instruction so as to make students’ varieties more central and to develop students’ critical language awareness (CLA) (Beaudrie, 2015; Leeman, 2005; Leeman & Serafini, 2016; Martínez, 2003; among others). CLA is an approach originally developed in the 1990s by researchers in the United Kingdom, who argued that it was essential to include discussions of power issues as part of language and literacy instruction. Clark, Fairclough, Ivanič and Martin-Jones (1990, 1991), in a two-part article, presented CLA as an improvement over language awareness (LA) programs. The authors described CLA as seeking to provide “operational and descriptive knowledge of the linguistic practices of their world, but also a critical awareness of how these practices are shaped by, and shape, social relationships of power” (Clark et al., 1990, p. 52). Whereas LA programs in the European context had sought to promote learners’ social integration into the dominant social order by teaching the properties of language and language use, the goal of CLA was to achieve social emancipation (Clark et al., 1990). Most importantly, the primary objective of CLA is to "empower learners by providing them with a critical analytical framework to help them reflect on their own language experiences and practices, the language practices of others in the institutions of which they are a part and in the wider society within which they live" (Clark and Ivanič, 1997, p. 217). In addition, the CLA approach promotes equality both inside and outside the classroom, and nurtures students’ agency to make their own linguistic choices (Leeman, 2014). After Fairclough (1989) first developed CLA in Britain, it quickly expanded across the globe (Granville, 2003).

Recently, several instructional proposals to develop students’ CLA in the classroom have begun to emerge. However, one of the key gaps that remains unexplored is how to measure students’ CLA reliably in order to determine the effectiveness of such instructional programs. This study contributes to the current literature by describing the design and validation of an instrument to measure CLA in the HL context.

Literature review

Academic discussions around the need to include critical approaches in the classroom have a relatively long history in the English language context (Godley & Minnici, 2008). The continued devaluation of African American Vernacular English (AAVE) in particular has attracted the attention of many researchers and educators who seek to reclaim the value of this variety and its speakers. Wolfram (1998) highlighted the groundbreaking impact of the Oakland Ebonics controversy¹ on discussions of dialect and bidialectal appreciation. Wolfram also advocated that educators must have sociolinguistic awareness and, in particular, be informed about the language patterns their students use. As Ali (2005) argued in his review of the history of Black language and education in the United States, CLA can empower students to challenge the attitudes that mark their way of speaking as deficient. As many linguists have previously pointed out, the varieties spoken by disempowered groups have historically been disparaged and institutionally disenfranchised (e.g., Native Americans, Latinos, African Americans, and Hawai’ians). These minority speakers have often faced physical or psychological punishment for using their home language or variety. For instance, AAVE and other nonstandard varieties of English have been targeted for eradication and replacement with prestige varieties. The study of instructional practices to promote CLA has, however, received only minimal attention from educational researchers (Godley & Minnici, 2008). Crucially for the present study, no researchers have developed reliable, well-validated measures to gauge the instructional effectiveness of the CLA components of their curricula.

Reaser (2006) conducted one of the first studies that attempted to measure the effectiveness of an instructional module on language awareness with 129 students enrolled in ninth-grade, honors-level, Language Arts classes in a high school in North Carolina. The researcher developed a language attitude survey and administered it before and after the implementation of a 450-minute curriculum about dialect diversity. The survey was constructed to ask participants to respond to, for example, the kinds of statements about language that non-linguists commonly make. The results showed that students were more sensitive to language variation after the instruction and that, for most of the survey items, they provided more tolerant or more informed responses. The 20-item survey was not submitted to any type of statistical analysis to measure its psychometric properties, however.

Similarly, for a first-year English composition course in the United States, Murphy (2012) designed and implemented an experimental dialect awareness unit adapted from a range of sources, including Reaser and Wolfram (2007), Do You Speak American? (2005), Groome and Gibbs (2004), among others, together with instructional goals taken from Gooden (2007). Murphy sought to determine whether the explicit teaching of dialect awareness can “provide a successful counter to pervasive negative attitudes towards stigmatized dialects” (p. 1). This educational intervention introduced students to the concepts of language change, language variation, stereotypes, dialect discrimination, dominant language ideologies, and code-switching. Both control and experimental groups responded to an identical pre- and post-questionnaire that consisted of Likert items to measure whether their language awareness and tolerance for linguistic diversity had increased in each of the courses. Statistical analysis was used to determine whether students’ attitudes toward stigmatized dialects changed. Additional open-ended responses provided qualitative data about changing attitudes. The results showed marked increases in the experimental group regarding tolerance and knowledge of nonstandard dialects. Unfortunately, the instrument used was not adequately validated before use.

Several classroom studies have been conducted using qualitative assessment rather than quantitative measures to determine the effectiveness of CLA instruction. Godley and Minnici (2008) conducted one such study. Thirty-one tenth graders, who self-identified as AAVE speakers, partook in a weeklong unit where they discussed language variety and dialects. In particular, the students focused on identifying and critiquing dominant language ideologies through dialogues in which they were recognized as experts on AAVE. Data collection consisted of video- and audio-taping, observations and field notes, and written student reflections. The researchers concluded that the weeklong unit exposed students to the stigmatization of AAVE and especially why this stigmatization must be challenged. Similar classroom studies using qualitative methods include Janks (1999), who evaluated student journals for CLA; Ali (2011), who had students analyze an article from a well-known newspaper; Huang (2013), who explored CLA development through writing; and Granville (2003), who conducted an action research project.

In the context of HL in the United States, primarily for SHL, several projects have been designed to expose HL learners to CLA and to begin deconstructing ideologies that promote either English monolingualism or standard, monolingual Spanish in order to resist and reverse HL shift. Bernal-Henríquez and Hernández-Chávez (2003) highlighted the widespread negative ideology about heritage learners’ Spanish bilingual varieties. The authors explained that non-prestigious language features of heritage learners are often perceived as “barbarisms” (see also Villa, 1996, 2002). In the 1970s, textbooks for bilinguals were filled with grammar drills and focused exclusively on the teaching of a prestige variety through mechanical contrastive exercises (Valdés, 1995).

One of the earliest proposals was introduced by Martínez (2003), who argued for introducing CLA (which he called Classroom Based Dialect Awareness) to HL students at beginning levels, rather than only during advanced courses. In this seminal article, he also presented the idea of utilizing activities and games such as “euphemisms” or “nicknames” to promote CLA and, in particular, to “highlight the relationship between language, power, and social groups” (Martínez, 2003, p. 49). For the framework underlying these activities, he expanded on the three original principles proposed by Wolfram (1998): (1) dialects are natural; (2) dialects are regular; and (3) variation occurs on different levels. The latter added a social dimension with three additional elements related to dialect function, distribution, and evolution. Because many students bring to the classroom understandings and experiences of language subordination and variation, activities centered on CLA empower them to make their own choices about language use in their individual environments. Martínez’s article, though important, lacked any insights into how to measure students’ CLA in the classroom.

Leeman, Rabin, and Román-Mendoza (2011) contributed another proposal to develop students’ CLA. They designed a critical service-learning opportunity where students volunteered in local primary schools in an effort to build HL students’ sense of agency and promote language activism. Following a qualitative design, the researchers interviewed participants and found that 78% of the students claimed that this service-learning opportunity made them aware of their agency to promote HL maintenance and of their potential for activism for social change. In particular, students became aware of language ideologies that hinder Spanish maintenance and the reasons why people become language activists. Similar studies involving students in community service-learning opportunities followed (i.e., Lowther-Pereira, 2015; Martínez, 2010; Martínez & Schwartz, 2012; Moreno & MacGregor, 2016; Pascual y Cabo, Prada, & Lowther-Pereira, 2017).

Parra (2013) described an advanced language course, “Spanish and the Community,” that was designed to promote translingual and transcultural competence, critical thinking skills, and CLA. Addressing topics such as immigration, identity, and language practices, students explored a variety of genres grounded in sociocultural contexts (such as readings, film, music, and art) and highlighting a multiliteracy approach to language development. A unique aspect of her approach was the incorporation of art into course assignments as a tool to explore these connections. To foster their connections to the community, students participated in a service-learning component designed to deepen their understanding of and connection to the Latino community. Students successfully developed their translinguistic and transcultural competence, as reflected in a final written essay and oral presentation; CLA development was not formally evaluated. Subsequent articles presented similar approaches with courses designed to provide students with sociolinguistic awareness that develops their understanding of English–Spanish language use in the United States (see Velásquez, 2013; Shin & Hudgens Henderson, 2017). Leeman and Serafini (2016) presented probably the most complete proposal for how to incorporate CLA into the HL classroom to develop students’ “understanding of the social, cultural, and political aspects of language, language variation, and multilingualism” (p. 71). Following an in-depth explanation of the fundamental sociolinguistic concepts necessary to implement a CLA approach, they provided concrete classroom activities and assignments that developed students’ awareness of multilingualism, language variation, multilingual discourse and code-switching, and language ideologies.

Holguín Mendoza’s (2017) study was the only one in the Spanish heritage context that examined the impact of implementing a CLA approach in Spanish HL courses using a quantitative approach. She utilized a linguistic attitudinal survey containing 30 sentences that incorporated stigmatized words of Mexican Spanish used in Spanish spoken in the United States. To complete the survey, students marked whether they considered a sentence correct or incorrect, and whether they would or would not use these specific sentences. The survey was presented at the beginning and at the end of the term. The results showed that students as a group increased their sociolinguistic awareness but, as the author pointed out, this type of attitudinal survey may not be the best tool to fully evaluate progress in CLA after an instructional intervention.

As this review shows, several recent studies have contributed to defining the construct of CLA by describing instructional programs to teach CLA in various contexts, but few studies have sought to study empirically the results of teaching CLA in either the English or the HL context. None of these studies have sought to develop a psychometrically sound instrument to measure CLA. Empirical research is needed first to examine how to assess CLA reliably, and then to measure the effectiveness of CLA instruction in HL contexts. Although a few qualitative or quantitative measures have been used in the past, the present study is the first of its kind in that it is an attempt to design, validate, and implement a questionnaire that has sound psychometric properties to measure this construct in the heritage context.

Study

The purpose of the present study was to design and validate a questionnaire to measure CLA in the HL context. We followed a systematic, seven-step design and validation process for the development of the instrument in order to ensure that the validity and reliability criteria set for this study were met to produce a psychometrically sound instrument. This process follows recommended practices in social sciences (Artino, La Rochelle, Dezee, & Gehlbach, 2014; Dörnyei, 2007). A summary of the steps followed for the development and validation process is presented in Table 1, and each of the seven steps are discussed in detail below.

Table 1.

Seven-step questionnaire development and validation process.

Step 1	Construct definition
Step 2	Item development
Step 3	Consultation with two groups of naïve and experienced test-takers and content experts to ensure content validity
Step 4	Pilot test with first group of students
Step 5	Full test
Step 6	Statistical analysis Reliability: Cronbach’s alpha Validity: principal components analysis
Step 7	Questionnaire administration to measure CLA before and after CLA instruction Pre- and posttests

Questionnaire construction

Step 1: Construct definition

After a review of the relevant literature in dialect awareness and CLA, succinctly summarized above, the research team generated the key thematic areas to be included within the general construct of CLA in the HL context. The topics for the first theme, found in Table 2, address the important sociolinguistic topics of language variation and linguistic diversity as well as the intrinsic validity of all ways of speaking. The foundation of this first area is an understanding that language is inherently and naturally variable, prone to linguistic change and contact over time (Bills & Vigil, 2008; Otheguy, 1993; Otheguy & García, 1993; Otheguy & Zentella, 2012; Poplack, 1980; Silva-Corvalán, 1994; Zentella, 1982). It also challenges preconceptions students may have about the concepts of dialect and language and the superiority of one over the other (Bills, 2005; Galindo, 1995, 1996; Hidalgo, 1986; Rivera-Mills, 2000). Lastly, prior research in language ideologies specific to SHL pedagogy (Leeman, 2012; Lowther Pereira, 2010; Villa, 2002) have indicated the predominance and emphasis of the standard in university-level language education (Beaudrie, 2015) and a privileging of European varieties of Spanish (Leeman, 2010, 2012). Students have a tendency to internalize the disparaging attitudes of educators, language programs, and society toward non-dominant dialects, such as U.S. Spanish, who may view them as inappropriate for class use or instruction, and as a chaotic corruption of both English and Spanish (Bernal-Henríquez & Hernández-Chávez, 2003; Zentella, 2008).

Table 2.

Thematic areas for CLA.

Theme	Topics
1. Language variation and linguistic diversity	1. Language vs. dialect 2. Dialectal variation 3. Standard vs. nonstandard forms
2. English hegemony, language ideologies and linguistic prejudice	1. Language and power (language prejudice and subordination) 2. The privileged position of English (majority vs. minority languages) 3. Dominant language ideologies (e.g. monolingualism and the standard language)
3. Spanish in the United States, bilingualism, and code-switching	1. Characteristics of U.S. Spanish 2. Phenomena of languages in contact (calques, lexical borrowings, semantic extensions, etc.) 3. Code-switching
4. Language policies and language maintenance/shift	1. HL maintenance/loss and the benefits of bilingualism 2. Language policies in the United States 3. Developing student agency in HL maintenance.

The second set of themes in Table 2 speaks to the hegemonic role of English in the United States and, consequently, the pressures Spanish faces from a dominant U.S. culture that promotes monolingualism as the gold standard. In the first topic, “language and power,” the historical privileging of English over other immigrant languages (e.g. Spanish) is included (Schmid, 2000), in order to identify the ideological power instilled in the majority vs. minority language contrast. In particular, attention is given to the Hispanophobia prevalent in society that promotes an anti-bilingual sentiment against the U.S. Spanish-speaking communities (Zentella, 1997). Moreover, the “privileged position of English” topic covers the real-world consequences of not adhering to English proficiency as a core component of American national identity (Pavlenko, 2002, 2003). The final topic, “dominant language ideologies,” points to the institutional ideology that excludes minority languages, such as educational and public institutions (Ruiz, 1988; Wright, 2005).

The third set of items, also in Table 2, focuses on Spanish in the United States in an effort to debunk common linguistic beliefs held about U.S. Spanish and the common phenomenon associated with the umbrella term: Spanglish. A common attitude held by Spanish heritage students views the so-called mixing of English and Spanish as a deficit due to the failure to keep both languages separate. To the contrary, the first topic, “characteristics of in the U.S.,” draws from seminal works in recent decades (Poplack, 1980; Zentella, 1997, among others) that indicate that code-switching is rule-governed, serves discourse functions, and is a marker of group membership. In particular, the second topic, “phenomena of languages in contact,” focuses on the linguistic features of Spanglish and illustrates the common situation of languages in contact from around the world. Lastly, the topic, “code-switching,” focuses on an important area, as it is “the most salient, and this is frequently perceived by both insiders and outsiders as indicative of disfluency or an inability to speak only one language at a time” (Carvalho, 2012, p. 139).

Finally, the fourth theme relates to language maintenance and policies. Under the topic “HL maintenance/loss and the benefits of bilingualism,” attention is given to the combination of dominant societal pressures on Spanish as a minority language and the internal communal deficit views of U.S. Spanish that contribute to its eventual generational shift (Mejías & Anderson, 1988). The “language policies in the U.S.” topic covers the language shift toward Spanish that is a well-documented phenomenon (Bills, 2005; Bills, Hernández-Chávez & Hudson, 1995; Jenkins, 2010; MacGregor-Mendoza, 2005; Villa & Rivera-Mills, 2009) upheld by societal support for language policies that restrict the teaching of minority languages in public schools and its use in public spaces (Cashman, 2006; Wiley, 2005; Wright, 2005). The topic “developing student agency in HL maintenance” explores some preventative measures essential to mitigate language loss, while promoting active language use and advocacy of the heritage language. The four main themes selected as part of what CLA entails based on prior research are summarized in Table 2.

Step 2: Item development

Existing questionnaires were consulted, but none was fully appropriate for the heritage context (Hudgnes-Henderson, 2016; Murphy, 2012; Reaser, 2006). Therefore, the research team adapted some items from existing questionnaires, but generated the majority of the items from scratch. Items were carefully constructed to be succinct and avoid double-barreled items (Dörnyei & Taguchi, 2010), as well as to measure individuals’ attitudes and intended behaviors in relation to the topics in Table 2. The items were constructed using a Likert-type scale (6 = strongly agree or very likely to 1 = strongly disagree or very unlikely). The original item pool consisted of 48 items (12 per thematic area), which were developed and categorized into four thematic sections with three topics per theme, as indicated in Table 2:

Language variation and linguistic diversity (Example of ‘standard vs. non-standard forms’ topic)

Attitude: People from Spain speak the purest form of Spanish.

Behavior: If I were an English teacher, I would correct a student who says “I ain’t got none” in a formal context such as school.

Linguistic prejudice and language ideologies (Example of ‘dominant language ideologies’ topic)

Attitude: In my opinion, families can speak Spanish among themselves at home but when in public they should speak English to each other.

Behavior: I would not use Spanish in public, just at home or more private places.

Spanish in the United States, bilingualism, and code-switching (Example of ‘phenomena of languages in contact’ topic.

Attitude: I don’t like it when people say “parquear” instead of “estacionar.”

Behavior: I would support U.S. Spanish to be taught in Spanish classes in the United States.

Language policies and language maintenance/shift (Example of ‘language policies’ and ‘HL maintenance/loss and the benefits of bilingualism’ topic).

Attitude: In my opinion, primary schools should not allow students to use Spanish among themselves.

Behavior: I would speak to Hispanic students at my university about the benefits of keeping Spanish alive in the United States.

Step 3: Consultation with two groups of naive and experienced test-takers and content experts to investigate content validity

To explore the content validity of the questionnaire, an electronic version the original questionnaire was circulated among a group of six Ph.D. students in Spanish linguistics, a group of six bilingual young adults who served as naïve informants, and two questionnaire design experts to ensure clarity and elicit their technical input on the item quality. Their feedback was solicited in the form of short-answer questions through email. In addition, cognitive interviews were conducted with the first two groups (12 participants) to identify areas where interpretation of the questions was ambiguous (Collins, 2003). A revised version of the questionnaire, with ambiguous or confusing wording of items modified, was then distributed to four content experts (as recommended by Henk and McKenna, 2004), namely, tenured professors at U.S. universities with expertise in Spanish sociolinguistics and who had experience teaching about CLA at the graduate level, to review the questionnaire items and confirm that the items were appropriate to measure CLA in the HL context.

They were provided with a word version of the questionnaire and instructed to use track changes to give feedback and make any changes they considered necessary. Specifically, they were asked to answer the following questions about the items in each thematic area: (1) Do you feel the statements accurately address the topics in the thematic areas? (2) Would you add or take out any of the 48 statements? (3) Do you have any suggestions in relation to the thematic areas or the questionnaire? Based on their comments, 16 items were reworded or eliminated to produce a total of 32 items for the pilot phase. Most of the items eliminated were deemed redundant by the experts. Other items were reworded for clarity or accuracy. For example, the item: “I believe people who speak Spanglish should switch to monolingual Spanish” was changed to “I believe people who speak Spanglish should switch to just Spanish or English” to make it less technical. The item “I don’t like it when people codeswitch because it is not correct” was eliminated because it was considered repetitive. The content experts were also asked if the questionnaire was an appropriate tool to measure CLA and they all provided a positive response. The 32 items (16 for attitudes and 16 for behaviors) were distributed in the four themes as follows: eight items on language variation and diversity; eight items on language ideologies and linguistic prejudice; eight items on Spanish in the United States and bilingualism; and eight items on language maintenance.

Step 4: Pilot test with first group of students

The CLA questionnaire was piloted with 33 young Latino adults. These students were completing or had completed a Bachelor of Arts at a university in California. Most of their 33 young Latinos resided in California and had lived there most of the lives. Most of the participants were females (24, or 73%) and there were nine (27%) males. None of them had taken a HL course at the university level, which makes them comparable to the group of young Latino students who will be taking this questionnaire at the beginning of a heritage course.

The item scores were submitted to two statistical methods to evaluate internal consistency reliability: Cronbach’s alpha and corrected item-total correlations. Examination of individual items by alpha if item deleted indicated that deleting eight items increased Cronbach alpha levels. A Cronbach’s alpha of 0.77 was achieved for the scale scores after these deletions. By these means the questionnaire was reduced to 24 items: seven items on language variation and diversity; four items on language ideologies and linguistic prejudice; 8 items on Spanish in the United States and bilingualism; and five items on language maintenance.

Step 5: Full test

The 24-item CLA questionnaire was administered to 305 students enrolled in SHL courses at four large, public universities in the U.S. Southwest. The questionnaires were administered at the beginning of the semester during class time and took approximately 20 minutes to complete. We excluded all responses with missing data on any of the 24 questionnaire items, resulting in a final sample of 301 students. The participants ranged in age from 18 to 66, with a mean age of 21.60. The majority of students were female (196, or 65.1%); 101 (33.5%) were male, and four respondents did not identify their gender as female or male. Most were born in the United States (262, or 88.2%) and only 35 (11.8%) were born in foreign countries. Regarding their dominant language (i.e., Spanish, English, or other), 247 students (84.3%) reported being English dominant, 26 (8.9%) were Spanish dominant, and 20 (6.8%) reported either equal fluency or another dominant language. Respondents were also asked to rate their level of proficiency from 1 (excellent) to 6 (very poor). The mean scores were as follows: speaking Spanish: 2.89 (good); writing Spanish: 3.33 (good); speaking English: 1.75 (excellent–very good); writing English: 1.89 (excellent–very good).

Step 6: Statistical analysis

The 24-item CLA questionnaire results were submitted to statistical analyses to estimate their reliability and investigate construct validity using SPSS for Mac 25.0. Each item was scored individually, and those phrased negatively were reverse coded. Reliability, or how well the items fit together statistically (Haladyna 1999), was tested using Cronbach’s alpha to assess internal consistency. Construct validity was investigated through factor analysis. Factor analysis is a statistical method designed to measure “the number of distinct constructs needed to account for the pattern of correlations among a set of measures” (Fabrigar & Wegener, 2012, p. 3). Each construct is hypothesized based on the content of the items that load on the corresponding factor. Items that do not load on any of the factors are deleted because they do not contribute to the intended test constructs (Munro, 2005). For this study, the extraction method used was a principal component analysis (PCA). In PCA, all the variance of a factor (total variance) is analyzed and is assumed to be perfectly reliable and without error (Bryman & Cramer, 2005).

Construct validity

The 24 items in the scale were included in a PCA with Varimax rotation. Listwise deletion of missing values, eigenvalues, and factor loading scores of .4 were used to sort items into components (Stevens, 2002). The Kaiser–Meyer–Olkin (KMO) and Bartlett’s test was applied to ensure the sample size was adequate to undertake the factor analysis. A KMO value close to 1 is good and 0.6 is acceptable (Field, 2005). The obtained KMO value of .84 is therefore considered very good. We used three criteria to establish the number of components to rotate: Kaiser’s rule of keeping components with eigenvalues greater than 1, the scree plot of eigenvalues, and the interpretability of the component solution (Tabachnick & Fidell, 2001). Based on these criteria, three components were extracted that explained 37.33% of the variance. Two items were deleted because they did not load onto any of the components (items 15 and 24), and one item was dropped because it did not fit conceptually into the factor that it loaded (item 19). In the final run, the three-component model explained 40.5% of the item variance (Factor 1: 24.05%, Factor 2: 9.38%, component 3: 7.09%).

The final version of the questionnaire contained 21 items (12 attitudinal; nine behavioral). This model is interpreted as follows: Component 1: language variation, language prejudice, and English hegemony (nine items); Component 2: bilingualism, Spanglish, and code-switching (eight items); and Component 3: Language maintenance (four items). Table 3 presents the component loadings for the 24 initial questionnaire items and their weights.

Table 3.

Component loadings for PCA.

	Component 1	Component 2	Component 3
1. Language variation, language prejudice, and English hegemony (α = .79)
1. People from Spain speak the purest form of Spanish.	.69
2. In my opinion, people should use standard Spanish to communicate all the time.	.52
3. I feel speaking Spanish is less prestigious than speaking other foreign languages such as French.	.64
4. I believe Spanish-speaking Hispanics in the United States don’t speak correct Spanish.	.51
5. In my opinion, speaking English is an essential element of being American.	.59
6. In my opinion, it’s more important for immigrants to learn English and adopt the American culture than to keep their own language and culture.	.64
7. I believe growing up with both Spanish and English confuses children.	.49
8. In my opinion, families can speak Spanish among themselves at home but when in public they should speak English to each other.	.60
9. If I were to give a professional presentation, I would try to avoid working with a partnerwho has a strong foreign accent in English.	.57
2. Bilingualism, Spanglish, and code-switching (α=.76)
10. I believe people who speak Spanglish should switch to just Spanish or English.		.62
11. I don’t like it when people say ‘parquear’ instead of ‘estacionar’.		.54
12. I don’t like it when people codeswitch because it is not a proper way of speaking a language.		.67
13. I believe Spanglish should not be allowed in Spanish classes.		.66
14. If I were an English teacher, I would correct a student who says “I ain’t got none” in a formal context such as school.		.47
15. I would correct a friend if they spoke Spanish improperly.
16. I would tease a friend when speaking Spanish if they forget or invent words.		.42
17. I would try to avoid mixing Spanish and English in the same conversation as much as I can.		.67
18. I would not codeswitch in front of my teachers because they may think I am less intelligent.		.60
3. Language maintenance (α = .55)
19. I would codeswitch when I am with my bilingual friends.			.49
20. I would be willing to join a club who meets every week to discuss how to maintain Spanish within the family and community.			.60
21. I would support U.S. Spanish to be taught in Spanish classes in the United States.			.68
22. After college, I would commit to reading, writing, speaking, and listening in Spanish every day to continue developing my language.			.68
23. I would speak to Hispanic students at my university about the benefits of keeping Spanish alive in the U.S.			.51
24. I plan to speak only in Spanish to my children.

Internal reliability results

Cronbach’s alpha and corrected item-total correlations were calculated to estimate the internal consistency reliability of each of the three subcomponents of the questionnaire and to identify any items that did not contribute to reliability. Table 4 reports the corrected-item total correlations and Cronbach’s alphas if each item were deleted for factors 1, 2, and 3 for the 21 remaining items. With regard to item-to-total correlation, which tests for scale homogeneity, we followed the rule that items should be discarded if their item-to-total correlation is less than 0.3 and they are not essential for content coverage (Ferketich, 1991). Items 14 and 19 were deleted for not meeting the 0.3 standard, resulting in a Cronbach’s alpha of 0.79 for factor 1, 0.77 for factor 2, and 0.58 for factor 3. For Cronbach’s alpha, 0.7 is the minimum score considered to be reliable for the purpose of this questionnaire (Bland & Altman, 1997). The final questionnaire with 19 items shows an overall scale reliability of 0.71, after averaging the three subscale reliability scores, which is an adequate score (see Appendix A for the final version of the questionnaire).

Table 4.

Results of item analysis.

	M	SD	Corrected item total correlation	Cronbach’s alpha if item deleted
1. Language variation, language prejudice, and English hegemony (α = .79)
1. People from Spain speak the purest form of Spanish.	4.29	1.40	.55	.76
2. In my opinion, people should use standard Spanish to communicate all the time.	4.25	1.18	.43	.78
3. I feel speaking Spanish is less prestigious than speaking other foreign languages such as French.	5.06	1.17	.49	.77
4. I believe Spanish-speaking Hispanics in the United States don’t speak correct Spanish.	4.59	1.16	.44	.78
5. In my opinion, speaking English is an essential element of being American.	3.46	1.44	.48	.77
6. In my opinion, it’s more important for immigrants to learn English and adopt the American culture than to keep their own language and culture.	5.02	1.14	.51	.77
7. I believe growing up with both Spanish and English confuses children.	5.08	1.14	.44	.78
8. In my opinion, families can speak Spanish among themselves at home but when in public they should speak English to each other.	5.34	1.07	.53	.77
9. If I were to give a professional presentation, I would try to avoid working with a partner who has a strong foreign accent in English.	4.87	1.17	.46	.77
2. Bilingualism, Spanglish, and code-switching (α = .76)
10. I believe people who speak Spanglish should switch to just Spanish or English.	4.67	1.30	.53	.72
11. I don’t like it when people say ‘parquear’ instead of ‘estacionar’.	4.28	1.35	.43	.73
12. I don’t like it when people codeswitch because it is not a proper why of speaking a language.	4.69	1.16	.62	.71
13. I believe Spanglish should not be allowed in Spanish classes.	4.39	1.43	.53	.72
14. If I were an English teacher, I would correct a student who says “I ain’t got none” in a formal context such as school.	2.35	1.35	.27	.76
15. I would tease a friend when speaking Spanish if they forget or invent words.	4.24	1.49	.34	.75
16. I would try to avoid mixing Spanish and English in the same conversation as much as I can.	3.10	1.48	.49	.72
17. I would not codeswitch in front of my teachers because they may think I am less intelligent.	3.88	1.52	.45	.73
3. Language maintenance (α = .55)
18. I would be willing to join a club who meets every week to discuss how to maintain Spanish within the family and community.	4.36	1.25	.35	.47
19. I would support U.S. Spanish to be taught in Spanish classes in the United States.	4.68	1.26	.21	.58
20. After college, I would commit to reading, writing, speaking, and listening in Spanish every day to continue developing my language.	4.53	1.29	.38	.44
21. I would speak to Hispanic students at my university about the benefits of keeping Spanish alive in the United States.	4.65	1.23	.41	.41

Step 7: Questionnaire administration to measure CLA before and after CLA instruction

In order to measure the instrument’s ability to detect change in CLA, the final version of the questionnaire with 19 items was utilized in a subsequent study that sought to design, implement, and test a curriculum to teach CLA to a group of 19 fluent SHL learners over a period of 14 weeks using a pretest/posttest design. This group of students can be considered to have the profile identified in the SHL literature as the most typical, and therefore representative of the intended users of the scale (Beaudrie, 2011, 2012; Carreira, 2014, 2017). The change in the students’ levels of CLA, which ranged from 1 to 6, is reported separately for each factor in the scale. Pretest and posttest measures are presented in Table 5.

Table 5.

Pretest and posttest measures on CLA questionnaire ( = 19).

	Pretest		Posttest		Difference
	M	SD	M	SD	M
Factor 1 items	4.91	.53	5.13	.54	.22
Factor 2 items	4.67	.63	4.94	.63	.27
Factor 3 items	4.19	.79	4.61	.75	.42
Total	4.59	.65	4.89	0.64	.30

Paired-sample t-tests performed on the pretest and posttest scores showed significant post-instruction changes for factor 1 [t (18) = −2.38, p < .05], factor 2 [t (18) = −2.18, p < .05], and factor 3 [t (18) = −3.25, p < .05]. Further, Cohen’s effect size, which indicates the relative size of the effect of an intervention, for factor 1, (d = .41), factor 2 (d = .43), and factor 3 (d = .61), suggests small to medium-sized effects (Plonsky & Oswald, 2014). As a classroom-based study, smaller effects are expected than in laboratory studies (Plonsky & Oswald, 2014). It is also important to note that this group of learners showed a high level of CLA (4.6 out of 6) in the pretest. In addition, the results indicate that the instrument was appropriate to capture changes in CLA as students’ scores were significantly higher at the end of the instructional intervention.

Discussion

This study is the first study to attempt to design a psychometrically sound tool to measure CLA within the context of HL instruction. The questionnaire respondents during the development phase were 301 students enrolled in SHL courses in four large universities in the U.S. Southwest. After eliminating some items, the results indicate that the questionnaire meets adequate validity and reliability criteria for use in both research and educational contexts. Content validity analysis showed the items were relevant to the construct under investigation, and the internal reliability results were adequate. The PCA assessed the theoretical construct of the questionnaire and confirmed that three components were addressed in the scale. Principal component analyses resulted in the deletion of additional items that did not fit well into one of the subscales. The resulting average Cronbach’s alpha across the three subscales of 0.71 is adequate, especially for a newly developed questionnaire with a small number of items, and is acceptable for the intended low-stakes purpose.

The final questionnaire contains 12 attitude items and seven behavior items relevant to three themes: (1) language variation, linguistic prejudice, and English and hegemonic language ideologies; (2) bilingualism, Spanglish, and code-switching; and (3) language maintenance. Originally, we designed items in four thematic areas but areas one (language variation and linguistic diversity) and two (English hegemony, language ideologies, and linguistic prejudice) were related conceptually, which resulted in their unification after the statistical analysis.

These findings suggest that becoming critically language aware entails learners becoming cognizant of the naturalness of language variation and its loading of social, political, and economic power structures, while accepting the intrinsic value of their own bilingual varieties and all others. From this awareness, past researchers and pedagogues have discussed the importance of student agency in recognizing inequity and, more importantly, resisting dominant language ideologies by making their own decisions about language use and bilingualism (Clark, Fairclough, Ivanič and Martin-Jones, 1990, 1991; Clark & Ivanič, 1997; Holguín Mendoza. 2017; Leeman, 2018; Leeman & Serafini, 2016; Martínez, 2003; Parra, 2016. Finally, becoming critically aware involves positive behaviors toward maintenance and advocacy toward HLs. However, some of the items in this last thematic area did not endure the rigorous validation process and, therefore, the final questionnaire does not contain an appropriate number of items in this area, which results in an inadequate reliability score. In the future, researchers should continue to design a larger set of items for this section, possibly expanding the topics originally included to add items that are focused on issues of social justice and equity related to language.

In general, behavioral items did not perform as well as attitudinal items, so future research needs to continue to study the fluid structural and internal relationships between attitudes and behavior in the CLA construct. Future research should also contribute to furthering the evaluation of the tool, as well as explore the inclusion of additional possible topics to expand the questionnaire. Additionally, alternative ways to measure CLA should continue to be developed and evaluated in order to test successfully this important construct in HL research and education.

Conclusion

The results indicate that the CLA questionnaire needs further revision in order to meet the construct validity and reliability criteria investigated in this study, and that it possibly could be used to assess CLA in Spanish heritage learners enrolled in a university-level Spanish heritage course. Until now, the field of HL education has lacked studies on the effectiveness of CLA instructional programs, and it is our hope that this questionnaire eventually makes a significant contribution to begin to address this gap. However, one limitation of the study is that the sample population was limited to students enrolled in university-level SHL courses in the Southwest. Future studies should evaluate the generalizability of this questionnaire by expanding its use to other settings, other HLs, and other sample populations. This will also allow further measurement of the effectiveness of the instrument in capturing and comparing potential varying levels of CLA in diverse groups, who may be at different stages of CLA. A second limitation was that the Likert-scale used in this research had only 6 points. Research in educational measurement has shown that larger scales, with 7 to 11 points, may better capture questionnaire-takers’ underlying response variance on individual items and increase subscale reliability. Further revisions in such directions may prove useful.

Footnotes

Appendix A

Final statements used in the Likert-scale questionnaire annotated for theme and topic (summarized in Table 2). All items are scored 1–6, with 6 indicating strongly agree/very likely or, in case of “reversed scoring,” (indicated with *) strongly agree/very likely is scored as 1 and strongly disagree/very unlikely is 6.

Acknowledgements

We would like to express our great appreciation to all the Spanish heritage students, Spanish linguistic graduate students, and sociolinguistic experts who participated in several steps of the design, piloting, and testing of the instrument in this study. We are particularly grateful to the editors and reviewers of this journal for their valuable and insightful feedback on earlier versions of this article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Sara Beaudrie

Notes

References

Ali

(2005). Critical language awareness in the United States: Revisiting issues and revising pedagogies in a resegregated society. American Educational Research Association, 34(7), 24–31. doi:10.3102/0013189X034007024

Ali

(2011). Critical language awareness in pedagogic context. English Language Teaching, 4(4), 28–35. doi:10.5539/elt.v4n4p28

Artino

A. R.

Jr La Rochelle

J. S.

Dezee

K. J.

Gehlbach

(2014). Developing questionnaires for educational research: AMEE guide no. 87. Med Teach, 36(6), 463–74. doi:10.3109/0142159X.2014.889814

Beaudrie

(2011). Spanish heritage language programs: A snapshot of current programs in the southwestern United States. Foreign Language Annals, 44(2), 321–337. doi:10.1111/j.1944–9720.2011.01137.x

Beaudrie

(2012). Research on university-based Spanish heritage language programs in the United States: The current state of affairs. In Beaudrie

Fairclough

(Eds.), Spanish as a heritage language in the US: State of the field (pp. 203–223). Washington, DC: Georgetown University Press.

Beaudrie

S. M.

(2015). Approaches to language variation: Goals and objectives of the Spanish heritage language syllabus. Heritage Language Journal, 12(1), 1–21.

Beaudrie

Ducar

(2005). Beginning level university heritage language programs: Creating a space for all heritage language learners. Heritage Language Journal, 3(1), 1–26.

Bernal-Henríquez

Hernández-Chávez

(2003). La enseñanza del español en Nuevo México: ¿Revitalización o erradicación de la variedad chicana? In Roca

Colombi

(Eds.), Mi lengua: Spanish as a heritage language in the United States (pp. 78–95). Washington, DC: Georgetown University Press.

Bills

G. D.

(2005). Las comunidades lingüísticas y el mantenimiento del español en Estados Unidos. In Ortiz

L. A.

López Lacorte

(Eds.), El español en los Estados Unidos y en contacto con otras lenguas (pp.55–83). Frankfurt/Madrid: Vervuert/Iberoamericana.

10.

Bills

G. D.

Vigil

N. A.

(2008). The Spanish language of New Mexico and southern Colorado: A linguistic atlas. Albuquerque, NM: UNM Press.

11.

Bills

Hernández-Chávez

Hudson

(1995). The geography of language shift: Distance from the Mexican border and Spanish language claiming in the Southwestern US. International Journal of the Sociology of Language, 114(1), 9–27. doi: 10.1515/ijsl.1995.114.9

12.

Bland

J. M.

Altman

D. G

. (1997). Statistics notes: Cronbach’s alpha. British Medical Journal, 314–572. doi:https://doi.org/10.1136/bmj.314.7080.572

13.

Bryman

Cramer

(2005). Quantitative data analysis with SPSS12 and 13. A guide for social scientists. East Sussex: Routledge.

14.

Carreira

(2014). Teaching heritage language learners: A study of programme profiles, practices, and needs. In Trifonas

Peter

Aravossitas

Themistoklis

(Eds.), Rethinking heritage language education (pp. 20–44). Cambridge, UK: Cambridge University Press.

15.

Carreira

(2017). The state of institutionalization of heritage languages in postsecondary language departments in the United States. In Kagan

O. E.

Carreira

M. M.

Chik

C. H.

(Eds.), The Routledge handbook of heritage language education: From innovation to program building (pp. 347–362). New York: Routledge.

16.

Carvalho

A. M.

(2012). Code-switching from theoretical to pedagogical considerations. In Beaudrie

S. M.

Fairclough

(Eds.), Spanish as a heritage language in the United States: The state of the field (pp. 139–157). Washington, DC: Georgetown University Press.

17.

Cashman

H. R.

(2006). Who wins in research on bilingualism in an anti-bilingual state? Journal of Multilingual and Multicultural Development, 27(1), 42–60. doi:10.1080/17447140608668537

18.

Clark

Fairclough

Ivanič

Martin-Jones

(1990). Critical language awareness part I: A critical review of three current approaches to language awareness. Language and Education, 4(4), 249–260. doi:10.1080/09500789009541291

19.

Clark

Fairclough

Ivanič

Martin-Jones

(1991). Critical language awareness part II: Towards critical alternatives. Language and Education, 5(1), 41–54. doi:10.1080/09500789109541298

20.

Clark

Ivanič

(1997). The politics of writing. New York: Routledge.

21.

Collins

(2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12, 229–238. doi:10.1023/A:102325422

22.

Commission on Language Learning. (2017). America’s languages: Investing in language education for the 21st Century. American Academy of Arts & Sciences: Cambridge, MA. Available from www.amacad.org/multimedia/pdfs/publications/researchpapersmonographs/language/Commission-on-Language-Learning_Americas-Languages.pdf

23.

Do you speak American? (2005). Video documentary. Washington, DC: MacNeil-Lehrer Productions.

24.

Dörnyei

(2007). Research methods in applied linguistics. New York: Oxford University Press.

25.

Dörnyei

Taguchi

(2010). Questionnaires in second language research construction, administration and processing (2nd ed.). New York: Routledge.

26.

Fabrigar

L. R.

Wegener

D. T.

(2012). Exploratory factor analysis. New York: Oxford University Press.

27.

Fairclough

1989. Language and power. London: Longman.

28.

Ferketich

(1991). Focus on psychometrics: Aspects of item analysis. Research in Nursing and Health, 14, 165–168. doi:10.1002/nur.4770140211

29.

Field

A. P.

(2009). Discovering statistics using SPSS. London: Sage.

30.

Galindo

D. L.

(1995). Language attitudes toward Spanish and English varieties: A Chicano perspective. Hispanic Journal of Behavioral Sciences, 17(1), 77–99. doi:10.1177/07399863950171005

31.

Galindo

D. L.

(1996). Language use and language attitudes: A study of border women. Bilingual Review/La Revista Bilingüe, 21(1), 5–17.

32.

Godley

A. J.

Minnici

(2008). Critical language pedagogy in an urban high school English class. Urban Education, 43(3), 319–346. doi:10.1177/0042085907311801

33.

Gooden

(2007). Diversity in the linguistics classroom. In Branche

Mullennix

J. W.

Cohn

E. R.

(Eds.), Diversity across the curriculum: A guide for faculty in higher education (pp. 102–106). Pittsburg, PN: Anker Publishing.

34.

Granville

(2003). Contests over meaning in a South African classroom: Introducing critical language awareness in a climate of social change and cultural diversity. Language and Education, 17(1), 1–20. doi:10.1080/0900780308666835

35.

Groome

Gibbs

J.P.

(2004). Dialect awareness in literature and life. In Learn NC: K–12 Teaching and Learning. Retrieved from www.learnnc.org/lp/pages/3588?ref=search

36.

Haladyna

(1999). Developing and validating multiple-choice test items. Mahwah, NJ: Lawrence Erlbaum.

37.

Henk

W. A.

McKenna

M. C

., 2004. Developing affective instrumentation for use in literacy research. In Duke

N. K.

Mallette

M. H.

(Eds.), Literacy Research Methodologies (pp. 197–226). New York: The Guildford Press.

38.

Hidalgo

(1986). Language contact, language loyalty, and language prejudice on the Mexican border. Language in Society, 15(2), 193–220. doi: 10.1017/S004740450000018X

39.

Holguín Mendoza

. (2017). Critical language awareness (CLA) for Spanish heritage language programs: Implementing a complete curriculum. International Journal of Multilingualism, 11(5), 1–15. doi:10.1080/19313152.2017.1401445

40.

Huang

S. Y.

(2013). Revising identities as writers and readers through critical language awareness. English Teaching: Practice and Critique, 12(3), 65–86.

41.

Hudgens Henderson

. (2016). Sociolinguistics for kids: A curriculum for bilingual students (Unpublished doctoral dissertation). University of New Mexico, Albuquerque.

42.

Janks

(1999). Critical language awareness, journals, and student identities. Language Awareness, 8(2), 111–122. doi:10.1080/09658419908667122

43.

Jenkins

(2010). The state of Spanish in the Southwest: A comparative study of language maintenance and socioeconomic variables. In Rivera

S. V.

Villa

D. J.

(Eds.), Spanish of the U.S. Southwest: A language in transition (pp. 133–156). Norwalk, CT: Iberoamericana Vervuert.

44.

Leeman

(2005). Engaging critical pedagogy: Spanish for native speakers. Foreign Language Annals, 38(1), 35–45. doi:10.1111/j.1944–9720.2005.tb02451.x

45.

Leeman

(2010). Introduction. The sociopolitics of heritage language education. In Rivera

S. V.

Villa

D. J.

(Eds.), Spanish of the U.S. Southwest: A language in transition (pp. 309–317). Norwalk, CT: Iberoamericana Vervuert.

46.

Leeman

(2012). Investigating language ideologies in Spanish as a heritage language. In Beaudrie

Fairclough

(Eds.), Spanish as a heritage language in the United States: The state of the field (pp. 43–59). Washington, DC: Georgetown University Press.

47.

Leeman

(2014). Critical approaches to the teaching of Spanish as a local-foreign language. In Lacorte

(Ed.), The handbook of Hispanic applied linguistics (pp. 275–292). New York: Routledge.

48.

Leeman

(2018). Critical language awareness and Spanish as a heritage language: Challenging the linguistic subordination of US Latinxs. In Potowski

(Ed.), Handbook of Spanish as a minority/heritage language (pp. 345–358). New York: Routledge.

49.

Leeman

Rabin

Román-Mendoza

(2011). Identity and activism in heritage language education. The Modern Language Journal, 95(5), 481–495. doi:10.1111/j.1540–4781.2001.01237.x

50.

Leeman

Serafini

(2016). Sociolinguistics for heritage language educators and students: A model of critical translingual competence. In Fairclough

Beaudrie

(Eds.), Innovative strategies for heritage language teaching: A practical guide for the classroom (pp. 56–79). Washington, DC: Georgetown University Press.

51.

Lowther Pereira

K. A

. (2010). Identity and language ideology in the intermediate Spanish heritage language classroom (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses Global. (305185056).

52.

Lowther Pereira

. (2015). Developing critical language awareness via service-learning for Spanish heritage speakers. Heritage Language Journal, 12(2), 159–185.

53.

MacGregor-Mendoza

(2005). El desplazamiento intergeneracional del español en los Estados Unidos: Una aproximación. In Ortiz López

L. A.

Lacorte

(Eds.), Contactos y contextos lingüísticos: El español en los Estados Unidos y en contacto con otras lenguas, (pp. 287–300). Madrid and Frankfurt: Iberoamericana-Vervuert.

54.

Martínez

(2003). Classroom based dialect awareness in heritage language instruction: A critical applied linguistic approach. Heritage Language Journal, 1(1), 44–57.

55.

Martínez

(2010). Medical Spanish for heritage learners: A prescription to improve the health of Spanish speaking patients. In Rivera-Mills

Trujillo

J. A.

(Eds.), Building communities and making connections (pp. 2–15). Newcastle, UK: Cambridge Scholars.

56.

Martínez

Schwartz

(2012). Elevating “low” language for high stakes: A case for critical, community-based learning in a medical Spanish for heritage learners program. Heritage Language Journal, 9(2), 37–49.

57.

Mejías

H. A.

Anderson

P. L.

(1988). Attitude toward use of Spanish on the south Texas border. Hispania, 71(2), 401–407. doi:10.2307/343088

58.

Mejias

H. A.

Anderson-Mejias

P. L.

Carlson

(2003). Attitude update: Spanish on the South Texas border. Hispania, 86(1), 138–150. doi:10.2307/20062822

59.

Moreno

MacGregor

(2016). Introduction to special issue on service-learning and heritage language learners: Impact on communities, students, and classrooms. Heritage Language Journal, 13(3), i–vi.

60.

Munro

B. H

. (2005). Statistical methods for health care research. Philadelphia, PA: Lippincott, Williams & Wilkins.

61.

Murphy

(2012). Teaching dialect awareness in the college composition classroom: An evaluation (dissertation). Muncie, IN: Ball State University.

62.

Otheguy

(1993). A reconsideration of the notion of loan translation in the analysis of U.S. Spanish. In Roca

Lipski

(Eds.), Spanish in the United States: Linguistic contact and diversity (pp. 21–45). New York: Mouton de Gruyter.

63.

Otheguy

García

(1993). Convergent conceptualizations as predictors of degree of contact in U.S. Spanish. In Roca

Lipski

(Eds.), Spanish in the United States: Linguistic contact and diversity (pp. 21–45). New York: Mouton de Gruyter.

64.

Otheguy

Zentella

A. C.

(2012). Spanish in New York. New York: Oxford University Press.

65.

Parra

M. L.

(2013). Expanding language and cultural competence in advanced heritage and foreign language learners through community engagement. Heritage Language Journal, 10(2), 253–280.

66.

Parra

M. L.

(2016). Critical approaches to heritage language instruction: How to foster students’ critical consciousness. In Fairclough

S. Beaudrie

(Eds.), Innovative approaches in heritage language teaching: From research to practice. Washington, DC: Georgetown University Press.

67.

Pascual

Cabo

Prada

Lowther-Perreira

(2017). Effects of community service-learning on heritage learners’ attitudes towards their language and culture. Foreign Language Annals, 50(1), 71–83. doi:10.1111/flan.12245

68.

Pavlenko

(2002). “We have room for but one language here”: Language and national identity in the US at the turn of the 20th century. Multilingua, 21(2/3), 163–196. doi:10.1515/mult.2002.008

69.

Pavlenko

(2003). “Language of the enemy”: Foreign language education and national identity. International Journal of Bilingual Education and Bilingualism, 6(5), 313–331. doi:10.1080/13670050308667789

70.

Plonsky

Oswald

F. L.

(2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64(4), 878–912. doi:10.1111/lang.12079

71.

Poplack

(1980). I’ll start a sentence in Spanish y termino en español: Toward a typology of code-switching. Linguistics 18(7–8). 591–618.

72.

Reaser

(2006). The effect of dialect awareness on adolescent knowledge and attitudes (dissertation). Durham, NC: Duke University.

73.

Reaser

Wolfram

(2007). Voices of North Carolina dialect awareness curriculum. Retrieved from www.ncsu.edu/linguistics/dialectcurriculum.php

74.

Rivera-Mills

(2000). Intraethnic attitudes among Hispanics in a Northern California Community. In Roca

(Ed.), Research on Spanish in the United States: Linguistic issues and challenges (pp. 377–389). Somerville, MA: Cascadilla Press.

75.

Ruiz

(1988). Orientations in language planning. In McKay

S. L.

Wong

S. C.

(Eds.), Language diversity: Problem or resource? (pp. 3–26). Boston: Heinle and Heinle.

76.

Schmid

(2000). The politics of English Only in the United Status: Historical, social and legal aspects. In González

Melis

(Eds.), Language ideologies: Critical perspectives on the Official English movement (pp. 62–86). Philadelphia, PA: John Benjamins.

77.

Shin

Hudgens Henderson

(2017). A sociolinguistic approach to teach spanish grammatical structures. Foreign Language Annals, 50(1), 195–213. doi:10.1111/flan.12249

78.

Silva-Corvalán

(1994). Language contact and change. New York: Cambridge University Press.

79.

Stevens

(2002). Applied multivariate statistics for the social sciences (4th ed.). Mahwah, NJ: Lawrence Erlbaum.

80.

Tabachnick

B. G.

Fidell

L. S.

(2001). Using multivariate statistics (4th ed.). Boston, MA: Allyn & Bacon.

81.

Valdés

(1995). The teaching of minority students as academic subjects: Pedagogical and theoretical challenges. The Modern Language Journal, 79, 299–328. doi:10.1111/j/1540-4781.1995.tb01106.x

82.

Velázquez

(2013). Getting it: Sociolinguistic research and the teaching of U.S. Spanish. Critical Inquiry in Language Studies, 10(3), 191–214. doi:10.1080/15427587.2013.816826

83.

Villa

D. J.

(1996). Choosing a ‘standard’ variety of Spanish for the instruction of Native Spanish Speakers in the U.S. Foreign Language Annals, 29(2), 191–200. doi:10.1111/j.1944-9720.1996.tb02326.x

84.

Villa

D. J.

(2002). The sanitizing of U.S. Spanish in academia. Foreign Language Annals, 35(2), 222–230. doi:10.1111/j.1944-9720.2002.tb03156.x

85.

Villa

D. J.

Rivera-Mills

(2009). An integrated multi-generational model for language maintenance and shift. The case of Spanish in the Southwest. Spanish in Context, 6(1), 26–42. doi:10.1075/sic.6.1.03vil

86.

Wiley

T. G.

(2005). Language planning, language policy, and the English-only movement. In Finegan

Rickford

J. R.

(Eds.), Language in the USA: Themes for the twenty-first century (pp. 319–238). New York: Cambridge University Press.

87.

Wolfram

(1998). Language ideology and dialect: Understanding the Oakland ebonics controversy. Journal of English Linguistics, 26(2), 108–121.

88.

Wright

W. E.

(2005). The political spectacle of Arizona’s Proposition 203. Educational Policy, 19(5), 662–700. doi:10.1177/0895904805278066

89.

Zentella

A. C.

(1982). Spanish and English in contact in the United States: The Puerto Rican experience. Word, 33(1–2), 41–57. doi:10.1080/00437956.1982.11435721

90.

Zentella

A. C.

(1997). Growing up bilingual: Puerto Rican children in New York. Malden, MA: Wiley-Blackwell.

91.

Zentella

A. C.

(2008). Preface. In Niño-Murcia

Jason Rothman (Eds.), Bilingualism and identity (pp. 3–9). Philadelphia, PA: John Benjamins.