Considering Dialect in Survey Research

The Role of Dialect in Surveys

A dialect can be defined as a variety of a language spoken by a particular subgroup of speakers of the language within a society, based on region, social class, ethnicity, and other social categories. Dialects differ linguistically in terms of pronunciation, vocabulary, syntax, and norms of usage. They can significantly manifest themselves in various ways within surveys, both in questionnaire design and in the survey interview. With respect to the interview interaction, the use of dialect can provoke significant interviewer effects. For example, in Germany, in response to the question of whether respondents felt more comfortable using their own dialect or the standard, a high percentage confirmed being more comfortable in the standard when the interviewer was using the standard, whereas a high percentage declared themselves more comfortable using dialect when the interviewer was speaking dialect (see Hoag and Allerbeck, 1981). In Germany, the use and perception of German dialects has been surveyed regularly (see Noelle and Köcher, 1993, IfD-Allensbach, 2008). However, as we describe in this article, besides these specific cases, there are more important ways in which dialects may interfere with survey responding.

With respect to survey design, it is generally the case that instruments are crafted within a “standard” of a language within a country; that is, the most prestigious variety of the language within a society, spoken natively usually by higher status populations and encoded in the written language. This makes much sense from a practical standpoint. First, standard varieties are generally more likely than nonstandard dialects to be socially acceptable forms of communication for respondents within survey discourse. Second, standards in many countries tend to be understood by a greater number of people than particular regional or class-based nonstandard dialects. Moreover, survey designers are often native speakers of standards and construct questions using standard written forms.

The advantages of this approach for many surveys, at least from a theoretical and practical viewpoint, might account for why the role of dialect is usually neglected. Its relevance, however, may rise to the surface in particular linguistic settings. For example, in international contexts where there is no single standard and where there is the need for language harmonization (Spanish in Latin America, or varieties of English within English-speaking countries around the world), survey designers or translators must choose very carefully appropriate terms that are common to all varieties, avoiding for instance terms that belong to a particular dialect that may not be understood by all or that may have negative connotations for some respondents. In some contexts, potential respondents may not be able to understand the spoken standard variety of their language, as in China, where many regional dialects are extremely different from standard Mandarin. There are other countries that share these characteristics to some extent (such as Turkey and Italy). To take another example, non-native speaking migrants may have learned and understand only a regional or nonstandard variety of the language of the country they reside in, and may have difficulties responding to questions in the standard language.

Finally, there are contexts where surveys must be crafted or administered in local or regional dialects because the standard would not be a socially acceptable form of communication for this particular kind of interaction. Such is the case for “diglossic” countries ³ , where a formal standard variety is encoded in the written language, but is not used for everyday life situations involving forms of spoken communication. In such circumstances, it is generally not appropriate to use the standard for orally administered surveys.

Some of the dialect-related problems described above are aggravated where there is no written form of regional or nonstandard dialects (that differ greatly linguistically from the standard). In such cases, there is no real option to write questions in a form that would render them understandable or acceptable to respondents. Thus, with no resort to a written script, it is up to interviewers to adapt and reformulate the written survey questions to make them comprehensible and culturally appropriate for respondents. The following section describes the specific case of Switzerland, where in the Swiss German-speaking regions of the country interviewers commonly reformulate written standard German questions into spoken Swiss German dialects during the survey interview.

Survey Practice and Use of Dialect in Swiss German-speaking Switzerland – A Question of Identity

All national surveys in Switzerland are conducted across linguistic and cultural borders, generally in German, French, and Italian, with about 65 percent of the Swiss population having a Swiss-German dialect as a mother tongue, according to figures of the last census. Reading, writing, and speaking standard German is taught in school, and people generally have a sound passive understanding of spoken standard German, whereas active competence is more variable. In oral interviews, it is generally avoided because it gives the impression of a test situation, related to school, administrative procedures, and other official situations that can be perceived as negative, and can make respondents feel uncomfortable in responding to questions on beliefs, attitudes, or sensitive issues. ⁴ Speaking dialect is an important part of regional, cantonal, and national identity, and there are only a few specific settings where speaking standard German is needed or polite (Siebenhaar and Wyler 1997).

In the Swiss German-speaking part of the country, telephone or face-to-face survey questionnaires are therefore commonly administered in the spoken dialects. Given that there is no standardized written form of the many Swiss-German dialects, interviewers usually freely adapt questions from the scripted questionnaire in standard German, according to their own understanding of how it would be best understood in the dialect. Although the process of how to make questions suitable for oral transmission is sometimes addressed by data collection agencies during interviewer training, there is no unified policy or practice on this. Interviewers sometimes develop their own formulations that they keep over time, and sometimes they find different “translation solutions” for each survey, or for each interview.

Most written questionnaires to be administered in Swiss-German dialects are already adapted in the texts of questions by survey designers (even more so when it comes to the Swiss part of international surveys where often there are changes of wording that are more suited to particular cultural understandings and political circumstances). For example, words from the standard that are correctly understood by respondents when read in the standard, but that would be unnatural for an oral interaction in dialect, are avoided. The same applies to the standard German imperfect and future tenses, which do not exist in Swiss-German dialects. ⁵ However, these normally modest changes still do not allow for the reading of the questions as they are written, because they are still in a standard form. In addition to quite extreme changes in pronunciation, interviewers must adapt specific lexical and syntactic properties of questions. For example, the equivalent of to work in Swiss German is schaffe instead of the standard arbeiten. However, it is very unlikely that you would come across this word in any questionnaire, because an interviewer would find it confusing to see it in writing, and it is assumed anyway that the interviewer will do the conversion automatically. ⁶ A very natural sounding formulation in Swiss German is tüend Sie …, which can introduce almost any question. This is also never found in any questionnaire, because its equivalent in the standard looks awkward, and even silly, especially if it appears in every second question. This means that the interviewer must reorganize at times the whole wording of a sentence, and must introduce new vocabulary. Furthermore, the extent of necessary adaptation varies according to the regional dialect of the interviewer. Interviewers therefore still modify the scripted questions − that were already adapted for the Swiss-German respondent − appropriately without recourse to a provided written aid.

Writing questions directly in dialect has never been seen as a valid option in Switzerland because of the diversity of dialects and the lack of a widely acknowledged standard written code. Moreover, while there does exist a limited literary tradition of writing in Swiss German, reading or writing a text in dialect is a very unusual exercise. In contrast, it is common among Swiss people to think that standard written German can easily be translated spontaneously into spoken dialect. Thus, it is generally believed that interviewers will routinely ensure the equivalence of question meaning from written standard German to spoken dialect in the same way that Swiss-German speakers adapt information from the standard in everyday life. It is assumed that the changes made by interviewers have a minimal impact on the meaning of survey items, and that respondents are answering the “right” questions as intended.

However, an exploratory study by Renschler et al. (2010) conducted in Switzerland found that such dialect-driven interviewer adaptation carries a high risk of compromising measurement equivalence in various ways. That study assessed the extent to which Swiss German-speaking interviewers deviated from the scripted questionnaire to render questions more comprehensible and socially acceptable for respondents. It found that a very large portion of the questions examined were modified by interviewers in telephone survey interviews to make them fit to spoken Swiss-German dialects. For reasons of social appropriateness, this was to be expected, and interviewers are generally allowed and advised to do so. However, in this process, the meaning of a large portion of the questions was altered. In addition, a high percentage of the initial responses were coded as “not adequate”, suggesting problems with the question formulations.

Within the study, there were 1,032 transcribed instances of questions from the European Social Survey conducted by telephone in Switzerland. A portion of these was coded along different dimensions, including the number of specific changes to question wording introduced by interviewers, and whether or not changes involved a change of meaning. Analysis revealed that adaptations involved three or more changes by interviewers for nearly 70 percent of the coded questions, more than two-thirds of which were dialect-based adaptations. Considering the semantic effect of the departures from the original scripted wording, the meaning was changed for nearly 40 percent of the questions. To take an example, for the question “Would you say it is generally bad or good for Switzerland’s economy that people come to live here from other countries?”, “it is generally bad or good” was replaced by some interviewers with “they are generally bad or good”. To take another example, for the item “Modern science can be relied upon to solve our environmental problems”, some interviewers said instead the equivalent of “Modern science is capable of solving our environmental problems”. Finally, for the item “Political parties that wish to overthrow democracy should be banned”, the interviewer literally said “Political parties that wish to overthrow politics should be banned”.

Review of the findings for particular questions showed that no questions were immune to interviewer adaptation, and all questions were roughly equally prone to changes, including changes of meaning. Also, the specific changes to questions varied, even for the same questions across interviews by the same interviewer. With regard to the initial responses of respondents to questions, more than 40 percent were coded as “problematic” and did not involve an immediate adequate answer.

The observed high level of departures and changes of meaning could be attributed to the specific task demands of on sight translation. Interviewers are not only expected to carry out their normal work (asking questions, interpreting whether or not responses fit response categories, repair work), but they must also interpret questions simultaneously and modify them appropriately without recourse to any written aid. In the examined interview setting, this can be viewed as a quite complex cognitive task with heavy demands on working memory and attention (Japec, 2008). Since working memory and attention represent systems of limited capacity, it could be hypothesized that the cognitive load in this task sometimes reaches a critical level where cognitive control fails and problematic questions are produced. This could be the underlying cause for the many observed disturbances in the interview process, including unintentional changes of meaning.