Abstract
Despite its long term impact on planning, evaluation and documentation of Internet development, measurement of Internet usage is largely neglected in government policy actions in rural China. For researchers interested in evaluating government policy outcomes and Internet usage patterns in rural China, the only option is collecting primary data, which is resource-intensive and often fraught with data quality issues that entail many tradeoffs. Such quality issues have not been discussed in the literature in sufficient depth; this paper fills this void by discussing the challenges of collecting primary data on rural Internet users and suggests strategies for handling these issues based on the authors’ experiences of fieldwork in rural China.
Doing surveys in rural China is a very challenging process.
Introduction
Research has identified three important functions of measuring Information and Communications Technology (ICT) use: (1) informing network rollout plans, (2) evaluating the benefits of ICT, and (3) documenting development process for future reference (National Research Council, 1998; Ramirez and Richardson, 2005). Despite its importance, in practice measurement often comes as an afterthought, and it tends to be especially lacking in developing countries. China is the focus of this paper and is also a case in point: the Chinese government has poured copious resources into rural informatization projects in the past decade; however, these projects in general focus on rollout and overlook measurement and the evaluation of results.
At present, the only source of Internet development and user studies in rural China is the agency CNNIC (China Internet Network Information Center), which releases annual reports on rural Internet development and usage. Based on telephone surveys aggregating rural Internet users throughout China, these reports provide a broad stroke description of user profile and usage, which can serve as a basis for urban-rural comparison. However, for nongovernment organizations and researchers interested in greater detail about rural Internet users and in measuring benefits, currently the only option is to collect primary data, which is resource-intensive and often fraught with data quality issues that entail many tradeoffs. These quality issues have not been discussed in the literature in sufficient depth. Drawing on the authors’ experience of fieldwork in rural China, this paper discusses the challenges of collecting primary data on rural Internet users and suggests strategies to handle these challenges.
General challenges to conducting surveys in China
Roy, Walters and Luk’s synthesis (2001) of conducting business research in China provides an overview of general issues related to conducting surveys in China. They caution about directly applying theories and research instruments developed in the West to China. Due to different cultural and political contexts, survey instruments need to be revised throughout the process to make sure that they are understood and accepted by the local authorities and respondents, and that the questions validly capture the concepts of interest. In addition to instrument validation, also pointed out in the paper are the frequent need for authorization, lack of direct control of the data collection process, difficulties in random sampling, ambiguity about what is permissible or confidential, and self-censorship of local collaborators and respondents. These are recurring themes in the literature on conducting surveys in China.
Figure 1 illustrates the processes of conducting surveys in China, based on which this section is structured.
Process for conducting survey research in China.
Surveys, especially surveys about policy outcomes, can be politically sensitive in the Chinese context because data can reflect the performance of those in charge of policy; and there is an incentive to control the collection and flow of data (Tsai, 2010). As a result, domestic researchers often have to obtain permission from multiple levels of authorities in order to conduct surveys. The authorization requirements for research involving foreign affiliates are even more stringent (Manion, 2010): such studies must be conducted through agencies authorized by the National Bureau of Statistics, 1 and most of them usually do not cover rural areas. 2
As the next section will describe in greater detail, many factors on the ground can compromise data quality. Therefore, depending on the intended research subject and area, the selection of research site and partner is often dictated by the extent to which the researchers can obtain direct access to and control over the survey process, which tend to depend on guanxi (關係, personal connections. Peng and Nune, 2008; Roy et al., 2001; Tsai, 2010).
In a widely cited paper, Manion (1994) stated that collecting random samples in China was both impossible and impractical. Unfortunately, after almost 20 years this is still the case in most of rural China. Nevertheless, studies based on limited-scale random samples have appeared in a number of fields, and some recent literature focuses on issues with sampling and recruiting subjects when collecting primary data in China. Tang (2003) describes processes and considerations with working with the three types of organizations conducting surveys in urban China: government and quasi-government units, academic institutions and commercial survey companies. More broadly, Gustafsson and Shi (2006) share their insights on three approaches to collecting household income data in China: researcher-initiated surveys, secondary data released by provincial statistical agencies, and collaboration with government statistical agencies. Given a multitude of limits facing researcher-initiated projects, inflexibility and potential self-censorship involved in secondary government data, Gustafsson and Shi advocate collecting data through collaboration with provincial statistical agencies. The main strength of working with government statistical agencies is that this approach takes advantage of the panels the agencies have established, and researchers can have some control over questionnaire design (but not the actual data collection process). On the other hand, while staffers at the provincial statistical agencies are experts in collecting household-level income and expenditure data, they might not be as knowledgeable and experienced with collecting other kinds of information such as local Internet service and Internet usage. This may give rise to data quality issues discussed in the next section. The fact that researchers have limited control over the survey process also makes data quality a cause of concern.
An alternative for sampling or recruiting subjects is to use the hukou system (戶口, official records containing contact information and basic data of household members) as the sampling frame. However, even in the few cases where researchers have access to the hukou system, which supposedly is the most accurate and comprehensive record, problems still abound. Dai et al. (2012) collaborated with officials in Mianyang County, Sichuan Province to study suicide in rural communities, and they found that migration of rural population to urban areas has caused serious mismatch between the hukou records and the people who actually live in the areas studied. Their research also demonstrates the difficulty of using secondary government data: two authorities overseeing mortality records release inconsistent data because they use different sampling frames and methods. Their experience suggests that researchers must carefully decide the sampling frame according to the specific population they intend to study; in addition, secondary data should not be used uncritically.
Although often discussed separately, the steps of recruiting subjects, training staff, survey execution and questionnaire design are usually an iterative process. This is especially so for questions that may be politically sensitive or cognitively demanding for the respondents. Tsai’s (2010) discussion on studying provisioning of public goods in rural areas highlights the importance of staff training in obtaining valid data, which can manifest itself in different ways depending on research contexts. To obtain valid answers to complicated questions or concepts unfamiliar to the respondents, researchers often have to go through a challenging trial-and-error phase, gradually finding out how training and survey administration affect results and how to optimize the questionnaire design accordingly.
All the papers discussed above suggest that, either implicitly or explicitly, there is a tradeoff between data quality or validity and sample size, which is especially true in rural areas. Facing the obstacle of gaining access to a comprehensive and up-to-date contact list and securing full cooperation of local officials, researchers often resort to stratified sampling and non-probability samples. For research topics involving geographic, generational or social economic variation, stratified sampling is desirable because China is very heterogeneous along geographical, generational, and economic dimensions; lumping all categories together would mask important patterns (Adler et al., 1989). On the other hand, the use of non-random samples always requires caution. Manion (1994) points out that whereas it is not advisable to characterize a Chinese population based on descriptive statistics from non-probability samples, such samples can nevertheless produce unbiased estimates of relationships between variables, provided that the zero conditional mean of errors assumption holds.
The literature also cautions about informed consent: like researchers working in other developing countries, scholars studying rural China often find that the meaning of informed consent, which has strong cultural roots in developed western countries, often does not translate to people with low educational levels in developing countries (Goduka, 1990; Thorn, 1980). Liang and Lu (2006) further point out – on potentially sensitive issues – asking respondents to sign an informed consent form may actually threaten the confidentiality of the respondents’ identities.
In summary, the literature has identified many issues that have remained challenging for researchers collecting primary data in China. Although there has been considerable discussion on methods, in general how survey execution may affect data quality and validity is a key issue that has not been discussed in sufficient depth. This paper provides a more detailed account of the sources of potential quality issues that may arise during the survey process on the ground, and suggests coping strategies.
An account of the trial-and-error process of collecting primary Internet use data in rural China
One of the authors has studied government information technology policy in rural China, and the idea of measuring rural residents’ information needs and factors affecting their actual Internet usage came as a natural extension to the initial study, that focused on government interventions. After one year of qualitative fieldwork interviewing provincial and county level officials, we had acquired background information about rural Internet services in a few locations and established some contacts, on which we drew to select a research site, Wengyuan County, Guangdong. Wengyuan is located in the northern mountainous area in Guangdong Province. According to the 2010 census data, the average annual income of the rural population 3 in Wengyuan is 6,534 Renminbi (RMB), 4 falling between the national average of 5,919 RMB 5 and the provincial rural average annual income 7,890 RMB. 6 Meanwhile, although comparison of Internet penetration is unavailable, an unofficial estimate places Wengyuan’s Internet penetration between 16.6 percent and 22.6 percent (for 2010); not dramatically different from the national rural Internet penetration of 17.5 percent. 7
Choosing the survey site based on established contacts has a number of benefits: (1) with one year of continuing dialogue with the local officials we can more accurately gauge in advance the level of cooperation we can secure; (2) the relationship established helps resolve potential communication difficulties; (3) prior knowledge about the locale helps with the questionnaire design. In addition to resource constraints, these were the main reasons we focused on one county to start with. The downside is twofold: (1) local officials have little training in research methods; (2) lack of coordination between different local government agencies means that there is no guarantee that a contact list from the hukou system can be secured for random sampling. Given that our goal was to study factors affecting information needs and Internet use instead of providing summary statistics, we decided that the benefits of starting with established contacts outweighed the drawbacks.
Concerns over response rate, in addition to time and resource constraints, dictated that the most viable option to start with was face-to-face surveys with a non-probability sample (more in next section). The questionnaires were collected from passersby at a crowded location near the County Government and major commercial areas.
It took a number of smaller pilot studies (mainly for testing question order and wording) and two failed attempts to conduct the actual survey before satisfactory administration of the survey was achieved. The first attempt to conduct the actual survey was in July 2012 (we always picked weekends in order to cover people of all occupations). Literature review did not prepare us for the surprise: people were uninterested until they learned that by filling in a questionnaire they could get an umbrella in return. As soon as word about the umbrellas spread out, a mob quickly formed and all the questionnaires and umbrellas were soon snatched up. Unfortunately, most of the returned questionnaires were judged to be useless because it was clear that many people just checked answers without reading them, and there were a lot of missing data. This happened despite the presence of eight staffers; inexperience led us to underestimate the difficulty of maintaining order.
After thorough review and preparation, a second attempt was made in October 2012. Special arrangements and additional staff for maintaining order were employed to prevent chaos and these measures worked (more details in the next section). However, the data cleaning process revealed yet another problem: in spite of using trained staff as individual instructors, there was discernable data noise and some staffers’ results were a lot noisier than others’ (unanswered questions, contradictory answers, etc.) This is not uncommon in survey research. What was unexpected was the magnitude of noise considering the time spent on staff training.
The survey questions mainly concern local residents’ information needs, Internet use and factors that might be related to their Internet use. By most standards, these questions should be considered as straightforward and factual, so it was very puzzling why the instructors had such significant impact on survey results. On both occasions we spent at least a day socializing with the staff, and they seemed to be very confident and cooperative, although during training some staffers seemed to be more laid-back than others. It was a warning sign that we probably should have dealt with beforehand, but it did not seem to be the only problem, as the data patterns did not show clear correlation with our observations during the training period.
Not certain about what caused the observed patterns, we decided to re-conduct the survey in January 2013, but this time taking research assistants to observe the instruction process in order to find out the source of irregularities. With the presence of the additional monitoring the patterns observed in the second survey disappeared. Research assistants’ observations revealed a surprising finding: one staffer told his respondents what answers to select in order to speed up the survey. Based on our interactions with this staffer we would have never been able to tell; his attitude during the training period could only be described as exemplary and he came across as very efficient and resourceful. This certainly is an anomaly among the eight instructors, most of whom were very conscientious and professional. Nevertheless, such a rare occurrence, when it does happen, is very difficult to identify without additional monitoring.
Discussion
This section selects the main issues we encountered and discusses tradeoffs that must be made and strategies that may improve overall efficiency when collecting primary data of Internet users in rural China. The topics are arranged in the order in which decisions on these issues are to be made.
Interview method
Unlike conducting surveys in developed areas, the interview method is the most important decision a researcher makes when conducting surveys in rural China. In developed areas, respondents are familiar with surveys and communication infrastructures are mature; therefore, telephone surveys are commonly used. In such cases, researchers in general maximize sample size subject to total budget constraints. The considerations in rural areas are much more complicated: on top of cost increases due to sparse population, remote and/or difficult terrain, a more important complication is the issue of data quality. Three issues in rural areas tend to directly affect data quality: respondents’ cognitive ability, lack of trust and incentive to participate, and the need for monitoring.
Cognitive ability
As the Chinese countryside is going through rapid economic transition, working age people continue to migrate to cities, leaving the old and the young in their rural hometowns. Age aside, rural residents tend to be less educated and their reading comprehension level lower. Our pilot tests indicated that face-to-face and one-on-one instruction is necessary to guarantee data quality.
Trust and incentive
Many rural people tend to be wary of strangers, who are likely fraudsters selling fake goods targeting people with low education levels. Also, there is little incentive for them to cooperate with surveyors ; survey and social research are foreign concepts, and people do not warm to these ideas quickly. With the presence and endorsement of local officials, local people are more likely to participate in the survey. The face-to-face survey method not only inherently produces a higher response rate, but also addresses the issues with trust and poor infrastructure that plague telephone and mail surveys in rural China.
Monitoring
The presence of individual instructors and additional monitors not only makes people take the survey more seriously; their main function is to make sure respondents understand the questions and answer accordingly. As our experiences show, monitoring is essential in guaranteeing data quality as there can be significant variation across staffers regarding the quality of their work. One solution is to bring in trained research assistants from outside and pair them up with local staffers. With training, the majority of local staff has little trouble following survey protocols, but they can often use help handling the questionnaires, distributing the gift incentives and making on-the-ground adjustments. In this regard, we also observe that the outside research assistants seem to have a morale-boosting effect; local staffers enjoy their company and help, and the teamwork context also motivates them to present their best in a way that improves data quality.
Taken together, the issues of respondents’ cognitive ability, trust and monitoring should make it clear why there is an important tradeoff between data quality and cost/sample size. Focusing on data quality, human resources for one-to-one instructions and monitoring become a major constraint for sample size. When the budget is limited, conducting the survey at a central location is the most practical approach. Although the nature of such non-probability sampling is a cause for concern, some strategies can be used to minimize potential biases. Local governments down to the county level publish basic census data, including population makeup by gender, education level, age and industry (in very crude classification.) Such information can be used to gauge whether the sample is biased.
Importantly, as Manion (1994) points out, researchers must recognize what is achievable with a non-probability sample. Non-probability samples can be biased regarding population makeup; nevertheless, if one is looking at the relationship between variables, such samples may not necessarily produce biased estimates provided that the researcher can persuasively argue that the zero conditional mean of errors assumption holds.
Establishing contacts
The major difference between conducting social science surveys in China and democratic societies is uncertainty over what is politically sensitive. There are many reasons why releasing data or working with researchers from outside the system may have unexpected consequences (Schwartz, 2001, p. 114). Facing this uncertainty, Chinese officials at all levels tend to err on the safe side and shun requests for research assistance. This is why guanxi is necessary. As guanxi usually involves requests from higher level in the hierarchy, it eases concerns over political sensitivity and can help secure the cooperation of those who may otherwise have no incentive to facilitate research.
Many researchers mention the necessity of mobilizing personal networks to gain access to the intended organizations/persons. 8 In the authors’ experience, gaunxi is necessary, but it does not mean that an outsider without a pre-existing network cannot establish one in a reasonably short period of time. Here is where the interesting evolution of guanxi manifests itself in contemporary China: a positive development the authors witnessed is the emergence of professional technocrats at the middle-level of provincial governments and commercial entities. These people are usually young, trained professionally and open-minded. They are willing to spend time to listen if one approaches them with a useful idea. This is especially true in places where cooperation among industries, government and academic institutions is encouraged by government policy.
To start this process, researchers should first narrow down the search to a specific area, usually a province. In looking for initial contacts, researchers need to identify suitable candidates to approach, and Chinese media coverage can be very useful in this phase. The best place to start is commercial newspapers: if an issue is of substantial social and policy concern, a thorough review of recent media reports will usually produce a few leads in the form of press interviews with key stakeholders, such as government officials and representatives of carriers. Based on the content of the interviews, the researcher can make an educated guess as to who is more likely to be professional and open-minded, thereby increasing the possibility of successful contact. This is where initial guanxi can be established. The right contacts can introduce the researcher to those who can assist the survey, and can also be tremendously helpful in pointing out which other entities should be contacted for a more complete picture. Of course, this process takes trial and error. Persistence and common sense are necessary, but it is not impossible to establish such contacts in the timeframe of one year. At this level, doing research and establishing research contacts are not that different from doing research in developed countries; the differences usually reside at the top and bottom of organizations.
Other than starting from scratch, another possibility is to approach people with existing guanxi, such as an active researcher in the area. This is a good way to quickly expand one’s research network. On the other hand, Tang (2003) points out that “It is important to note that the actual success of a survey project depends on the compatibility of interest between the collaborators.” Tang (2003) cited an example of failed collaboration between a team of western political economists and Chinese government officials who have to confirm government education policies. As explained in the literature review, self-censorship may also be an issue (Gustafsson and Shi, 2006). These issues may arise when collaborating with either local officials or academics; the major difference is about the extent of the limits. Other than these issues that are specific to the Chinese context, the general considerations regarding joint work also apply here: agreements on both authorship attribution and complementarity of resources are important foundations for cooperation.
For non-Chinese researchers, a few additional issues must be taken into consideration. First, surveys involving foreign affiliates must be conducted through agencies authorized by the National Bureau of Statistics (NBS). 9 This means that most of the interactions with respondents will be mediated by Chinese staff. For non-Chinese researchers, their control over the survey process can be compromised, but when the boundaries for survey questions are relatively clear, the upside of having experienced Chinese staff in charge is smoother survey execution.
Compared to standard closed-ended question surveys, being a foreign national probably has much greater impact on research involving interviews with open-ended questions. Depending on the topic, the presence of non-Chinese researchers can be either a blessing or a curse. People are welcoming and friendly to foreign nationals, but in general it is quite hard for non-Chinese researchers to approach people about issues with policy implications. When it comes to such topics, people tend to be alarmed and reserved in the presence of foreign nationals (Schwartz, 2001). In contrast, on cultural topics people are very welcoming to foreign researchers and enthusiastic to help. Sæther (2006) actually reports having more leeway in conducting fieldwork in China because people expect foreign nationals to act differently and they tend to provide more contextual details when being interviewed by foreign researchers.
Site selection/access
The key function of guanxi in survey research is to secure access to administrative help; therefore it is very important to gauge in advance the extent to which established guanxi can guarantee an adequate level of support. Importantly, it is not recommended to aim for a survey right away because a successful survey requires substantial trust between the researcher and local collaborators. A good strategy is to start with an exploratory study based on interviews with important local stakeholders. This provides a good opportunity to work with local contacts and establish rapport and trust. The knowledge gained during this process will also be very helpful in subsequent stages. 10 It cannot be overemphasized that while cultivating one’s guanxi network, the researcher should also pay attention to selecting communities with statistical profiles similar to that of the population under study.
Although research is not yet integrated into China’s policymaking process, officials at all levels are aware of the trend and they are quite cooperative as long as all required approvals are cleared. Guanxi may have some negative connotation as it is often associated with nepotism and corruption, but in our experience local officials were quite respectful of our research plan and guanxi did not entail ethical compromises. Another positive impact of guanxi on research is that researchers can obtain documents through contacts that may otherwise be unknown to the researcher. Such documents may not be confidential, they have just not been actively promoted to the public and there are usually no efforts to make them accessible to the public (Nordveit, 2011).
As described in the literature review, many researchers have pointed out the difficulty of compiling a comprehensive and up-to-date contact list (Dai et al., 2011; Peng and Nunes, 2008). These problems are even more pronounced in rural areas due to the lack of professionally-trained personnel and administrative fragmentation (multiple government agencies in charge of overlapping functions). Administrative fragmentation and departmental rivalry are common in China (Dai et al., 2011; Qiang et al., 2009; Roy et al., 2001; Ting and Yi, 2013). These agencies usually do not coordinate with each other, so researchers looking for a comprehensive list have to contact all relevant agencies. For those new to the topic, a thorough background research on media coverage of the issue may help; the initial contacts may also help identify other agencies in charge of the same functions.
Survey instrument design
The literature has identified two main issues with instrument design: concept equivalence and measurement equivalence (Liang and Lu, 2006; Peng and Nunes, 2008; Roy et al.,2001; Stening and Zhang, 2007). Depending on the research questions, translation and concept equivalence may be more difficult in some studies than others. The heterogeneity along geographic, generational and social economic lines in China is reflected in language. It is therefore essential for researchers to consult people with similar backgrounds to the intended respondents to test concept equivalence prior to the survey. In addition, researchers must work with local staff to ensure that the wording of questions can be easily understood and the questions can be meaningfully interpreted and answered by the respondents. Questions need to be revised based on observed validity issues and new translations tested. This process usually takes quite a few iterations until pilot tests no longer show further validity issues.
Measurement equivalence is a particularly tricky issue with conducting survey research in rural China because of problems related to rural residents’ reading comprehension and cognitive ability. Measurement tools commonly used in developed economies often pose problems for respondents in rural China. For example, our pilot survey reveals that many rural people have trouble answering the commonplace question: “On a scale of 1-5, how would you rate your knowledge of X?” The concepts of scale and intensity matching take a lot of explanation and are a potential cause of confusion. To elicit valid and more accurate responses, using verbal descriptions such as “None at all”, “Very little”, “Some knowledge”, “Quite knowledgeable” and converting the responses back to ordinal measures may be a more effective and efficient approach. It must be noted, though, that comparing to the typical Likert scale, this may compromise granularity and therefore preclude some statistical techniques that require normally distributed variables.
As mentioned, not many rural respondents are interested in the purpose and benefits of surveys, and if they choose to participate in a survey, it is usually because of gift incentives. Uninterested in the purpose and benefits of surveys, the respondents tend to see a survey as trading a filled-in form for a gift. They just want to get over the part of filling the questionnaire and go home with their gifts; the quality of the information is not a concern to them. Even young people often check answers randomly or mechanically without reading the questions. It is therefore worth considering inserting a few questions to test the consistency of answers. For example, someone who uses the Internet on a monthly basis should not check that she searches for entertainment news frequently. Such questions can be quite useful in singling out questionnaires with quality issues.
Survey administration
Resource constraints may render conducting the survey at a central location the most practical approach. Setting up such a central location requires the approval of local city management authorities. Cooperating with local officials can expedite the approval process.
As explained above, gift incentives are important because many rural residents see surveys as irrelevant distractions and they have little incentive to participate even with officials’ encouragement. The average annual income for rural Chinese is 5,919 RMB, and the appropriate gift incentives should be between 10-30 RMB, approximately the price of an umbrella or a low-end thermal cup. Additional attention should also be given to the fact that males are harder to recruit, so gender-specific gift incentives such as belts or cigarettes may be considered to recruit a more gender-balanced sample.
With gift incentives, researchers often have to address the opposite problem of having too many respondents in too short a period. A few factors often conflate to form a chaotic situation that severely undermines the quality of responses. People’s indifference to survey purpose and quality, combined with the fact that queuing is not in the system of normal behavior in rural China, often leads to disorderly behavior such as jumping queues (if there is one at all) and verbal confrontations. If the weather is hot, tempers flare and further chaos might ensue. On top of maintaining order, researchers also need to prevent repeated participants and the theft of gift incentives: when not monitored, many people would just come back repeatedly to get as many gifts as possible, while some would simply take the gifts without filling in a questionnaire. Such behaviors can deteriorate once a mob is formed. Indeed, this was how the authors’ first attempt to conduct the survey failed.
To avoid these problems, the researcher must ensure an orderly queue to start with. Because queuing is not yet established in local norms, the solution requires some physical setup and policing. One way is to use tents, 11 tables, and/or nearby building structures to create an area reserved for filling-in the questionnaires. People can access this area only through a narrow entrance and a narrow exit on the opposite side. Both the entrance and exit must be watched to ensure that only authorized people enter the reserved zone. The queuing zone and lines should be clearly indicated outside of the entrance and at least one person is required to be on watch at all times to maintain order. A cautionary measure is to have staffers pretending to be recording the line to deter repeated participants.
Monitoring individual respondents as they are filling in the questionnaire is equally important in maintaining order and ensuring data quality. Assigning one instructor to each respondent is necessary because respondents often have trouble understanding the written questions, and without being watched many of them tend to check the answers randomly or mechanically without reading them. The presence of instructors/monitors also tends to make those waiting in line more patient and serious about the survey.
It is not recommended to conduct the survey over multiple days because the problem of repeated respondents becomes intractable. On the other hand, collecting a large sample within a day requires a large workforce, which may pose its own problems.
Informed consent
Informed consent is a familiar concept for people trained in the western tradition; however, the spirit of it tends to be lost when conducting surveys in rural China. A common reaction from respondents (when being read a statement of informed consent) is: “Hurry, what is your question?” In this situation, a more effective and practical alternative may be using posters and speakers to make the disclosures when people are waiting to fill in the questionnaires. Another viable solution is to distribute a separate consent form to respondentsz when they are waiting in line.
Face (Mianzi 面子)
A relevant and very interesting issue about conducting research in China is whether the concept of face (mianzi) is more pronounced in the rural context. Hwang (2011) provides an excellent discussion of Chinese concepts of face, lian and mian, and how they impact on social interaction. Ho (1976) defines face as the respectability and/or deference which a person can claim for himself from others, by virtue of the relative position he occupies in his social network and the degree to which he is judged to have functioned adequately in that position as well as acceptably in his general conduct.
From this definition it is clear that the concept of face in the Chinese culture goes beyond the concept of esteem or social status, and is inseparable from the group context in which one positions oneself. However, as China is undergoing rapid transformation, the concept of mianzi is also changing. In modern China, the degree to which mianzi affects social interactions depends on the situation: the extent of formality of the occasion and the age of those present are two important modulators. The more formal the occasion, the more sensitive people are about mianzi; older people also are much more sensitive than younger generations about mianzi (Hwang, 2012).
Our experiences suggest that there is some variation among people, but on average rural Chinese are proud of their straight-forward, down-to-earth lifestyle and values, which makes collaboration easier and more enjoyable. For example, in Chinese culture, a banquet is a way to show hospitality and respect for guests, who are sometimes put to the test of incessant toasting. Declining to drink would be seen as not honoring the person proposing a toast. This phenomenon seems to be more prevalent in cities; in contrast, banquets in rural areas usually are simple and the hosts are considerate and seldom pressure guests to drink.
Another factor may be the age and experience of the rural officials involved. In China it can take years for one to climb the bureaucratic ladder and be promoted to cities; during this process, one may become more accustomed to the implicit power rules and mianzi. However, quite a few of the rural officials we worked with were young and at an early stage of their careers, so our interactions tended to be informal. Compared to our experience in other places, it did not take extra effort to maintain face in rural Guangdong.
Mianzi was indeed a concern when we first realized that some instructors’ work had quality issues. Because no one staffer’s data stood out as the source of the problem, we could not identify who was responsible. Instead of interrogating individual staffers, which would not work well in any culture, our solution was to bring in research assistants from outside and pair them up with local staffers. Throughout this process we did not use the word ‘monitoring’ as most staffers did not need monitoring and those who violated survey protocols soon changed their behavior when the research assistants approached them. Overall, there are many challenges for researchers on rural issues such as rural staffers’ lack of professional training, but mianzi does not stand out as an issue in this context.
Conclusion
Collecting primary data in developing rural areas is a process often fraught with unanticipated complications and requires painstaking background research, prior planning, and continuing on-the-ground adjustments. Drawing on the authors’ experience of conducting surveys in rural China, this paper focused on problems threatening data quality and suggested strategies to address these problems. A key issue raised in this paper is the need for face-to-face interview surveys and additional monitoring, which are required to guarantee data quality. In addition, this paper also discusses approaches to establishing contact, site selection and instrument design; it also suggests monitoring arrangements to prevent common problems.
The advices offered in this paper are based on small-scale non-probability samples, but many of them are applicable to large-scale surveys as well. For a large random-sample survey, the researcher has to rely on local staff sent to each survey unit and the key to success probably lies more in selection of outside/local staffers than incentive design and auditing mechanisms. To make the process smoother, the authors suggest a phased strategy: start with an exploratory interview study, which can pave the way for a smaller non-probability sample survey. Only when able to manage smaller surveys successfully should the researcher move on to the next stage.
Doing surveys in rural China is a very challenging process, but the rewards are also tremendous. Measurement of technology use is essential to planning, evaluation and documentation of infrastructure development; by collecting primary data on Internet use, independent studies can supplement and verify government sources, thereby helping the Internet to achieve its full potential in rural development.