Abstract
This paper reports on the challenges faced by two researchers during data collection and translation of data and analysis in two public sector universities in Pakistan. Data collection from each institute involved different procedures and a different set of issues, including negotiating access with gatekeepers and participants, dealing with teachers’ attitudinal issues, and getting consent from female participants. Challenges related to access to the institution, to female participants and recording permissions are discussed. Both researchers faced issues in translation of the key constructs with no equivalent lexical items in the context of study. It is further argued that awareness of such issues beforehand will help novice researchers be better prepared. Implications for novice researchers from Pakistan, and researchers from other contexts, have also been discussed.
Introduction
Research studies across various contexts have shown that researchers face challenges related to access and gatekeeping (Riese, 2019; Vuban & Eta, 2019; Wanat, 2008) as well as translation (Birbili, 2000; Fersch, 2013). However, the nature of challenges and the strategies required to deal with them vary from context to context. Therefore, novice researchers must familiarise themselves with factors that debilitate and facilitate access to research site and participants long before the fieldwork in order to develop locally workable strategies. This paper, therefore, shares insights into the specific nature of the challenges and the context-specific strategies the authors faced during data collection and analysis in the context of Pakistan. The first study reported in this paper was conducted by Researcher-1 who investigated university students’ willingness to communicate in a second language (L2 WTC) using 14 structured observations, 28 hours of video-recordings, 86 diary entries, 18 stimulated recall interviews, biographic questionnaires and field notes. Researcher-1 was a 34 years old Sindhi-Speaking teacher with six years of English language teaching experience at various universities in Sindh province of Pakistan (Syed and Kuzborska, 2020). He had been educated in Pakistan up to master level and had obtained his MA in English literature from a public sector university in Pakistan in 2007. He had an international experience of research completed his master in English Language Teaching (ELT) in 2012 from a reputable university in the UK, and was enrolled in a PhD program at a renowned university in the UK. He had a six months’ experience of teaching English at this university in 2011 (i.e. Jan-Sep 2011) which served to build rapport with the administration and made way for getting access to teacher and participants. While the class teacher was somewhat familiar with Researcher-1, the research population did not know him.
The second study was conducted by Researcher -2 and involved 26 semi-structured interviews with teachers and students regarding their perceptions of identity, rights and duties in relation to fiction. Researcher-2 had been a former teacher in department of language and literature and knew the teachers as former colleagues. Though she had not taught the participants in the study but some of them knew her from when she was part of the department (Syed, 2019). Being a Sindhi, female researcher from the same context, helped gain access to the participants.
The article begins with a description of the contexts of the two studies, followed by specific issues during data collection, including negotiating access with gatekeepers, teachers and female participants; and challenges faced during translation. The article ends with implications and suggestions for novice researchers aiming to conduct research in Pakistan and other similar contexts.
Challenges in data collection
The main challenges faced by researchers in the two contexts were related to: Negotiating access to gatekeepers; getting across teachers’ attitudinal issues; and most importantly, obtaining consent from female participants.
Negotiating access with gatekeepers in higher education (HE)
Access to research participants is one of the major issues in all qualitative research. Access is shaped by the relation between the researcher, research participants, site and the power dynamics. It is a relational process in the sense that it allows both researcher and the research participants to access each other’s’ mental process. Moreover, it depends on not only the research site’s accessibility but also a researchers’ ability to access it (Riese, 2019). While access and cooperation are used interchangeably, a formal approval from the gatekeepers does not entail a cooperation from them (Wanat, 2008). Gatekeepers can be defined as people in a position of power to grant access to the site and participants. Gatekeepers include both formal, i.e. bureaucrats, who may grant official permission to enter a site, and informal gatekeepers which include people in charge of protecting participants and research settings. Thus, it was necessary to choose a site which ensured not only access but also cooperation throughout the process of data collection. These considerations led us to decide upon the university where we had worked as teachers before leaving for PhD in the UK.
Negotiating access with management
It is interesting to note that while both research scholars belonged to universities governed by the rules and regulations of the Higher Education Commission (HEC) of Pakistan, both universities had a different system in place for giving permission for collecting data. For instance, while his past association with the university helped Researcher-1 in icebreaking, he was obliged to follow the standard operating procedure (SOPs) by writing an application to the office of the registrar followed by a short presentation, and seeking approval from the concerned head of department (HOD), and the teacher. Researcher-1 had to fill and sign a form ensuring administration that the study involved no potential threat to the institution’s established reputation, any physical or emotional harm to participants (Riese, 2019; Vuban and Eta, 2019; Wanat, 2008). In order to build trust with them, he shared ethical approval from the university he was doing his PhD in, and the data collection procedure and timeline. The admin office provided me with an approval conditioned to an approval from the concerned HOD.
The HOD, however, had a different set of concerns including: the continuation of the classes, and convenience of the teacher whose classes were to be observed, and the students who were to be observed. Since there had been changes in the department’s management since Researcher-1 had left for PhD, the new HOD was not familiar with him personally. Thus, he used his past contacts, i.e. fellow teachers and colleagues, within that department to convince the new HOD for permission, reassuring him that the class schedule would not be disturbed and there would be no extra-curricular demands on participants.
Researcher-2, however, had to go through a different ordeal for getting access. Her university did not have any ethics department in place, so she obtained the consent of the HOD on telephone without having to fill any forms and submit any documents. While she managed to avoid all the paper work, she had to spend considerable time following individual teachers and students to obtain their consent separately as they had not been informed of her plans at all. Therefore, in such situations, knowledge of the official procedure, or its lack thereof, is important. Leaving time to plan things on-site also helps.
Negotiating access with teachers
Teachers also played the gatekeeping role in both contexts. Teacher’s consent and cooperation was highly necessary in order to conduct the study successfully and smoothly. For instance, the first study aimed at observing and examining learners’ L2 WTC inside classroom; it was essential to observe postgraduate students in communication-oriented classes, i.e. Business Communication (BC). While the classroom teacher knew Researcher-1 well, and did not object to his conducting research in his classroom, he expressed some strong reservations regarding video-recordings of his classes; he agreed to observations but was reluctant to allow video-recordings. He believed it would affect his natural performance in the class. Second, he was also concerned about the security of the data, that is, who will have access to these videos and how they would be reported in the research. Third, he was even more apprehensive of the fact that since the study involved students’ diaries and interviews, the students might report negatively about him in their responses. Finally, the teacher was concerned about the longitudinal observations within the class for ten weeks. One of the possible reasons could have been the frequent use of code switching in the class. Since English is officially the medium of instruction in higher education in Pakistan, teachers are expected by their department heads to make exclusive use of English inside the classroom.
In reality, however, both teachers and students preferred to use code switching for instruction and interaction inside the classroom. However, these issues were mitigated by reassuring the teacher that the research did not focus on the teacher but on students and their communicational behavior. Secondly, he was also informed that video-recordings will be protected in a passworded folder accessible only to the researcher for the purposes of current research (Thummapol et al., 2019).
In case of the second study, Researcher-2 had to wait for weeks to obtain the teachers’ permission for classroom observations. This was despite the fact that each teacher had three classes in a week with the same group of students, yet they showed hesitation and paranoia. Even when they allowed observations, they made sure they did not have to teach the lesson; so they asked students to do presentations on the topic of their choice. This might possibly be due to unfamiliarity with and discomfort towards a peer observer in the classroom.
Varying and vacillating attitudes of the teachers
Despite familiarity between researchers and their teacher participants, the attitudes of the teachers were characterised by a heightened sense of insecurity regarding observations and interviews (Vuban and Eta, 2019). For instance, Researcher-2 reported that some of the older teachers, especially the ones who were members of the Board of Studies were reluctant to give interviews. When she talked to them informally about her project, they had a lot to say. However, when she turned on the recording and started the formal interview, their responses were limited and there was this sense of distrust. They did not wish to go on record and say things that could be later traced back to them. Researcher-2 also reported on the attitudinal issues she faced during interviewing teachers. Additionally, she realised that being teachers of English language, the teachers felt obliged to respond to interview questions in English which limited their responses and eventually rendered superficial data. She overcame this issue by conducting follow-up interviews with these teachers using their L1 only which seemed to help. More importantly, teachers who at first readily consented to interview and expressed appreciation of the objectives of the study, eventually turned a cold shoulder. Others blatantly refused to give permission to record the interview and, after the researcher’s (i.e. Researcher-2) persuasion through assurance of anonymity and safety, only allowed note-taking during the interview. Apparently, these teachers were trying to shy away of admitting that the curriculum had not been revised as it might not reflect well on their authorities.
Negotiating access with students
In L2 WTC study, access to participants and students was even harder than access to other stakeholders. The BC class consisted of 45 (Male (m) =25; Female (f) =20) students with varying lingua-cultural backgrounds. Students were invited via email to participate in the study. Surprisingly, there was no response on the part of students. While students were informed of the aims and objectives of research, the reason students did not respond was that they were unfamiliar with and had never met or seen Researcher-1. Consequently, it was decided to directly approach the students and meet them face-to-face in the class. It turned out to be a useful idea as students’ queries regarding the process of research, confidentiality and anonymity were answered in detail which helped a great deal to alleviate their apprehensions. As a result, eight students, (m = 4; f = 4), volunteered to participate in the study.
In case of the second study, most of the participants in the sample were enthusiastic about the interviews and eager to contribute to the discussion. Even though there had been cases of unwilling participants, most of the ones who participated were excited to take part and wanted to be listened to. It relates to what Jansen (2015) says about her own participants- children were willing to talk to Researcher-2 because they saw their status as a resource in the situation rather than a problem. The participants in this study may have been willing to talk as they considered this as their way of helping out- it was a reverse power situation.
Issues in obtaining consent from female participants for video-observations
Access to female participants proved to be challenging due to several socio-cultural and technical reasons. Female participation in HE in Pakistan is relatively lower than their male counterparts due to reasons, including the patriarchal family system, a sense of vulnerability and insecurity among females, and the cultural and religious norms (Human Rights Watch, 2018). Moreover, research from Pakistan reported plethora of issues that make female students’ admission and study in a university challenging for them. The main issues reported by studies include, i) religious norms and values and ii) sexual harassment (Bradley and Saigol, 2012; Mahmood et al., 2018).
Given this context, Researcher-1’s male identity was the initial stumbling block. Since his study required video-recorded observations, multiple sessions of stimulated recall interviews, chances were that some of the female students might refuse to participate. Additionally, the study required the consent from female participants without a Niqab (Veil) with a purpose to observe and video-record their verbal and non-verbal communicational behaviour inside classroom. Of the three female participants who agreed to participate, one of the female participants showed reluctance for video-recordings. A separate session was held with her to address her fears and inspire confidence in her. Also a telephonic conversation was also held with her elder brother wherein the Researcher-1 introduced himself and explained the nature of his sister’s participation in the study and the use of data, especially the video-recordings. The talk with the brother of the participant enhanced her confidence and she consented for the video-recordings. Another problem was obtaining permission from other female students in the class, some of whom wore a veil while others had headscarves. The class teacher was a great help in ensuring female students of the transparency and confidentiality of the data. For these female students' convenience, the camera was installed at a place where it could focus the participants of the study.
In case of the second study, the female participants did not have the gender gap and, therefore, no excuse for not participating in the study based on the researcher’s gender. However, despite that, there was very little voluntary participation from the female students as only 5 of the twenty participants in the study were female. Researcher-2 tried to contact some more through repeated invitations, however, they refused to participate. When asked the reason, they said their final exams were close and they did not have time to do anything other than studying. Thus, the pressure of exams on students and timing of data collection around the exams in the university affected female students’ voluntary participation. Quite surprisingly though, none of them mentioned their parents’ permission or hesitation as barriers to their participation. However, since the students did not know the researcher-2 at all, it was easy for them to not turn up for the interview on the given date and time. This put the research scholars in a difficult situation since they had to find a replacement for these students, and it also delayed the completion of the data collection process.
Translation issues
Even though the context was the department of English and English is an official language in the context of both the studies, not every participant was an equally proficient user of English. The interviews, in both studies, were conducted in Sindhi, Urdu (the provincial and national languages) as well as English depending on the language the participants chose to speak in. Using three different languages, Sindhi, Urdu and English in the interviews while presenting the findings in English makes these studies cross-language research, as defined by Temple (2002). Even though the researchers spoke the same language as the majority of the participants, speakers of more than one language were involved and there was the need of translation. This posed challenges in translation of the constructs during data collection and interpretation of data during analysis (Birbili, 2000; Temple and Young, 2004). First challenge was making authentic translation related decisions (Fersch, 2013). Temple (1997) suggests that preference should be given to conceptual decisions rather than focusing on lexical equivalence. So, when we came across such decisions I focused on the meaning. For example, one of the participants in Researcher-2’s study used Siraiki words in his interview which is a language very close to Sindh. In this case, the researcher focused on the concept rather than lexical equivalence.
At times, there was no exact word to translate terms important to the research, as in the case of “willingness to communicate” in Researcher-1’s study. Three English language teachers speaking Sindhi and Urdu as their first languages were consulted and asked for help in translation of the questionnaires involving the word “willingness”. “Willingness” in English involves both ‘volition’ (razakarana/razaamandi se) and ‘readiness’ (tayyaar) to communicate; in Sindhi, however, it does not have a one-word equivalent that could encompass both readiness and volition; rather it covers only volition but misses out on the readiness component. It was, therefore, difficult to translate the open-ended questions for participants. A near equivalent of willingness in Urdu (also occasionally used in Sindhi) is ‘aamaadah’ meaning accepting/agreeing voluntarily to do something. However, its usage made the sentences complicated and difficult to understand. Therefore, it was decided that students will be explained the concept of willingness and the English word will be used throughout.
Special consideration needed to be given to the time of translation- before or after data collection as Santos et al. (2015) argue that it affects the quality of data. Before we went for data collection, we translated all the interview questions in Sindhi and Urdu. The interview data, including student responses, however, were obviously transcribed and translated after the data collection. Thirdly, piloting in different languages needed to be considered (Squires, 2009). The interview questions were piloted in Sindhi and English when piloting for time with participants in York. This helped check and verify my translations against those of peers who were also doing their PhDs here. For example, the word “identity” was important in the citizenship study, and can be translated as “
” but that has a slightly different connotation and the Urdu words “
” or “
” used commonly by Sindhi speakers are a better match to “identity”. In such cases, the Researcher-2 talked to other multilingual speakers in order to reach the final decision. After consulting with other multilingual speakers, it was decided to use “identity”, rather than “,” “
” or “
” with all students so as to retain the meaning of the word.
In case of the translation of willingness, during piloting of the instruments, it was revealed that few students mistook the word for ‘would like to’ and responded the questions accordingly. It was found that the word ‘amaadah’ elicited their trait-like willingness, while it failed to tap into their state-like willingness. Moreover, some of the Sindhi-speaking students found the word “amaadah” difficult to understand. Having observed these problems, it was decided for the main study to have brief training sessions with students elaborating on the definition of the word willingness, and explaining the distinction between desire and willingness. Research into L2 WTC has made a clear distinction between the two constructs, that is the desire to communicate and willingness to communicate; the desire to communicate might be a person’s wish to talk to a specific person (Wen and Clement, 2003) and therefore, it was essential that the distinction be made clear to the participants. It is important to note that it was possible to accomplish this simply because the study was a case study involving small number of participants (n = 6). In studies involving a large cross-section of population, researchers will need to (i) write the definition of the construct and the difference between desire and willingness on the top of the questionnaire; (ii) elaborate and explain the constructs while administering the questionnaires, and/or doing interviews.
Lastly, translation needed to be authentic and be verified to retain the reliability of data. The researchers’ native language is Sindhi but having grown up in a mixed Sindhi and Urdu speaking population, we perceive ourselves to have a near-native proficiency in Urdu as well. We have also learnt English from a very young age and felt comfortable and confident translating the interviews ourselves. Birbili (2000) suggests that the researcher’s knowledge of the people and context helps in this situation. Being from the same context as the participants helped understand the participants. However, we did not solely rely on our interpretation. Help with translation decisions was taken from other multilingual speakers who had conducted cross language research in the same context in order to make a reliable decision. Peer translation for part of the interview data was also carried out in order to ensure that my translation is valid and meaningful. English language teachers were referred to in case of study on L2 WTC while the study on citizenship asked a native Sindhi speaker, who is a proficient speaker of English as well to partly translate one of the interviews. The translations matched in meaning and context and thus, the translation was considered reliable.
Conclusion
This note shows that the main issues in conducting research in Pakistan include getting through the gatekeepers, getting across participants’ attitudinal issues, obtaining consent from female participants for video-graphic data, and translation issues. Therefore, it is necessary for novice researchers from Pakistan, and elsewhere, who are planning to conduct research in a university in Pakistan, to familiarise themselves with hierarchy of gatekeeping in each university context. It is advisable to select a university context wherein the researchers have personal and influential contacts who could mediate between gatekeepers and researchers. Secondly, while it is better to plan a study and contact the gatekeepers well in advance, things might not go as planned during actual field work. It is, therefore, necessary for researchers to make a realistic timeline for data collection with some time for contingencies, and patience for dealing with teachers’ delaying tactics in giving consent for classroom observations, and attitude interviews. Notably, in studies with a complex mixed-method research design that demand longitudinal commitment on the part of students, it is necessary to approach students directly in person to build rapport with them and obtain consent for participation.
Most importantly, in a study with video-observations as the main data collection source, researchers need to be careful in sampling female participants because students vary in terms of their beliefs and outlook It is necessary to be fair in sampling and if students with a veil have agreed to participate in the study but have reservations regarding video-recorded observations, it is important for the researcher to try to build their confidence by ensuring the confidentiality and anonymity of the data, and if possible, talk to their parents or guardians, or use some mutual relations, to obtain their consent. In cross-language research studies it is important that the researchers be aware of translation issues and seek opinion of other researchers with similar language competence. Being aware of the possible challenges and preparing for them accordingly can help novice researchers gather richer data in considerably lesser time.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
