Framing Experience: Concept Maps,Mind Maps,and Data Collection in Qualitative Research

Introduction

Qualitative research provides an interpretation of the social world of research participants by focusing on their “experiences, perspectives and histories” (Ritchie & Lewis, 2003, p. 3) and thus privileges their constructed realities when reporting social science research findings. In addition to more traditional qualitative research approaches, in the past decade new approaches have sought to explore the utility of technological packages that can manage the qualitative data analysis process (Fielding & Lee, 1998; Gibbs, Friese, & Mangabeira, 2002). New perspectives have also emerged that consider the challenges posed by partnerships with quantitative approaches through the construction of mixed methodologies (Johnson & Onwuegbuzie, 2004; Poole & Davis, 2006). Nevertheless, a long-standing concern in qualitative research is the role of the researcher in assigning value to one of what may be many possible meaningful interpretations of the same data (Guba & Lincoln, 1989). To address this concern, qualitative researchers study the experiences, influences, and activities of research participants while explicitly considering personal and epistemological reflexivity to acknowledge their own biases (Willig, 2001). Given the myriad of research decisions (Palys, 2003) that are made in the construction and analysis of any study, the acknowledgement of the potential for researcher bias is an important contribution to social science research. New approaches to data collection might offer another means to explore reflexive analysis within qualitative research.

In this paper we explore the utility and complications associated with using concept maps to gather qualitative data from research participants. We detail the use of maps to gather data in a pilot study of Canadian participants who worked on a legal technical assistance project in Latvia. Although concept maps have been used in social science research (Trochim, Cook, & Setze, 1994), we have built this paper on more recent scholarship (Daley, 2004) to argue that traditional definitions need to be expanded if they are to meet the open-ended and flexible foundations of qualitative research. Against strict one-size-fits-all conceptions of concept maps, we present in this paper a broader definition and suggest how the use of maps in data collection assists research participants in framing their experience in more unsolicited ways. By basing data analysis on participant-centric visual representations of experience, maps offer a unique means to ground theory within data (Glaser & Strauss, 1967) and can assist researchers in refining subsequent data collection strategies. As such, using maps might provide a middle ground in the long-standing discussion about how systematic analysis within qualitative research can unfold.

Defining maps

Concept mapping as is traditionally understood today was first referred to in the 1970s by Stewart, Van Kirk, and Rowell (1979) and subsequently developed by Novak and Gowin (1984). The latter researchers remain involved in the discussion and dissemination of the value and utility of maps, mapping techniques, and analysis. In general terms, concept mapping is a technique that can demonstrate how people visualize relationships between various concepts (Lanzing, 1996). Related to cognitive maps in psychology (Tolman, 1948), concept maps provide a visual representation of dynamic schemes of understanding within the human mind (Mls, 2004), yet some debate exists about what is and what is not a concept map (Ahlberg & Ahoranta, 2004). Traditionally, concept maps have been used in quantitative research based on strict definitions in the fields of science education, engineering, mathematics, psychology, and health, yet the potential for the wider use of maps in the social sciences might require a less rigid definition (Axelrod, 1976). Although concept maps can include labeled concepts, linking words, and clear hierarchies, they might also include other sorts of visual or graphic representation of concepts and/or propositions that attempt to convey an understanding or relationship among different concepts within a map. These might include word links, directional arrows, or just simple connectors like lines or overlapping circles (Ahlberg & Ahoranta, 2004). In Figures 1 and 2 we provide two examples of maps. The first adheres to a more traditional understanding of concept maps, as it includes clear and unique concepts, lines suggesting hierarchical relationships, and linking words.

OPEN IN VIEWER

Figure 1.

A simple concept map

OPEN IN VIEWER

Figure 2.

Free-form concept map: Where do your values come from?

The second map (Figure 2) is much more free form. Although it also contains useful data and demonstrates relationships, it is unlike the first example as it relies on overlapping circles to denote different kinds of nonhierarchical connections.

If a traditional definition is used, Figure 2 would not be considered a concept map. As it is without a clear hierarchy, linking words, or even unidirectional arrows, some might suggest that this sort of map would be better described as a mind map (Buzan 1974; Buzan & Buzan, 2000). Yet Figure 2 does identify individual concepts and suggest a relationship between them. Although it might not be appropriate to attempt to use this map alone to understand how an individual perceived the origin of his or her values, it does offer a view of individual understanding. Indeed, the way in which this map is constructed might give way to more qualitative coding schemes or assist in the development of subsequent data collection approaches, including interviews or focus groups (Wheeldon, 2007). The immediate value of this definitional flexibility is that it can greatly expand the use of maps. According to Ebener et al. (2006), concept maps offer an opportunity to assist with analysis of complex processes and can play a role in knowledge translation. In addition, because concept maps can be designed in a variety of ways, they may be important tools for qualitative researchers to organize research, reduce data, analyze themes, and present findings (Daley, 2004). As such, maps provide a valuable means to collect data from research participants in social science research projects (Trochim et al., 1994).

Concept maps in social science research

Although the use of concept mapping varies widely, how participants construct maps can demonstrate their belief in the importance and commonality between different concepts and the nature of perceived relationships (Hammersley, 1996; Jackson & Trochim, 2002). Nesbit and Adescope (2006) discovered, through selective searches of education and psychology databases, that more than 500 peer-reviewed articles have been published on the application of concept or knowledge maps. Although most of these publications came after 1997 (Nesbit & Adescope, 2006), concept maps have been used in quantitative research primarily as a means to assess knowledge integration (Besterfield-Sacre, Gerchak, Lyons, Shuman, & Wolfe, 2004). In education, they have been shown to be more effective in promoting knowledge retention than attending class lectures, reading, or participating in class discussion (Poole & Davis, 2006). Furthermore, concept maps can influence concentration and overall test performance, in part because they promote interaction and engagement between the student and material (Hall & O'Donnell, 1996). It has also been suggested that concept maps are an easier way to communicate one's knowledge than text writing (Czuchry & Dansereau, 1996). Using quantitative research designs, researchers have developed strategies to score the quality of concepts and propositions (Rye & Rubba, 2002), and identify different levels of concepts and subconcepts (Bayram, 1995) within concept maps. Other approaches rely on maps to compare experts and learners (McGaghie, McCrimmon, Mitchell, & Thompson, 2000) and develop scoring systems based on the convergence of concepts between a novice and expert mapmaker (Ruiz-Primo, Shavelson, Li, & Schultz, 2001). More recent studies rely on technology such as CMap, among others, to create defined interfaces and examine how participants consider map structure and concept relationships through the comparison of electronically created maps (Derbentseva, Safayeni, & Canas, 2007).

Perceptions of international trainers through the LLRP: Overview of the pilot study

The LLRP was a 20-month initiative funded by the Canadian International Development Agency (CIDA) that ran from 2002 to 2004. The project offered targeted legislative support, institutional capacity development, and human resources training to the Latvian Ministry of Justice as it established the State Probation Service (SPS). The LLRP is one of a number of governance projects in the Baltic countries that CIDA funded during the period from 1990 to 2004. The project is considered one example of successful governance programming in the region (Caldwell, 2005; Lukensaite, 2005). Within CIDA's government and civil society sector, thousands of experts have worked overseas on technical assistance contracts. These contracts provided training and advisory services to government agencies in developing and transition countries (CIDA, 2003). Of interest to this pilot study was how participants presented their experience as international trainers and how their experiences connected to the broader literature within the field.

As part of a graduate qualitative methods course, the research method was designed to use a two-pronged data collection technique designed to both provide a “snapshot” of a participant's experience and develop a means for the deeper consideration of experiences and perceptions (Wheeldon, 2007). This included (a) concept mapping by participants on their perceived role as trainer and (b) follow-up interviews to explore areas identified in the map and to uncover the perceived connections between their experience and their role. Following data collection, the results were analyzed based on a revision of Kvale's (1996) seven stages of doing qualitative research. This process involved identifying common themes through an amalgam of the maps and the interview transcripts. These themes were then connected to broader knowledge contributions drawn from international development, education and training, and justice reform literature.

In this study four participants were invited to complete a concept map on their role as trainer during their time in Latvia on the LLRP. Respondents were selected based on convenience and their proximity to the researcher. Participants all lived in British Columbia and were accessible by phone, fax, and e-mail. In addition, they were drawn from a variety of institutions and covered different training topics while in Latvia. In agreeing to be part of the study, all participants signed an informed consent document that outlined the nature of the research and the means of data collection. Although made aware that the data collected could be used in future assignments, papers, or publications, participants were guaranteed anonymity. In addition, the course instructor approved the project prior to the commencement of data collection, and the research was conducted under Simon Fraser University's Research Ethics Policy 20.01.

The St. Nicolas concept map (Figure 3) was provided to all participants. As an example of a traditional concept map, it was selected because it demonstrates a variety of ways in which the concept of St. Nicholas might be understood. It includes different names, physical features, activities, and additional characteristics. In addition to these features, the example concept map also includes connectors to provide participants with an illustration of how maps could be constructed. In addition to this example, the following instructions were provided to participants:

OPEN IN VIEWER

Figure 3.

Exemplar concept map of St. Nicolas

Following the completion of the map, each participant was asked to return it by fax or e-mail, and themes were identified based on the completed concept maps. Based on these themes, follow-up telephone interviews were designed. These interviews then focused on content mining, or asking participants questions “designed to explore the detail which lies within each dimension, to access meaning it holds for the interviewee, and to generate an in-depth understanding form the interviewee's point of view” (Legard, Keegan, & Ward, 2003, p. 148).

Analyzing maps: Challenges and constructions

A general challenge in qualitative research is the difficulty in analyzing the data generated (Patton, 1980). Generated qualitative data have been described variously as an “attractive nuisance” (Miles, 1979, p. 590) and an “embarrassment of riches” (Chenail, 1995, p. 2). Although concept maps can provide an interesting new tool for data collection, similar challenges remain. In Figures 4 and 5 we provide examples of the two types of maps returned in this study. There are a variety of ways a researcher might analyze concepts or constructions within concept maps. One might simply rely on “concept counting” to identify which concepts were identified by participants within the maps and how often (Turns, Atman, & Adams, 2000) or decide that the placement of the concept within the map, or the connection between and among different levels of concepts (Bayram, 1995), are more relevant. Still other approaches might involve considering how the maps were created based on the physical construction of the maps, the degree of formality involved, and any similarities or variances in the kind of concepts included among the sample. A variation of this approach was used in this study based on the maps that were returned.

OPEN IN VIEWER

Figure 4.

Example of traditional and informal map construction

OPEN IN VIEWER

Figure 5.

Example of nontraditional, more formal map construction

Participant maps in this study were analyzed based on an observation of their physical construction and the degree of formality involved in the mapping as well as analysis of the concepts identified in the maps. Although some did not meet the traditional definition of concept maps, we decided that they represented participant-centric and user-generated constructions of experience and were inherently valuable. Yet, in this study, instead of basing the analysis on the maps, we designed the interview questions to explore themes within them through an initial division of the maps by style and substance.

Using map analysis to guide follow-up questions

As might be evident from reviewing the participant maps in Figures 4 and 5, individuals completed different sorts of maps. Some were drawn by hand, whereas others were developed using a word-processing program. Other differences were also observed. The map in Figure 4 included references to personalities, relationships, public roles, and private lives, and identified trust issues as a part of their experience. The map in Figure 5 also included references to individuals and contacts but focused in more depth on agencies, specific training activities, and outcomes and results. Because of these differences, the map analysis was based on a distinction between more and less formal construction of the maps. Those who returned less formal maps including specific reference to the role of relationships and informal networks were asked specific questions about project tools and outcomes delivered by the trainers. Participants who completed more formal maps and included specific reference to project tools and outcomes were asked two additional questions about the role of relationships and informal contacts within the training sessions. In this way, although common themes could be explored in depth among all participants, each group was given a chance to explore themes emphasized by the other.

In the end, 10 structured but deliberately open-ended questions were presented to the participants through a telephone interview. They were designed to “open up the research territory and to identify the dimensions or issues that are relevant to the participant” (Legard et al., 2003, p. 148). Although eight were common questions to all participants, questions 7 and 8 were developed based on the constructed maps and subsequent analysis for the two groups as detailed in the previous section. These questions are bolded in Table 1.

OPEN IN VIEWER

Table 1.

Follow-up question sets

Question Set 1: Less Formal Maps	Question Set 2: More Formal Maps
1 Most positive experience?	Most positive experience?
2 Most negative/challenging experience?	Most negative/challenging experience?
3 Most memorable experience?	Most memorable experience?
4 Are you still in touch with the Latvians?	Are you still in touch with the Latvians?
5 Describe the concept map exercise?	Describe the concept map exercise?
6 Role of the translator?	Role of the translator?
7 Biggest project result?	Role of formal relationships?
8 Training tools developed?	Role of informal contacts?
9 Role of the government and/or community in reform process?	Role of the government and/or community in reform process?
10 Anything else you would like to add?	Anything else you would like to add?

Framing experience: Findings and discussion

These findings suggest that maps can be used in qualitative research as part of multistage data collection processes. Providing more open-ended and unsolicited data collection means that maps can help researchers to refine subsequent data collection strategies, and appear to offer a unique means of grounding theory within data (Glaser & Strauss, 1967). Although the use of maps allowed participants to represent their time abroad graphically, they also served as a useful means of recall, providing a venue to capture experiential context cues. By requiring research participants to begin by framing their experience, maps might have provided a means for participants to “unlock unique memories” of past experiences and prompted recall in ways that traditional data collection might not (Legard et al., 2003, p. 148). Some participants identified the opportunity to map their experience as allowing them to “rethink their time in Latvia” and “think about their experiences in different ways.” Others suggested that the maps were an “informative way to consider how [they] experienced a new environment.” Still others suggested that they had learned from the exercise how much they had absorbed during the workshops but also during the many other interactions that working abroad affords and described the maps as “a useful and concise reminder of their time in Latvia.”

The maps also served as a useful means of refining subsequent data collection strategies. Within the analysis, the construction of the maps was related to some of the concepts identified. In this study two different groups emerged. In one group, two of the completed maps were developed using a computer word processor and consisted of numerous boxes and connectors between the boxes. The physical construction of the maps was more formal and hierarchical, and provided a large breadth of information. Substantively, these maps focused on institutional considerations, including a specific focus on training outputs, developed tools, and project outcomes. The second group's maps were constructed quite differently. Physically, they were developed by hand, were less formal, and included fewer boxes but provided more depth, including detailed and personalized accounts of their experiences in Latvia, and focused on the role of relationships and network building. Although many common themes emerged when the maps were compared together, the division between the groups related to the physical construction of the maps and substantive detail. This provided a justification for the design of subsequent interview questions.

An interesting dichotomy emerged between the role of formal tools and informal processes within legal technical assistance. The dichotomy that appeared in both the map groups is one that also appears within organizational development literature (Rogers, 2005). Formal organizational tools are a general feature of many legal technical assistance projects (Shaw & Dandurand, 2006) and include training tools such as assessments, reports, laws, and policies. In Canada, however, it has been noted that project success within development also is related to more individual elements, such as the role of intercultural competence and relationship building (Kealey, Protheroe, MacDonald, & Vulpe, 2005). Although the maps suggested that this dichotomy existed, through subsequent data collection it became clear that for all of the participants in this study being a trainer was “all about relationships.” Common reflections focused on relationships formed both through the formal delivery of project activities and through the social exchanges and events that are an important part of Latvian culture. Through the development of personal networks between the Canadian trainers and Latvian participants, training sessions included “meaningful participation, networking, and collective problem solving.”

Of interest is that the maps provided an explicit basis for additional data collection strategies, but the subsequent themes that emerged challenged the initial analysis of the maps. Although based in literature and suggested in the construction of the maps, the dichotomy between formal tools and informal processes appeared less important within this sample. Through additional data collection a more complete picture of participant experience emerged. As revealed when probing the backstage of participant experience, it is networks and relationships, not tools or policies, that “can make or break” knowledge transfer. By allowing for broader thematic constructions to emerge through multiple stages of data collection, this approach allowed for a means to validate what appeared to be significant differences within the sample. Instead of forcing the data into existent theory, this study suggests that using caution in the initial identification of codes and categories is important. Although it might be that the outcome in this case justifies the ad hoc, iterative Straussian approach, there may be cases in which a more directive and structure approached to analysis would be favored. In these cases, the use of maps can provide a flexible and unsolicited participant-led approach to assist coding and analysis. Because any emergent codes can be tested through subsequent data collection, the use of participant maps might be seen as a unique and innovative data-gathering instrument of interest to a new generation of more visually oriented researchers.

Limitations and reflections

Although the use of maps to gather evidence is a relatively new phenomenon, they provide a useful and novel way to communicate meaning and knowledge. However, there are challenges associated with their use. In addition to the low number of participants in this study, another limitation in this study is how participants received the maps. There is evidence that important differences between people, groups, personalities, and learning styles can limit the utility of concept maps in gathering data (Rohm, 1994). For example, Czuchry and Dansereau (1996) found that women identified the mapping assignment to be easier than did men. In this study, gender differences were not identified as relevant in either the form or substance of the maps. Some participants did suggest, however, that they found the initial request for a map “odd,” and the process of creating a map “strange.”

Another limitation is based on the definitional confusion that exists in studies and discussions of concept maps and mind maps. Often, it appears, these two distinct types of maps are combined and referred to as though they are the same (Nesbit & Adescope, 2006). Although we have argued in this paper that one solution might be to expand the definition of concept maps to include a wide variety of visual representations of experience, this solution might be difficult for leaders in the field to accept. It appears that some researchers continue to contend that concept maps must be narrowly construed, formally structured, and hierarchically organized (Novak & Cañas, 2008). These requirements might be at odds with a focus on precision, credibility, and transferability as they relate to capturing individual experiences and perceptions in qualitative research (Tattersall, Watts, & Vernon, 2007). In this study, although participants were provided a concept map exemplar as part of data collection, the maps they returned did not meet the strict criteria of traditional concept maps. Without an acceptance of definitional elasticity regarding this term, it might be better to refer to the maps that were returned as mind maps. Although there is less specific research on mind maps when compared to concept maps (Farrand, Hussain, & Hennessy, 2002; Pressley, VanEtten, Yokoi, Freebern, & VanMeter, 1998), the focus on meaning and participant-centric data collection in qualitative research might require that researchers using maps to gather data ensure make room for user-generated and unsolicited reflections (Wheeldon, 2009).

A related limitation speaks directly to the means of analysis. A variety of options exist for researchers who seek to use participant-completed concept maps to guide further data collection. In some studies, the fact that the maps we received did not conform to more traditional definitions of concept maps could have led to an abandonment of the project entirely. In this study, another approach was employed. Returned maps were divided by style and substance, and subsequent interview questions were developed based on larger themes within the overall maps. So that we could get as complete a picture as possible, those who returned less formal maps were asked specific questions about project tools and outcomes delivered by the trainers, and participants who completed more formal maps were asked questions about the role of relationships and informal contacts within the training sessions. Although this procedure is defensible, another approach would have been to treat the absence of detail in the maps as a finding, and design follow-up questions based on relevant literature. This approach was not employed. As Tomas (1997) has suggested, “recall of experience is always selective and there will be many absences or gaps. People forget things or choose not to tell things or are not aware of things—for all sorts of reasons” (p. 75).

Conclusion

Although the use of concept maps and mind maps appears to be on the rise (Nesbit & Adescope, 2006), questions remain about how these maps are defined and differentiated, and how they can be used by researchers to identify themes and ultimately interpret meaning. In this paper we have presented the utility of mapping as a means of gathering data from research participants. Maps offer a means of gathering more unsolicited reflections, providing a visual snapshot of experience from which to ground theory within data, and thus can help researchers to refine subsequent data collection strategies. Although unlike some traditional requests for data in qualitative research, maps might improve the validity of the data collected by forcing participants out of practiced scripts and narratives. Maps can provide a visual means for people to share their experiences and perspectives in new and unique ways.

One challenge identified in this study is how to treat maps that did not conform to the traditional requirements of concept maps. By retaining the essential exploratory character of the research, we sought in this pilot study to explore in more depth the existing concepts identified by the total sample without challenging each individual's choice of mapmaking. However more studies, reflections, and deliberations are needed to consider how and why different approaches to maps and mapping might assist qualitative researchers. It appears that despite the complexity inherent in the exploration of any novel means of data collection, maps can provide an opportunity to demystify the analysis process. By offering a clear graphic snapshot of individual perception, researchers can ground any theoretical contributions in the visual representations of experience that participants create.