Advancing Qualitative Research Using Qualitative Data Analysis Software (QDAS)? Reviewing Potential Versus Practice in Published Studies using ATLAS.ti and NVivo,1994

Abstract

Qualitative data analysis software (QDAS) programs are well-established research tools, but little is known about how researchers use them. This article reports the results of a content analysis of 763 empirical articles, published in the Scopus database between 1994 and 2013, which explored how researchers use the ATLAS.ti™ and NVivo™ QDAS programs.* The analysis specifically investigated who is using these tools (in terms of subject discipline and author country of origin), and how they are being used to support research (in terms of type of data, type of study, and phase of the research process that QDAS were used to support). The study found that the number of articles reporting QDAS is increasing each year, and that the majority of studies using ATLAS.ti™ and NVivo™ were published in health sciences journals by authors from the United Kingdom, United States, Netherlands, Canada, and Australia. Researchers used QDAS to support a variety of research designs and most commonly used the programs to support analyses of data gathered through interviews, focus groups, documents, field notes, and open-ended survey questions. Although QDAS can support multiple phases of the research process, the study found the vast majority of researchers are using it for data management and analysis, with fewer using it for data collection/creation or to visually display their methods and findings. This article concludes with some discussion of the extent to which QDAS users appear to have leveraged the potential of these programs to support new approaches to research.

Keywords

qualitative data analysis software CAQDAS QDAS ATLAS.ti™NVivo™qualitative research literature review

Introduction

In the 25 years since qualitative data analysis software (QDAS) programs were developed, the use of such tools has been lauded as both a positive and worrying development for the field of qualitative research. When the first generation of QDAS programs was developed in the 1980s, researchers speculated that such tools could potentially advance qualitative research in several key ways. The first was by extending paper-based techniques for coding, retrieving, and analyzing data “beyond the feasible manual limits” (Richards & Richards, 1987, p. 29) by enabling more complex, adaptable, extensive, and exhaustive coding schemes (Podolefsky, 1987; Richards & Richards, 1987; Tallerico, 1991), easier, more efficient and more accurate retrieval of coded data for analysis (Hesse-Biber, Dupois, & Kinder, 1991; Richards & Richards, 1987; Tallerico, 1991; Shelly & Sibert, 1986) and better testing of the qualitative evidence for emerging theoretical propositions and hypotheses (Hesse-Biber et al., 1991). The second possibility was that QDAS could improve qualitative analysis and interpretation by supporting forms of analyses that “would be impossible to carry out manually” (Richards & Richards, 1991, p. 319), such as by enabling the “linking of text, analysis and non-text materials (graphics, sounds and video) in a single analytical space outside the mind’s eye [which] is not possible manually” (Dohan & Sanchez-Jankowski, 1998, p. 484). The third possibility was that by enabling more transparent analytical processes, QDAS could enable validity, rigor and trustworthiness to be more readily demonstrated (Dainty, Bagilhole, & Neale, 1997; Fritz, 1990; Morison & Moir, 1998) and advance understanding of its practical application, usefulness, and limitations (Blismas & Dainty, 2003; Bringer, Johnston, & Brackenridge, 2006). This in turn could help others avoid common mistakes (Sin, 2007) and help the qualitative research community identify how to best guide new QDAS users in avoiding pitfalls and problems in qualitative research (Mangabeira, Lee, & Fielding, 2004).

But the development of QDAS programs has also prompted concerns that the tools can influence qualitative research in undesirable ways. Arguably the most enduring worry is that their development for specific research contexts, such as grounded theory, might impose a methodological “straight jacket” around research activities (Holbrook & Butcher, 1996, p. 60). More narrowly, that the technological parameters of QDAS programs can compromise the researchers’ ability to design and execute research tailored to the needs of their projects is another concern (DeNardo & Levers, 2002). This could occur if researchers design their studies around the capabilities of software (Holland, 2002), defer to program requirements (Gilbert, 2002), adopt “programmatic” approaches to analysis (Morison & Moir, 1998, p. 114), or use a technique simply because the software allows it (Garcia-Horta & Guerra-Ramos, 2009). This also suggests that software can dominate the researchers’ understanding of their practices (Bryman & Burgess, 1994; MacMillan & Koenig, 2004; Séror, 2005), especially if QDAS are used without a critical and reflexive awareness of how the software influences qualitative research practices (Brown, 2002; Woods, Macklin, & Lewis, 2015).

Despite these hopes and fears, very few empirical studies have investigated how researchers actually use and experience QDAS tools (see Fielding & Lee, 1998; Gilbert, 2002; Mangabeira et al., 2004; Marshall, 2002 for exceptions). Some scholars have examined the forms of analyses that can be executed and have compared analyses using different programs (e.g., Evers, Silver, Mruck, & Peeters, 2011; Hutchison, Johnston, & Breckon, 2010; MacMillan, 2005; Woods & Dempster, 2011), but such accounts primarily report the experiences of individual researchers and teams. Little is known about how widely QDAS are used and what they are used for.

Greater understanding of how researchers use QDAS, and the implications of that usage, is needed to determine whether QDAS programs “really provide a panacea to the analysis of qualitative data” (Blismas & Dainty, 2003, p. 462) and/or have led to the feared problems. Better understanding of QDAS usage would also advance discourses about their implications, and thus inform decision making around and training in the use of such tools.

This article reports findings from a large-scale review of empirical studies undertaken to determine how QDAS are being used and reported by qualitative researchers. This article reports our specific findings about who QDAS are being used by, and how QDAS are being used: Our findings about how researchers are reporting this usage are detailed elsewhere (Paulus, Woods, Atkins & Macklin, Under Review).

Method

We conducted a qualitative content analysis (Finfgeld-Connett, 2014) of 763 empirical studies using QDAS published in peer-reviewed journals between 1994 and 2013. We selected 1994 as it was the year the Computer-Assisted Qualitative Data Analysis Software (CAQDAS) Networking Project at the University of Surrey was established as a focal point for discussions about QDAS programs. We specifically focused on studies using ATLAS.ti™ or NVivo™/NUD*IST™ (NUD*IST evolved into NVivo™ in 2007), because they are two of the longest used QDAS tools (Muhr, 1991; Richards & Richards, 1991). They are also the programs that we ourselves our familiar with; without this familiarity of our analysis would not have been possible. Consistent with techniques for systematic literature searching (Bandara, Miskon, & Fielt, 2011), we defined the search strategies for identifying and extracting relevant articles, including key words, sources and databases. We generated a data set using Scopus (Elsevier), a broad-based, multidisciplinary journal citation database to give us a comprehensive collection of peer-reviewed journal articles. Scopus was chosen over Web of Science (Thomson Reuters) and Academic Search Premier (EBSCO) because our evaluation of all three determined that Scopus offered comparable accessibility, multidisciplinarity, and data set size, but superior post-search analytics and citation download formats: each Scopus citation contains a URL for the full article.

To build our search set, we used Scopus to simultaneously search for citations by title, abstract, and key word. Using the words “atlas” and “nudist” retrieved too many articles that had nothing to do with QDAS. We thus limited our searching to proper program names (ATLAS.ti™, NVivo™, NUD*IST™). We then narrowed these search sets to peer-reviewed journal articles published in English. We chose to analyze peer-reviewed journal articles because they are arguably the highest quality academic publications. We chose English language publications because this is the dominant language of the research team. Finally, we concentrated exclusively on empirical rather than methodological articles because our focus was specifically on how researchers are incorporating QDAS into their research practices and reporting. As Scopus does not support filtering using these characteristics, we had to do this manually. We determined if an article was “empirical” or “methodological” when coding each article.

These strategies produced a final data set of 763 articles: 349 studies used ATLAS.ti™ and 414 studies used NVivo™/NUD*IST™. Our final data set heavily represented researchers from the health sciences fields, which we attribute to a possible subject-discipline bias within the Scopus database. Elsevier reports that Scopus includes 100% of the journal citations found in Medline, the premier health sciences database. Elsevier also categorizes 33% of Scopus’s 19,400 indexed peer-reviewed journals (approximately 6,400 titles) as health sciences journals. Arts, Humanities, and Social Sciences (AHSS) journals account for 20% of Scopus journal title coverage. In 2012, Scopus expanded its AHSS journal coverage to 4,000 peer-reviewed journals. However, at that time, 20% of these AHSS titles were not indexed before 2002 (Elsevier, 2014).

We analyzed the articles to determine the characteristics of the authors and how the authors reported using ATLAS.ti™ or NVivo™ in their research. Consistent with Bandara et al.’s (2011) recommendations, we developed and tested a priori coding schemes and protocols. This included defining the codes to be used, the data to be captured by each code, whether text would be coded by line, sentence, or paragraph, and how coder observations would be captured. Our coding scheme used three sets of codes: bibliographic codes, application codes, and coder’s reflections (see Table 1 for the complete set of codes used in the analysis). Our Scopus database searches generated basic bibliographic data, which was automatically populated into a spreadsheet. We then used a predefined code set for manually recording bibliographic codes which included the corresponding author’s country of origin and subject discipline of the journal.

Table 1.

Final Set of Codes Used to Classify Publications.

Bibliographic Codes		Application Codes			Coder’s Thoughts and Reflections
Journal discipline^a	Country of corresponding author	Type of studies	Types of data	Phase of research process using QDAS	Other points of interest
Coding rule: Code for discipline using key words in journal title, supplemented with information located at publishers’ websites	Coding rule: Code for country given for corresponding author	Coding rule: Excerpted from article text based on author description	Coding rule: Code for each separate type of data mentioned in Method section	Coding rule: Excerpted from article text based on author description	Coding rule: Notes taken by coder
Codes:	Codes:	Codes:	Codes:	Codes:
Agricultural sciences Arts, language, music, and humanities Business Communication and information sciences Computer science Education Engineering and Applied sciences Health sciences Math and statistics Multidisciplinary Physical and natural sciences Social sciences Social work Other	Selected from country list taken from ISO 3166 Country Codes (http://www.iso.org/iso/country_codes)	Action or participatory action research Case study or analysis Content analysis Critical incident technique Discourse or conversation analysis Ethnography or observation study Evaluation study Feminist policy analysis Focus group Formative research or rapid assessment Generic qualitative Grounded theory Interview study Mixed methods Narrative inquiry or analysis Naturalistic inquiry Phenomenology Survey Research Thematic analysis/qualitative content analysis Verbal autopsy method	Conversational data Documents Focus groups Interview data Observational field notes Online social media data Other Survey or questionnaire Video or image data Websites	Data collection/creation Data analysis/management Data display/representation of findings Literature review

^aAdapted from Paulus, Phipps, Harrison, and Varga (2012).

Application codes were used to categorize the types of research study conducted, types of data collected/created, and phases of the research process when QDAS was used (literature review, data collection/creation, data analysis/management, and data display/representation of findings). We used the author’s description of their study as the basis for coding the types of study. For example, where the authors claimed to use grounded theory, we coded the article as “grounded theory,” without assessing or evaluating how well they implemented this research design. In cases where the authors described their research designs more generically (e.g., as an “exploratory study” or a “qualitative study”), we coded the article “generic qualitative.” We used predefined codes for data types (e.g., interview, focus groups, video), modifying the categories as the analysis proceeded. Coding of the phase of the research process relied on authors’ descriptions of their QDAS use. Any additional detail about the articles or their coding was noted as free text in the Coder’s Reflections field. This allowed us to record and compare coder reflections and clarify coding decisions when necessary.

Our analysis proceeded through two stages. Stage 1 involved developing and piloting our coding strategies by using an Excel spreadsheet to code all articles in the data set that reported using ATLAS.ti™. We used Microsoft Excel™ because all research team members were proficient in using it, it facilitated combining and reviewing team member coding, and it enabled reflection on whether subsequent analyses might be enhanced by using a QDAS program. To ensure accurate data entry and to enable filtering by code, we used techniques developed by Wickham, Dunn, and Sweeney (2012) for coding large data sets of literature using spreadsheets. We created predefined drop-down menus to ensure that we used a consistent set of codes, typographical errors didn’t compromise accurate coding, and spreadsheets could be merged and searched. We ensured intercoder consistency by pilot testing the coding strategies with a sample of 10 articles and engaging in regular discussions on emergent issues. The 10 articles were coded independently by each coder, followed by discussions to rectify any inconsistencies. Once coding had been completed, individual spreadsheets were merged into one data set.

By the end of Stage 1, we had identified two limitations to using Excel for the remainder of our coding: The spreadsheets were unwieldy when working with 300+ articles and also unwieldy when trying to review the coding for the entire dataset. During Stage 2, we completed pilot analyses of 20 of the previously coded articles—one undertaken by M.W. using NVivo™, one undertaken by T.P. using ATLAS.ti™—to determine which program better supported our analytical approach and overcame Excel™’s limitations. Comparing the outputs of the analyses, we determined that coding with NVivo™ offered the unique advantage of creating an indexing system of data categories (called “nodes” in the NVivo™ lexicon) which all four team members could code into, providing an evolving visual guide to the analysis and integrating our coding so by opening a node we could view the data in any given category in its entirety. Consequently, we adopted NVivo™ to support the project, first recoding the articles that reported ATLAS.ti™ usage and then analyzing the data set of NVivo™ articles using the same coding process detailed above. This produced a final node system that displayed the full system of categories into which the data were coded, a quantitative tally of the number of articles coded into each category and, by opening the node, the source details and coded content for each article coded to the node. The coded content was then retrieved and reviewed to produce our findings about QDAS use. The node system was used to generate Tables 2–4; Figures 1 and 2 were generated by exporting the coded data and generating charts in Excel, as NVivo™ could not support the generation of such complex graphs.

Table 2.

Top 10 Countries of Corresponding Authors.

	ATLAS.ti™	NVivo™	Total
United States	158	109	267
United Kingdom	50	108	158
Australia	2	74	76
Canada	15	31	46
Netherlands	20	9	29
South Africa	9	4	13
Italy	10	1	11
New Zealand	2	8	10
Ireland	1	7	8
Germany	8	0	8

Table 3.

Types of Research Studies Conducted With QDAS Support.

	ATLAS.ti™		NVivo™		Total
Research Type	Articles	%	Articles	%	Articles	%
Generic qualitative	29	8.3	150	36.2	179	23.5
Interview study	112	32.1	35	8.5	147	19.3
Focus group	74	21.2	40	9.7	114	14.9
Grounded theory	45	12.9	55	13.3	100	13.1
Thematic analysis or qualitative content analysis	0	0.0	63	15.2	63	8.3
Content analysis	16	4.6	32	7.7	48	6.3
Ethnography or observation study	28	8.0	16	3.9	44	5.8
Case study or analysis	19	5.4	22	5.3	41	5.4
Mixed methods	18	5.2	18	4.3	36	4.7
Phenomenology	13	3.7	18	4.3	31	4.1
Narrative inquiry or analysis	16	4.6	5	1.2	21	2.8
Survey research	13	3.7	7	1.7	20	2.6
Action research or participatory action	8	2.3	7	1.7	15	2.0
Discourse analysis or conversation analysis	5	1.4	4	1.0	9	1.2
Evaluation study	0	0.0	7	1.7	7	0.9
Formative research or rapid assessment	2	0.6	5	1.2	7	0.9
Clinical study	0	0.0	4	1.0	4	0.5
Critical incident technique	0	0.0	3	0.7	3	0.4
Feminist policy analysis	0	0.0	1	0.2	1	0.1
Naturalistic inquiry	0	0.0	1	0.2	1	0.1
Verbal autopsy method	0	0.0	1	0.2	1	0.1

Table 4.

Data Types Used.

	ATLAS.ti™		NVivo™		Total
Data Type	Articles	%	Articles	%	Articles	%
Interview data	233	66.8	326	78.7	559	73.3
Focus groups	115	33.0	64	15.5	179	23.5
Documents	37	10.6	55	13.3	92	12.1
Observational field notes	47	13.5	40	9.7	87	11.4
Survey or questionnaire	27	7.7	51	12.3	78	10.2
Video or image data	16	4.6	11	2.7	27	3.5
Conversational data	11	3.2	15	3.6	26	3.4
Online social media data	5	1.4	8	1.9	13	1.7
Other	6	1.7	5	1.2	11	1.4
Websites	1	0.3	6	1.4	7	0.9

Figure 1.

Qualitative data analysis software articles published by year, 1994–2013.

Figure 2.

Subject disciplines of journals publishing ATLAS.ti™ and NVivo™ studies.

Findings

This section reports on the users and uses of QDAS as detailed in the 763 empirical articles: 349 articles published between 2000 and 2013 reported studies using ATLAS.ti™ and 414 articles published between 1994 and 2013 reported using NVivo™. As Figure 1 illustrates, the number of articles reporting QDAS use has increased each year over the last decade. In 2012, there was a 50% increase in the numbers of articles published over the previous year. Whether these increases are due to more researchers using the programs or QDAS users being increasingly successful in publishing their studies cannot be determined from our data.

We present our findings in two parts. First, we report on who is using QDAS, as illustrated by the subject disciplines of the journals and the geographic distribution of lead authors. Second, we report on how scholars are using QDAS, as indicated by types of studies, types of data analyzed, and phases of the research process where QDAS was used.

Part 1: QDAS Users

Subject disciplines of journals publishing QDAS-supported studies

We found that studies conducted using QDAS had been published in journals from a diverse array of disciplines (see Figure 2). The majority of studies (72%) were in journals related to health sciences. Medicine, nursing, general health care, and public health/epidemiology were the four disciplinary categories most represented in our data set. Within the health sciences, more studies used NVivo™ than ATLAS.ti™.

Geographic distribution/representation of QDAS usage

We found that ATLAS.ti™ and NVivo™ use were reported by researchers from a total of 58 countries. ATLAS.ti™ users were represented in 46 countries and NVivo™ users in 37 countries. Our review’s focus on English language articles undoubtedly contributed to the heavy distribution toward English-speaking countries. As Table 2 illustrates, the top four countries for ATLAS.ti™ were the United States, United Kingdom, Netherlands, and Canada; the top four countries for NVivo™ were the United States, United Kingdom, Australia, and Canada. The relatively high number of articles using NVivo™ published by Australian authors may be partly explained by NVivo™’s origins in Australia. ATLAS.ti™ originated in Germany, but we did not find a similarly high number of studies authored by Germans. We attribute this to our focus on English language publications.

Part 2: QDAS Use

Types of studies conducted with QDAS support

Authors reported using QDAS to support many different types of studies. The vast majority (95.3%) reported using QDAS to support qualitative studies; the remaining 4.72% were mixed method studies. A finer-grained analysis of the types of research using QDAS support was complicated by authors’ descriptions. Some researchers defined types of research in terms of research traditions, for instance, by describing their research as an ethnographic study. Others described their research in terms of the types of methodology being used, most notably, grounded theory. Still others described their study in terms of the methods used, such as an “interview study” or “focus group study.” Additionally, we found that in almost a quarter of the articles, authors described their research generically, such as a “qualitative design” or simply a “qualitative study”. Consequently, our exploration of the types of research being conducted using QDAS identified 21 different types (summarized in Table 3), including a variety of methodologies, research traditions, and methods.

The high number of studies citing a generic qualitative approach or describing their research as thematic or qualitative content analysis may be partially explained by the reporting practices of researchers in the health science disciplines. They commonly described their studies using general terms such as “a qualitative study” (Mutemwa et al., 2013, p. 1), “descriptive qualitative research” (Bedos, Loignon, Landry, Allison, & Richard, 2013, p. 1), a “qualitative, explorative study using open patient interviews” (Boeckxstaens et al., 2012, p. 183), or a “qualitative longitudinal study” (Grime, Richardson, & Ong, 2010, p. 597). However, cross-tabulating our coding by type of research and journal discipline indicated that while such practices were more common among health science researchers, they were also frequently used by researchers from other disciplines, including Business, Communication and Information systems, and Engineering and Applied Sciences.

The types of studies most frequently described by researchers using ATLAS.ti™ and NVivo™ differed for each program. For ATLAS.ti™, the most frequently cited type was interviews (32.1%), followed by focus groups (21.2%), grounded theory (12.9%), generic qualitative (8.3%), and ethnographic or observational studies (8%). For NVivo™, it was generic “qualitative studies” (36.2%), thematic analysis/qualitative content analysis (15.2%), grounded theory (13.3%), focus group (9.7%) and interviews (8.5%).

Data types used in QDAS-supported studies

We also explored the types of data that researchers reported analyzing with ATLAS.ti™ or NVivo™ as illustrated in Table 4. Many studies collected more than one type of data: 24% used two types and 9% used three or more types. Nevertheless, the five most frequently cited types of data were interview data (73%), focus group data (23%), documents (12%), observational field notes (11%), and responses to open-ended survey questions (10%). While interview data were the most common type used by both ATLAS.ti™ and NVivo™ users, interview data were more heavily used with NVivo™ (78.7%) than ATLAS.ti™ (66.8%). In contrast, ATLAS.ti™ users made heavier use of focus group data than NVivo™ users (33% and 15% respectively).

The phase of the research process supported by QDAS

We found that researchers reported using ATLAS.ti™ and NVivo™ to support three phases of the research process: data collection/creation, data analysis/management, and data display/representation of findings. Data analysis/management was the most frequently mentioned use, with 99.6% of the studies using software for this purpose. No articles reported using the software to support literature reviews and only six articles reported using software for data collection/creation. Only 10.4% reported using the software for data display/representation of findings. In the following sections, we detail how researchers reported using ATLAS.ti™ and NVivo™ in each of these three phases of the research process.

Data collection/creation

Both ATLAS.ti™ and NVivo™ offer functions for creating text files and for transcribing audio and video files, making it possible for researchers to create field notes, interview notes, reflective journal entries, and interview transcripts within the software (Friese, 2014; QSR International, 2014a). In our sample, authors reported using this functionality to create study journals and memos (e.g., Jakobsen & McLaughlin, 2004) and to generate transcripts of recorded interviews (Szeinbach, Seoane-Vazquez, & Summers, 2012) and focus groups (e.g., Hawthorne et al., 2011).

Since 2008, it has been possible to use both ATLAS.ti™ and NVivo™ to code audio or video multimedia files directly, eliminating the need to transcribe recorded interviews or focus group discussions prior to analysis. However, we only identified two studies (De Gregorio, 2011; Larkin, 2009) in which researchers reported coding directly from audio or video files. In both cases, ATLAS.ti™ was used to analyze observational data recorded on video. We did not identify any studies in which the authors reported coding interview or focus group data directly from audio and video recordings. In 16 cases, this could not be determined, as the authors provided no detail as to how their interview data were captured. In six other cases, authors reported recording their interviews or focus group discussions but did not specify whether or how they analyzed the recordings or transcripts. In the main, it appears researchers are not yet leveraging the functionality in programs to code directly from multimedia files.

Data analysis/management

As noted above, the vast majority of the studies we reviewed (99.6%) reported using ATLAS.ti™ or NVivo™ for data analysis and/or data management. Authors used both terms when describing how they used their QDAS program, but their descriptions left some doubt as to what distinguished these aspects of the research process. Some authors (e.g., Hurley, 2009) used the phrases “data analysis” and “data organization” synonymously and their subsequent descriptions indicated that by data management they primarily meant coding. Others distinguished between data management and analysis (e.g., Bennett et al., 2011; Laditka et al., 2009) or data management and coding (Waldrop, 2006) but rarely provided any additional detail or explanation of how data analysis and data management were distinct or executed distinctly. The few exceptions (e.g., Barton, Sulaiman, Clarke, & Abramson, 2005; Gilliam, 2007) indicated that ‘data management’ included tasks such as “organising and preparing the data for analysis” (Chirwa, Malata, & Norr, 2011, p. 33), with Hartel (2010) providing the clearest distinction between data management as the process of managing the growing set of data records collected through their ethnographic study and data analysis as the coding of materials. These findings suggest that researchers differentiate between using QDAS tools for data management and data analysis, but more research is needed to explore the distinction between these two usages.

Turning specifically to analysis, authors commonly reported using both programs to support coding and the development of coding schema summarizing the topics or concepts represented by the data. Illustrative examples included coding schemas (see Figure 3) developed to reflect aspects of lived experience (e.g., Newman, Bogo, & Daley, 2008), barriers and facilitators of behavioral processes (e.g., Ali, Baynouna, & Bernsen, 2010), and evaluations of interventions and educational models (e.g., Moore, Morris, Crouch, & Martin, 2003).

Figure 3.

Sample illustration of coding scheme reporting themes identified in qualitative data from Newman, Bogo, and Daley (2008, p. 221).

Most authors used ATLAS.ti™ or NVivo™ to assign codes to the data and then review all the data to which a specific code has been applied. In ATLAS.ti™, this involves creating “quotations” of data and assigning a code, after which all quotations with the same code can be retrieved by running a report or viewed in context by using the code manager. Assigning a code to a data segment (e.g., paragraph of text) in NVivo™ also assigns the data to a node (data category) and thus produces two outputs: a coded data set and a node system that provides an index of the major and subsidiary categories into which the data have been coded.

Researchers reported using the programs to retrieve and review the data in various ways, such as to “retrieve quotes to dimensionalize each theme … [and] substantiate and describe the findings” (Armour, Bradshaw, & Roseborough, 2009, p. 606), enable constant comparison of newly coded and previously coded material to determine whether the same concepts are apparent (Curry, Taylor, Chen, & Bradley, 2012), facilitate “retrieval of related quotations in order to examine patterns and trends in the data” (de Villiers, Koko-Mhlahlo, & Senekal, 2005, p. 523), enable auditing of analytical process and interpretations by peers (Thongpriwan & McElmurry, 2009), construct lineal narratives (Gibson, Callery, Campbell, Hall, & Richards, 2005), and enable the grouping of codes into categories representing broader and more abstract themes (e.g., Hannes, Janssens, & Wets, 2009). For a fuller discussion of how researchers described using QDAS to support their analyses, see Paulus, Woods, Atkins, and Macklin (Under Review).

Another analytical practice identified was researchers using the QDAS program to investigate potential relationships between concepts or between concepts and participant characteristics. ATLAS.ti™ users achieved this by using the co-occurrence feature to identify any data to which multiple codes have been applied. For example, O’Halloran (2011) used co-occurrences to compare and contrast the types of discussions that occurred in different types of reading groups (see Figure 4).

Figure 4.

Distribution of discussion types across reading groups (O’Halloran, 2011, p. 182).

Researchers using NVivo™ reported running a matrix coding query that identifies co-occurrences in the data fitting specified criteria and then reports it in a table format. Researchers determine the variables of interest by specifying the content of rows and columns and the Boolean search term that connects the data, enabling the development and testing of patterns in the data. Our study identified that researchers used matrices to investigate relationships in two ways. The first was to illustrate the distribution of data across the sample of cases or participants. The second approach uses matrices to determine that nodes represent discrete concepts. Bastos, Warson, and Barbour (2012), for example, use a two-part approach to explore whether any data occurred in more than one node and might indicate conceptual overlaps. They first generated matrices to compare nodes to identify any instances where the same terms came up more than once, thereby suggesting the concepts may overlap. They then generated a second set of matrices comparing nodes that did not overlap. Although ATLAS.ti™ can be used to generate a codes-primary documents table, which is somewhat similar to an NVivo™ coding matrix, no authors reported using this functionality.

Finally, our study found that researchers specifically reported using QDAS to support collaborative team-based analyses in which several researchers were involved in analyzing a data set. Collaborative analytical approaches primarily involved multiple coders, but there were several variations in how this was done. For some research teams, one researcher was primarily responsible for coding with other team members coding a sample to check for accuracy and validity. More commonly, collaborative approaches involved multiple analysts coding the same data set and then integrating their coding. Some teams also reported using collaborative analysis to inductively develop codes and coding rules that they then used to analyze the remainder of the data.

The current versions of ATLAS.ti™ (v. 7) and NVivo™ (v. 10) support collaborative analyses in a variety of ways. Both record and track which researchers have added codes to the data, thus making it possible to identify coders later when files are merged. NVivo™ also offers a “coding comparison” function to check the consistency and interrater reliability of two researchers’ coding. It calculates percentage agreement as well as a κ coefficient (QSR International, 2014b). However, only 7.1% of the articles explicitly discussed using software features to support collaboration but broad statements such as “coding was conducted by three independent analysts and checked for consistency using ATLAS.ti™ qualitative software” (Kennedy, Grant, Walton, & Sandall, 2013, p. 139) meant it was often unclear which software features they used. Some, such as Pilling et al. (2010) reported using QDAS to keep an audit trail of the analysis. Most referred to using the software to enable constant comparison of coded and uncoded material thereby facilitating intercoder consistency over time. Some authors did provide specific detail as to how this was done, with Kirchhoff et al. (2013, p. 379) providing one of the most detailed explanations that:

Two members of the research team coded all data independently using NVivo 8. At each analysis phase, the two coders compared their results to confirm intercoder reliability (final κ00.88), resolving discrepancies through discussion with the principal investigator and comparison of the raw data.

Data display/representation of findings

Both NVivo™ and ATLAS.ti™ offer functions for visually displaying data and research findings. However, only 10.4% of the studies reported using the software for this purpose. Of these, most used screen shots to illustrate coding processes, or program outputs that visually depicted coding/conceptual schemas and relationships between data and codes. One approach was to use NVivo™’s coding matrices or ATLAS.ti™’s co-occurrence tables to generate tables and charts of code distributions (see Figure 5).

Figure 5.

Tables and charts generated with NVivo’s coding matrix function to illustrate linkages between expanded scope of analysis and project management elements (Nair, Malhotra, & Ahire, 2011, p. 546).

Another practice was to use ATLAS.ti™’s network view (see Figure 6) or NVivo™’s modeling tool (see Figure 7) to illustrate relationships between codes, categories, and concepts, and to theorize conceptual relationships.

Figure 6.

Nursing students’ positive characterizations of the clinical seminar experience produced with ATLAS.ti™ (Granero-Molina et al., 2012, p. 444).

Figure 7.

Schematic representation of how working capital is managed in small business developed with NVivo (Orobia, Byabashaija, Munene, Sejjaaka, & Musinguzi, 2013, p. 139).

At times, we had difficulty ascertaining whether or not the visual displays included in the published articles were generated using the software. Researchers used QDAS to produce two types of output for illustrating analytical process or conclusions: “native” outputs and “hybrid” outputs. Native outputs were generated with the software and reported in the format produced by the program (such as the network view from ATLAS.ti™ in Figure 6). Examples included screen shots of data categorization systems or exported versions of tables generated to illustrate coding distributions. Hybrid outputs were figures that authors reported generating with the software but which had then been adapted or converted into other formats for inclusion in the article. Examples included tables of coding distributions generated using the software but then reformatted using Microsoft Word. Native outputs clearly demonstrate that QDAS programs are being used to illustrate analytical processes and conclusions, but hybrid outputs leave this unclear so it may be that researchers are making more use of QDAS to illustrate their analytical processes than our study could determine.

Discussion

This study was undertaken to determine how researchers are using ATLAS.ti™ and NVivo™ in empirical studies as reported in peer-reviewed journals. Our specific objective was to determine both who is currently using QDAS and how these researchers are leveraging the features, functionality, and methodological opportunities offered by the programs. Our analysis found that the programs are being used by researchers in a wide range of geographic and disciplinary areas and are primarily being used to analyze textual data from interviews, focus groups, documents, field notes, and open-ended survey responses. We found that researchers claimed to use QDAS to conduct many different types of studies, indicating that such tools are used to support a diverse array of research methods, methodologies, and analytical approaches. However, as many authors described their studies generically as “qualitative studies” rather than as studies adopting a specific research methodology, it is possible that actual usage may embrace an even wider range of research types than we could identify through our study.

Our investigation of the ways in which researchers used ATLAS.ti™ and NVivo™ provides empirical evidence that researchers are using QDAS to engage in analytical practices extending beyond the limits of manual/paper-based techniques, most notably to support coding and retrieval of data, differentiate coded data by participant characteristics, and investigate conceptual relationships. We also found some evidence that researchers are using QDAS to make their analytical processes more transparent, primarily by using program outputs to illustrate their coding processes and research outputs. However, our finding that only about 10% of the studies used program outputs for this purposes suggests there is more potential for researchers to do so. We hope that by reporting how researchers are using screen shots and other program outputs, we are providing insights into how it can be done.

We found little evidence of researchers leveraging QDAS to analyze new forms of data or adapting their research practices to leverage new program features. In recent years, new features have been added to both programs enabling transcription of multimedia files and supporting direct analysis of multimedia data, social media data, geodata, and survey data sets. We thought these features may have prompted researchers to analyze multimedia recordings directly, rather than first transcribing them. This would eliminate the need for and costs of transcription (Evers, 2011). However, we found little evidence of QDAS being used to analyze these new forms of data. We acknowledge that it takes time for researchers to adopt these programs, incorporate them into their research practices, and then publish their accounts of using the tools in their research which may create a time lag in doing so. It is also possible, as one of our reviewers suggested, that researchers have tried using QDAS in these ways but reverted to previous practices because they found the process too cumbersome. It may also be that researchers are using QDAS to develop new analytical techniques but are not reporting this because of space constraints, or because researchers consider such techniques part of their intellectual property. Alternatively, they may be reporting them in methodological articles, which our study did not examine.

Researchers are also not yet reporting the use of QDAS for literature reviews. This was unexpected, given reviewing literature is arguably the most universally utilized form of qualitative research, and the value of QDAS tools for this task has been acknowledged within the methodological literature (Beekhuyzen, 2007; Di Gregorio, 2000; Paulus, Lester, & Dempster, 2014) and promoted by program developers. It may be that researchers are using QDAS for this purpose but not reporting it. As noted in our Method section, we tested the utility of both ATLAS.ti™ and NVivo™ and found that both programs had utility for supporting systematic meta-reviews of literature such as this one. Hopefully, researchers experimenting with of using QDAS to review literature will encourage others to consider how they can leverage QDAS in this way.

Our research did not identify any examples of researchers using QDAS to develop “qualitatively new ways of doing things” (Bourdon, 2002, p. 7) or analytical approaches “which can truly be described as new methodologies in their own right” (Cousins & Macintosh, 2005, p. 597) but that may be due to the focus and scope of our study. Our review only examined English-language empirical studies that used ATLAS.ti™ or NVivo™ and were indexed in the Scopus database. Future studies on QDAS use that examine non-English publications, publications in other databases, or other QDAS tools may determine whether researchers are using the tools in other ways. It is possible that methodological innovations are being reported elsewhere, such as in conference papers, methodological articles, or user blogs. Future studies that explore how QDAS use is reported in these other forums could provide additional insights into the ways in which QDAS tools can advance qualitative research. Another possibility is that QDAS innovations are occurring in nonacademic settings (Mangabeira et al., 2004). Accounts from researchers in construction management (Dainty et al., 1997) and market research (Catterall & Maclaren, 1998) have demonstrated the pragmatic value of QDAS for conducting analyses efficiently when dealing with time constraints and funding pressures. Examining nonacademic usages of QDAS could provide insights for academic users into the ways in which these tools could overcome such constraints. Future studies could also explore QDAS-specific analytical techniques such as the use of auto coding/word searching strategies and the use of visualizations to see whether and how researchers use them to generate new analytical approaches.

Future studies could also explore the ways in which disciplinary norms might be influencing the QDAS usage and other research practices identified in our study. We found, for instance, that 86% of the articles where authors reported using generic “qualitative” approaches and 85% of the articles where authors claimed to be conducting thematic analysis or qualitative content analysis were published in health sciences journals. Examining how researchers are trained to use and describe QDAS programs through, for example, tertiary education curricula and the CAQDAS Network training programs may offer some explanation for the different usages and reporting practices identified in this study. We thank our anonymous reviewer for this suggestion.

Footnotes

Authors' Note

*A short abstract broadly detailing our preliminary findings about Atlas usage was previously published in the Proceedings of the 2013 Atlas Users conference,: Fostering dialog on Qualitative Methods, University of Technology Berlin, available at . Partial findings were also presented at the 25 years of CAQDAS conference, University of Surrey, England in May 2014, and at the Eleventh International Congress of Qualitative Inquiry in Champaign-Urbana, Illinois in May 2015, and have been significantly expanded and extended in this article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors would like to thank the University of Tasmania and the University of Tennessee for providing funding support for this study. Author 1 received study leave funding to visit authors 2 and 3 at the University of Tennessee to undertake the first stage of data analysis. Both institutions also provided visiting scholar funding for Authors 2 and 3 to travel to the University of Tasmania to complete the data analysis and develop our preliminary findings.

References

Ali

H. I.

Baynouna

L. M.

Bernsen

R. M.

(2010). Barriers and facilitators of weight management: perspectives of Arab women at risk for type 2 diabetes. Health and Social Care in the Community, 18, 219–228.

Armour

M. P.

Bradshaw

Roseborough

(2009). African-Americans and recovery from severe mental illness. Social Work in Mental Health, 7, 602–622.

Bandara

Miskon

Fielt

(2011). A systematic, tool-supported method for conducting literature reviews in information systems. ECIS 2011 Proceedings. Paper 221. http://aisel.aisnet.org/ecis2011/221

Barton

Sulaiman

Clarke

Abramson

(2005). Experiences of Australian parents caring for children with asthma: It gets easier. Chronic Illness, 1, 303–314.

Bastos

Warson

Barbour

(2012). Exploring American Indian adolescents’ needs through a community-driven study. The Arts in Psychotherapy, 39, 134–142.

Bedos

Loignon

Landry

Allison

P. J.

Richard

(2013). How health professionals perceive and experience treating people on social assistance: A qualitative study among dentists in Montreal, Canada. BMC Health Services Research, 13, 464–473.

Beekhuyzen

(2007). Putting the pieces of the puzzle together: Using Nvivo for a literature review. In Proceedings for the 4th QUALIT Conference Qualitative Research in IT & IT in Qualitative Research. Wellington, New Zealand, Victoria University of Wellington, 19-20 November.

Bennett

W. L.

Ennen

C. S.

Carrese

J. A.

Hill-Briggs

Levine

D. M.

Nicholson

W. K.

Clark

J. M.

(2011). Barriers to and facilitators of postpartum follow-up care in women with recent gestational diabetes mellitus: A qualitative study. Journal of Women’s Health, 20, 239–245.

Blismas

N. G.

Dainty

A. R. J.

(2003). Computer-aided qualitative data analysis: Panacea or paradox? Building Research & Information, 31, 455–463.

10.

Boeckxstaens

Deregt

Vandesype

Willems

Brusselle

De Sutter

(2012). Chronic obstructive pulmonary disease and comorbidities through the eyes of the patient. Chronic Respiratory Disease, 9, 183–191.

11.

Bourdon

(2002). The integration of qualitative data analysis software in research strategies: Resistances and possibilities. FORUM: Qualitative Social Research, 3, 2, Art. 11. Retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/850/1846/

12.

Bringer

J. D.

Johnston

L. H.

Brackenridge

C. H.

(2006). Using computer-assisted qualitative data analysis software to develop a grounded theory project. Field Methods, 18, 245–266.

13.

Brown

(2002). Going digital and staying qualitative: Some alternative strategies for digitizing the qualitative research process. FORUM: Qualitative Social Research, 3, 2, Art. 12. Retrieved from http://www.qualitative-research.net/fqs/fqs-eng.htm/

14.

Bryman

R. G

Burgess

. (1994). Reflections on qualitative data analysis. In Bryman

Burgess

R. G.

(Eds.), Analyzing qualitative data (pp. 216–226). London, England: Routledge.

15.

Catterall

Maclaren

(1998). Using computer software for the analysis of qualitative market research data. Journal of the Market Research Society, 40, 207–222.

16.

Chirwa

Malata

Norr

(2011). HIV prevention awareness and practices among married couples in Malawi. Malawi Medical Journal, 23, 32–37.

17.

Cousins

Mcintosh

(2005). More than typewriters, more than adding machines: Integrating information technology into political research. Quality & Quantity, 39, 581–614.

18.

Curry

Taylor

Chen

P. G.

Bradley

(2012). Experiences of leadership in health care in sub-Saharan Africa. Human Resources for Health, 10, 33–41.

19.

Dainty

A. R. J.

Bagihole

B. M.

Neale

R. H.

(1997). Analytical strategies for dealing with qualitative data in construction management research. In Stephenson

(Ed.), 13th annual association of researchers in construction management conference (pp. 491–500). Cambridge, MA: King’s College.

20.

De Gregorio

(2011). Dynamics of a robbery: Criminological aspects, security issues and prevention—An exploratory study. Police Practice and Research, 12, 253–264.

21.

Di Gregorio

(2000). Using Nvivo for your literature review. Paper presented at Strategies in Qualitative Research: Issues and Results from Analysis Using QSR Nvivo and NUD*IST conference, Institute of Education, London.

22.

De Villiers

Koko-Mhlahlo

Seneka

(2005). Nutritional well-being of young children in Duncan Village, East London, South Africa: Accessibility of primary health care clinics. Public Health Nutrition, 8, 520–532.

23.

DeNardo

Levers

L. L.

(2002, 6 14). Using Nvivo to analyze qualitative data. Monograph presented at the Ethnographic and Qualitative Research in Education 2002 Annual Conference, Pittsburgh, PA.

24.

Dohan

Sanchez-Jankowski

(1998). Using computers to analyze ethnographic field data: Theoretical and practical considerations. Annual Review of Sociology, 24, 477–498.

25.

Elsevier. (2014). Scopus: Content coverage guide. Retrieved from https://www-elsevier-com.web.bisu.edu.cn/__data/assets/pdf_file/0019/148402/SC_Content-Coverage-Guide_July-2014.PDF

26.

Evers

J. C.

(2011). From the past into the future. How technological development change our ways of data collection, transcription and analysis. Forum: Qualitative Social Research, 12, 1, Article 38. Retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/1636/

27.

Evers

J. C.

Silver

Mruck

Peeters

(2011). Introduction to the KWALON experiment: Discussions on qualitative data analysis software by developers and users. FORUM: Qualitative Social Research, 12, 1, Art.1. Retrieved from http://www.qualitative-research.net/fqs/fqs-eng.htm

28.

Fielding

N. G.

Lee

R. M.

(1998). Computer analysis and qualitative research. London, England: Sage.

29.

Finfgeld-Connett

(2014). Use of content analysis to conduct knowledge-building and theory-generating qualitative systematic reviews. Qualitative Research, 14, 341–352.

30.

Friese

(2014). Qualitative data analysis with ATLAS.ti^TM . London, England: Sage.

31.

Fritz

(1990). Computer analysis of qualitative data. In Lambert

(Ed.), The collection and interpretation of data from hidden populations (pp. 59–79). National Institute on Drug Abuse Research Monograph 98. DHHS Pub. No. (ADM) 90-1678. Washington, DC: Supt. of Docs., U.S. Govt. Print Office.

32.

Garcia-Horta

J. B.

Guerra-Ramos

M. T.

(2009). The use of CAQDAS in educational research: Some advantages, limitations and potential risks. International Journal of Research & Method in Education, 32, 151–165.

33.

Gibson

Callery

Campbell

Hall

Richards

(2005). The digital revolution in qualitative research: Working with digital audio and data through ATLAS.ti™. Sociological Research Online, 10, Retrieved from http://www.socresonline.org.uk/10/1/gibson.html doi:10.5153/sro.1044

34.

Gilbert

L. S.

(2002). Going the distance: ‘Closeness’ in qualitative data analysis software. International Journal of Social Research Methodology, 5, 215–228.

35.

Gilliam

M. L.

(2007). The role of parents and partners in the pregnancy behaviors of young Latinas. Hispanic Journal of Behavioral Sciences, 29, 50–67.

36.

Granero-Molina

Fernández-Sola

Castro-Sánchez

A. M.

Jiménez-López

F. R.

Aguilera-Manrique

Márquez-Membrive

(2012). The clinical seminar as a learning methodology: An evaluation of nursing students’ views. Acta Paulista de Enfermagem, 25, 441–447.

37.

Grime

Richardson

J. C.

Ong

B. N.

(2010). Perceptions of joint pain and feeling well in older people who reported being healthy: A qualitative study. British Journal of General Practice, 60, 597–603.

38.

Hannes

E. L.

Janssens

Wets

(2009). Does space matter? Travel mode scripts in daily activity travel. Environment and Behavior, 41, 75–100.

39.

Hartel

(2010). Managing documents at home for serious leisure: A case study of the hobby of gourmet cooking. Journal of Documentation, 66, 847–874.

40.

Hawthorne

Bennert

Lowes

Channon

Ronling

Gregory

J. W.

(2011). The experiences of children and their parents in paediatric diabetes services should inform the development of communication skills for healthcare staff (the DEPICTED Study). Diabetic Medicine, 28, 1103–1108.

41.

Hesse-Biber

Dupois

Kinder

T. S.

(1991). HyperRESEARCH: A computer program for the analysis of qualitative data with an emphasis on hypothesis testing and multimedia analysis. Qualitative Sociology, 14, 289–306.

42.

Holbrook

Butcher

(1996). Software in educational research: The literature, the hard questions and some specific research applications. Australian Educational Researcher, 23, 55–80.

43.

Holland

(2002). Computer-aided qualitative data analysis software. In Daymon

Holloway

(Eds.), Qualitative research methods in public relations and marketing communications (pp. 244–246). London, England: Routledge.

44.

Hurley

(2009). A qualitative study of mental health nurse identities: Many roles, one profession. International Journal of Mental Health Nursing, 18, 383–390.

45.

Hutchison

A. J.

Johnston

L. H.

Breckon

J. D.

(2010). Using QSR-Nvivo to facilitate the development of a grounded theory project: An account of a worked example. International Journal of Social Research Methodology, 13, 283–302.

46.

Jakobsen

C. H.

McLauglin

W. J.

(2004). Communication in ecosystem management: A case study of cross-disciplinary integration in the assessment phase of the interior Columbia basin ecosystem management project. Environmental Management, 33, 591–605.

47.

Kennedy

H. P.

Grant

Walton

Sandall

(2013). Elective caesarean delivery: A mixed method qualitative investigation. Midwifery, 29, 138–144.

48.

Kirchhoff

A. C.

Kuhlthau

Pajolek

Leisenring

Armstrong

G. T.

Robison

L. L.

Park

E. R.

(2013). Employer-sponsored health insurance coverage limitations: Results from the childhood cancer survivor study. Support Care Cancer, 21, 377–383.

49.

Laditka

S. B.

Corwin

S. J.

Laditka

J. N.

Liu

Friedman

D. B.

Mathews

A. E.

Wilcox

(2009). Methods and management of the healthy brain study: A large multisite qualitative research project. The Gerontologist, 49, S18–S22.

50.

Larkin

(2009). Socially mediated metacognition and learning to write. Thinking Skills and Creativity, 4, 149–159.

51.

MacMillan

(2005). More than just coding? Evaluating CAQDAS in a discourse analysis of news texts. FORUM: Qualitative Social Research, 6, 3, Art. 25. Retrieved from http://www.qualitative-research.net/fqs/fqs-eng.htm

52.

MacMillan

Koenig

(2004). The wow factor: Preconceptions and expectations for data analysis software in qualitative research. Social Science Computer Review, 22, 179–186.

53.

Mangabeira

W. C.

Lee

R. M.

Fielding

N. G.

(2004). Computers and qualitative research: Adoption, use, and representation. Social Science Computer Review, 22, 167–178.

54.

Marshall

(2002). What do we do when we code data? Qualitative Research Journal, 2, 56–70.

55.

Morison

Moir

(1998). The role of computer software in the analysis of qualitative data: Efficient clerk, research assistant or Trojan horse? Journal of Advanced Nursing, 28, 106–116.

56.

Moore

Morris

Crouch

Martin

(2003). Evaluation of physiotherapy clinical educational models: Comparing 1:1, 2:1 and 3:1 placements. Physiotherapy, 89, 489–501.

57.

Muhr

(1991). ATLAS/ti—A prototype for the support of text interpretation. Qualitative Sociology, 14, 349–371.

58.

Mutemwa

Mayhew

Colombini

Busza

Kivunaga

Ndwiga

(2013). Experiences of health care providers with integrated HIV and reproductive health services in Kenya: A qualitative study. BMC Health Services Research, 13, 18–28.

59.

Nair

Malhotra

M. K.

Ahire

S. L.

(2011). Toward a theory of managing context in six sigma process-improvement projects: An action research investigation. Journal of Operations Management, 29, 529–548.

60.

Newman

P. A.

Bogo

Daley

(2008). Self-disclosure of sexual orientation in social work field education: Field instructor and lesbian and gay student perspectives. The Clinical Supervisor, 27, 215–237.

61.

O’Halloran

(2011). Investigating argumentation in reading groups: Combining manual qualitative coding and automated corpus analysis tools. Applied Linguistics, 32, 172–196.

62.

Orobia

L. A.

Byabashaija

Munene

J. C.

Sejjaaka

S. K.

Musinguzi

(2013). How do small business owners manage working capital in an emerging economy? Qualitative Research in Accounting& Management, 10, 127–143.

63.

Paulus

Lester

Dempster

(2014). Digital tools for qualitative research. London, England: Sage.

64.

Paulus

Phipps

Harrison

Varga

M. A.

(2012). Re-envisioning instructional technology research in higher education environments: A content analysis of a grant program. Journal of Computing in Higher Education, 24, 164–181.

65.

Paulus

Woods

Atkins

Macklin

The discourse of QDAS: Reporting practices of ATLAS.ti and NVivo users with implications for best practices. International Journal of Social Research Methodology (Under Review).

66.

Pilling

S. A.

Williams

M. B.

Brackett

R. H.

Gourley

Vander Weg

M. W.

Christensen

A. J.

Kaboli

P. J.

Reisinger

H. S.

(2010). Activating patients to engage their providers in the use of evidence-based medicine: A qualitative evaluation of the VA Project to Implement Diuretics (VAPID). Implementation Science, 5, 23–34.

67.

Podolefsky

(1987). New tools for old jobs: Computers in the analysis of field notes. Anthropology Today, 3, 14–16.

68.

QSR International. (2014a). NVivo 10 feature list. Retrieved from http://download.qsrinternational.com/Resource/NVivo10/nvivo10-feature-list.pdf/

69.

QSR International. (2014b). Running a coding comparison query. Retrieved from http://webhelp-nv10.qsrinternational.com/nv10_help.htm#procedures/run_a_coding_comparison_query.htm/

70.

Richards

(1987). Qualitative data analysis: Can computers do it? Journal of Sociology, 23, 23–35.

71.

Richards

(1991). The NUD*IST qualitative data analysis system. Qualitative Sociology, 14, 307–324.

72.

Séror

(2005). Computers and qualitative data analysis: Paper, pens, and highlighters vs. screen, mouse and keyboard. TESOL Quarterly, 39, 321–328.

73.

Shelly

Sibert

(1986). Using logic programming to facilitate qualitative data analysis. Qualitative Sociology, 9, 145–161.

74.

Sin

C. H.

(2007). Using software to open up the ‘Black Box’ of qualitative data analysis in evaluations: The experience of a multi-site team using NUD*IST version 6. Evaluation, 13, 110–120.

75.

Szeinbach

S. L.

Seoane-Vazquez

Summers

K. H.

(2012). Development of a men’s preference for testosterone replacement therapy (P-TRT) instrument. Patient Preference and Adherence, 6, 631–641.

76.

Tallerico

(1991). Applications of qualitative analysis software: A view from the field. Qualitative Sociology, 14, 275–285.

77.

Thongpriwan

McElmurry

B. J.

(2009). Thai female adolescents’ perceptions of dating violence. Health Care for Women International, 30, 871–891.

78.

Waldrop

D. P.

(2006). At the eleventh hour: Psychosocial dynamics in short hospice stays. The Gerontologist, 46, 106–114.

79.

Wickham

Dunn

Sweeney

(2012). Analysis of the leading tourism journals 1999–2008. Annals of Tourism Research, 39, 1714–1718.

80.

Woods

D. K.

Dempster

P. G.

(2011). Tales from the bleeding edge: The qualitative analysis of complex video data using Transana. FORUM: Qualitative Social Research, 12, Art. 17, Retrieved from http://www.qualitative-research.net/fqs/fqs-eng.htm/

81.

Woods

Macklin

Lewis

(2015). Researcher reflexivity: Exploring the impacts of CAQDAS use. International Journal of Social Research Methodology. doi:10.1080/13645579.2015.1023964

Advancing Qualitative Research Using Qualitative Data Analysis Software (QDAS)? Reviewing Potential Versus Practice in Published Studies using ATLAS.ti and NVivo,1994–2013

Abstract

Keywords

Introduction

Method

Findings

Part 1: QDAS Users

Subject disciplines of journals publishing QDAS-supported studies

Geographic distribution/representation of QDAS usage

Part 2: QDAS Use

Types of studies conducted with QDAS support

Data types used in QDAS-supported studies

The phase of the research process supported by QDAS

Data collection/creation

Data analysis/management

Data display/representation of findings

Discussion

Footnotes

Authors' Note

Declaration of Conflicting Interests

Funding

References