Illumination with a Dim Bulb? What Do Social Scientists Learn by Employing Qualitative Data Analysis Software in the Service of Multimethod Designs?

Abstract

Although there has been much optimistic discussion of integrating quantitative and qualitative findings into sociological analysis, there remains a gap regarding the application of mixed approaches. We examine the potential gains and pitfalls of such integration in the context of the growing analytic power of contemporary qualitative data analysis software (QDAS) programs. We illustrate the issues with our own research in a mixed-methods project examining low fertility in Italy, a project that combines analysis of large nationally representative survey data with qualitative in-depth interviews with women across four cities in Italy. Despite the enthusiasm for mixed-methods research, the available software appears to be underutilized. In addition, we suggest that the sociological research community will want to address several conceptual and inferential issues with these approaches.

Get full access to this article

View all access options for this article.

References

Abell

1987. The Syntax of Social Life: The Theory and Method of Comparative Narratives. Oxford, England: Clarendon.

Abell

2004. “Narrative Explanation: An Alternative to Variable-Centered Explanation?” Annual Review of Sociology 30:287–310.

Abbott

1992. “From Causes to Events: Notes on Narrative Positivism. Sociological Research and Methods 20:428–55.

Abbott

1995. “Sequence Analysis: New Methods for Old Ideas.” Annual Review of Sociology 21: 93–113.

Adamson

Gooberman-Hill

Woolhead

Donovan

2004. “Questerviews: Using Questionnaires in Qualitative Interviews as a Methods of Integrating Qualitative and Quantitative Health Services Research.” Journal of Health Services Research and Policy 9(3):139–45.

Andrew

Salamonson

Halcomb

E. J.

2008. “Integrating Mixed Methods Data Analysis Using NVivo: An Example Examining Attrition and Persistence of Nursing Students.” International Journal of Multiple Research Approaches 2:36–43.

Ariès

1980. “Two Successive Motivations for the Declining Birth Rate in the West.” Population and Development Review 6(4):645–50.

Axinn

W. G.

Pearce

L. D.

2006. Mixed Method Data Collection Strategies. Cambridge, England: Cambridge University Press.

Bazeley

1999. “The Bricoleur with a Computer: Piecing Together Qualitative and Quantitative Data.” Qualitative Health Research 9(2):279–87.

10.

Bazeley

2003. “Computerized Data Analysis for Mixed Methods Research.” Pp. 385-422 in Handbook of Mixed Methods in Social and Behavioral Research, edited by Tashakkori

Teddlie

Thousand Oaks, CA: Sage.

11.

Bazeley

2004. “Issues in Mixing Qualitative and Quantitative Approaches to Research.” Pp. 141–56 in Applying Qualitative Methods to Marketing Management Research, edited by Buber

Gadner

Hampshire, England: Palgrave Macmillan.

12.

Bazeley

2006. “The Contribution of Computer Software to Integrating Qualitative and Quantitative Data Analysis.” Research in the Schools 13(1):64–74.

13.

Bazeley

2007. Qualitative Data Analysis with NVivo. London: Sage.

14.

Bazeley

2009. “Integrating Analyses in Mixed Methods Research” [editorial]. Journal of Mixed Methods Research 3(3):203–07.

15.

Bazeley

2010. “Computer Assisted Integration of Mixed Methods Data Sources and Analyses.” Pp. 431–67 in Handbook of Mixed Methods Research for the Social and Behavioral Sciences, 2nd ed., edited by Tashakkori

Teddlie

Thousand Oaks, CA: Sage.

16.

Bazeley

Kemp

2012. “Mosaics, Triangles, and DNA: Metaphors for Integrated Analysis in Mixed Methods Rresearch.” Journal of Mixed Methods Research 6(1):55–72.

17.

Bearman

Stovel

2000. “Becoming a Nazi: A Model for Narrative Networks.” Poetics 27:69–90.

18.

Bouchard

T. J.

1976. “Unobtrusive Measures: An Inventory of Uses.” Sociological Metlhods and Research 4:267–300.

19.

Bourdon

2002. “The Integration of Qualitative Data Analysis Software in Research Strategies: Resistances and Possibilities.” Forum: Qualitative Social Research 3(2):Art. 11.

20.

Bryman

2006a. “Integrating Quantitative and Qualitative Research: How Is It Done?” Qualitative Research 6(1):97–113.

21.

Bryman

2006b. “Paradigm Peace and the Implications for Quality.” International Journal of Social Research Methodology Theory and Practice 9(2):111–126.

22.

Burke Johnson

Onwuegbuzie

A. K.

Turner

L. A.

2007. “Toward a Definition of Mixed Methods Research.” Journal of Mixed Methods Research 1(2):112–33.

23.

Caldwell

J. C.

1977. “The Economic Rationality of High Fertility: An Iinvestigation Illustrated with Nigerian Survey Data.” Population Studies 31(1):5–27.

24.

Caldwell

J. C.

1982. Theory of Fertility Decline. London: Academic Press.

25.

Caldwell

J. C.

1988. The Causes of Demographic Change: Experimental Research in South India. Madison: University of Wisconsin Press.

26.

Caldwell

J. C.

Reddy

P. H.

Caldwell

1983. “The Causes of Marriage Change in South India.” Population Studies 37(3):343–61.

27.

Campbell

D. T.

Fiske

D. W.

1959. “Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix.” Psychological Bulletin 56:81–105.

28.

Caracelli

V. J.

Greene

J. C.

1993. “Data Analysis Strategies for Mixed-Method Evaluation Designs.” Educational Evaluation and Policy Analysis 15:195–207.

29.

Coffey

G. J.

Kaplan

Sampson

R. C.

Tucci

M. M.

2010. “The Meaning and Mental Health Consequences of Long-term Immigration Detention for People Seeking Asylum.” Social Science and Medicine 70(12):2070–79.

30.

Creswell

J. W.

Plano Clark

2007. Designing and Conducting Mixed Methods Research. Thousand Oaks, CA: Sage.

31.

Curtis

Gesler

Smith

Washburn

2000. “Approaches to Sampling and Case Selection in Qualitative Research: Examples in the Geography of Health.” Social Science and Medicine 50:1001–14

32.

De Meur

Rihoux

2002. L’Analyse Quali-Quantitative Comparée (AQQCQCA): Approche, Techniques et Applications en Sciences Humaines. Louvain-la-Neuve, France: Academia-Bruylant.

33.

Denzin

N. K.

1978. The Research Act: A Theoretical Introduction to Sociological Methods. New York: Praeger.

34.

Di Gregorio

Davidson

2008. Qualitative Research Design for Software Users. Berkshire, England: Open University Press.

35.

Driscoll

D. L.

Appiah-Yeboah

Salib

Rupert

D. J.

2007. “Merging Qualitative and Quantitative Data in Mixed Methods Research: How to and Why Not.” Ecological and Environmental Anthropology 3(1): 19–28.

36.

Duncan

G. J.

2008. “When to Promote, and When to Avoid: A Population Perspective.” Demography 45(4):763–84.

37.

Dzurec

L. C.

Abraham

J. L.

1993. “The Nature of Inquiry: Linking Quantitative and Qualitative Research.” Advances in Nursing Science 16:73–79.

38.

Fielding

N. G.

Lee

R. M.

1993. Using Computers in Qualitative Research. London: Sage.

39.

Firestone

1993. “Alternative Arguments for Generalizing from Data as Applied to Qualitative Research.” Educational Researcher 22(4):16–23.

40.

Franzosi

1989. “From Words to Numbers: A Generalized and Linguistics-Based Coding Procedure for Collecting Event-Data from Newspapers.” Pp. 263–98 in Sociological Methodology, vol. 19, edited by Clogg

Clifford

. Oxford, England: Basil Blackwell.

41.

Franzosi

1994. “From Words to Numbers: A Set Theory Framework for the Collection, Organization, and Analysis of Narrative Data.” Pp. 105–36 in Sociological Methodology, vol. 24, edited by Marsden

Peter

. Oxford, England: Basil Blackwell.

42.

Franzosi

1997. “Mobilization and Counter-Mobilization Processes: From the ‘Red Years’ (1919–20) to the ‘Black Years’ (1921–22) in Italy: A New Methodological Approach to the Study of Narrative Data.” Theory and Society 26(2/3):275–304.

43.

Franzosi

2003. From Words to Numbers. Cambridge, England: Cambridge University Press.

44.

Gibbs Graham

2002. Qualitative Data Analysis: Explorations with NVivo. Buckingham, England: Open University Press.

45.

Gilbert

, ed. 2008. Researching Social Life, 3rd ed. London: Sage.

46.

Greene

J. C.

2006. “Toward a Methodology of Mixed Methods Social Inquiry.” Research in the Schools 13(1):93–98.

47.

Greene

J. C.

2007. Mixing Methods in Social Inquiry. San Francisco: Jossey-Bass.

48.

Greene

J. C.

2008. “Is Mixed Methods Social Inquiry a Distinctive Methodology?” Journal of Mixed Methods Research 2(1):7–22.

49.

Greene

J. C.

Caracelli

V. J.

Graham

W. F.

1989. “Toward a Conceptual Framework for Mixed Method Evaluation Designs.” Educational Evaluation and Policy Analysis 11:255–74.

50.

Hesse-Biber

S. N.

1996. Am I Thin Enough Yet? The Cult of Thinness and the Commercialization of Identity. New York: Oxford University Press.

51.

Hesse-Biber

S. N.

Leavy

2005. The Practice of Qualitative Research. London: Sage.

52.

Hollingshead

A. B.

1949. Elmtown’s Youth. New York: Wiley.

53.

Hutchison

A.J.

Johnston

L.H.

Breckon

J.D.

2010. “Using QSR-NVivo to Facilitate the Development of a Grounded Theory Project: An Account of a Worked Example.” International Journal of Social Research Methodology 13 (4):283–302

54.

Jick

T. D.

1979. “Mixing Qualitative and Quantitative Methods: Triangulation in Action.” Administrative Science Quarterly 24:602–11.

55.

Kaasalainen

Williams

Hadjistavropoulos

Thorpe

Whiting

Neville

Tremeer

2010. “Creating Bridges Between Researchers and Long-term Care Homes to Promote Quality of Life for Residents.” Qualitative Health Research 20(12):1689–704.

56.

Kertzer

Hogan

1989. Family, Political Economy, and Demographic Change. Madison: University of Wisconsin Press.

57.

Kertzer

1990. “Household Organization and Migration in Nineteenth-Century Italy.” Social Science History 14(4):483–505.

58.

Kertzer

D. I.

Fricke

1997. “Toward an Anthropological Demography.” Pp. 1–35 in Anthropological Demography, edited by Kertzer

D. I.

Fricke

Chicago: University of Chicago Press.

59.

King

2010. “‘Membership Matters’: Applying Membership Categorisation Analysis (MCA) to Qualitative Data Using Computer? Assisted Qualitative Data Analysis (CAQDAS) Software.” International Journal of Social Research Methodology 13(1):1–16

60.

Knodel

Chamratrithirong

Debavalya

1987. Thailand’s Reproductive Revolution: Rapid Fertility Decline in a Third World Setting. Madison: University of Wisconsin Press.

61.

Knodel

Saengtienchai

Wassana

VanLandingham

2001. “The Impact of AIDS on Parents and Families in Thailand.” Research on Aging 23(6): 633–70.

62.

Knodel

Im-em

2004. “The Economic Consequences of Parents of Losing an Adult Child to AIDS: Evidence from Thailand.” Social Science and Medicine 59:987–1001.

63.

Knodel

J. E.

van de Walle

1986. “Lesson from the Past: Policy Implications of Historical Fertility Studies.” Pp. 390–419 in The Decline of Fertility in Europe, edited by Coale

Watkins

S. C.

Princeton, NJ: Princeton University Press.

64.

Lesthaeghe

1995. “The Second Demographic Transition in Western Countries: An Interpretation.” Pp. 17–62 in Gender and Family Change in Industrialized Countries, edited by Oppenheim Mason

Jensen

Oxford, England: Oxford University Press.

65.

Lewins

2008. “Computer Assisted Qualitative Data Analysis (CAQDAS).” Pp. 125-44 in Researching Social Life, 3rd ed., edited by Gilbert

London: Sage.

66.

Lewins

Silver

2007. Using Software in Qualitative Research: A Step-by-Step Guide. London: Sage.

67.

Livi-Bacci

2001. “Too Few Children and Too Much Family.” Daedalus 130:137–56.

68.

Manning

W. D

Cohen

J. A.

Smock

P. J.

2011. “The Role of Romantic Partners, Family, and Peer Networks in Dating Couples’ Views About Cohabitation.” Journal of Adolescent Research 26(1):115–49.

69.

Massey

1987. “The Ethnosurvey in Theory and Practice.” International Migration Review 21(4):1498–522.

70.

Massey

D. S.

Zenteno

2000. “A Validation of the Ethnosurvey: The Case of Mexico-US migration.” International Migration Review 34(3):766–93.

71.

Miles

M. B.

Huberman

A. M.

1994. Qualitative Data Analysis: An Expanded Sourcebook, 2nd ed. London: Sage.

72.

Moffitt

2005. “Remarks on the Analysis of Causal Relationships in Population Research.” Demography 42(1):91–108.

73.

Morgan

D. L.

1998. “Practical Strategies for Combining Qualitative and Quantitative Methods: Applications to Health Research.” Qualitative Health Research 8:362–76.

74.

Morse

J. M.

1991. “Approaches to Qualitative-Quantitative Methodological Triangulation.” Nursing Research 40(2):120–23.

75.

Morse

J. M.

2003. “Principles of Mixed Methods and Multimethod Research Design.” Pp. 189–208 in Handbook of Mixed Methods in Social and Behavioral Research, edited by Tashakkori

Teddlie

Thousand Oaks, CA: Sage.

76.

Niglas

2004. The Combined Use of Qualitative and Quantitative Methods in Educational Research. Tallinn, Estonia: Tallinn Pedagogical University Dissertation on Social Sciences.

77.

O’Cathain

Thomas

2006. “Combining Qualitative and Quantitative Methods.” Pp. 102–11 in Qualitative Research in Health Care, 3rd ed., edited by Pope

Mays

Oxford, England: Blackwell.

78.

O’Cathain

Murphy

Nicholl

2007. “Integration and Publications as Indicators of ‘Yield’ from Mixed Methods Studies.” Journal of Mixed Methods Research 1(2):147–63.

79.

Onwuegbuzie

A. J.

Leech

N. L.

2005. “The Role of Sampling in Qualitative Research.” Academic Exchange Quarterly 9:280–84.

80.

Onwuegbuzie

A. J.

Leech

N. L.

2007. “A Call for Qualitative Power Analyses.” Quality and Quantity: International Journal of Methodology 41:105–21.

81.

Onwuegbuzie

A. J.

Collins

K. M. T.

2007. “A Typology of Mixed Methods Sampling Designs in Social Science Research.” The Qualitative Report 12(2):281–316.

82.

Onwuegbuzie

A. J.

Witcher

A. E.

Collins

K. M. T.

Filer

Wiedmaier

2007. “Students’ Perceptions of Characteristics of Effective College Teachers: A Validity Study of Teaching Evaluation Form Using a Mixed-Methods Analysis.” American Educational Research Journal 44:113-60.

83.

Ragin

C. C.

1987. The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies. Berkeley, CA: University of California Press.

84.

Ragin

C. C.

1997. “Turning the Tables: How Case-Oriented Research Challenges Variable Oriented Research.” Comparative Social Research 16(1):27–42.

85.

Ragin

C. C.

2000. Fuzzy-Set Social Science. Chicago: University of Chicago Press.

86.

Rao

1997. “Can Economics Mediate the Relationship Between Anthropology and Demography?” Population and Development Review 23(4):833–38.

87.

Richards

2002. “Qualitative Computing: A Methods Revolution?” International Journal of Social Research Methodology 5(3):263–76.

88.

Richards

2005. Handling Qualitative Data. London: Sage.

89.

Richards

2002. “An Intellectual History of NUD*IST and NVivo.” International Journal of Social Research Methodology 5(3):199–214.

90.

Rihoux

2003. “Bridging the Gap Between the Qualitative and Quantitative Worlds? A Retrospective and Prospective View on Qualitative Comparative Analysis.” Field Methods 15(4):351–65.

91.

Rihoux

2006. “Qualitative Comparative Analysis (QCA) and Related Systematic Comparative Methods: Recent Advances and Remaining Challenges for Social Science Research.” International Sociology 21(5):679–706.

92.

Rihoux

Ragin

, eds. 2009. Configurational Comparative Methods. Los Angeles: Sage.

93.

Roberts

K. A.

Wilson

R. W.

2002. “ICT and the Research Process: Issues Around the Compatibility of Technology with Qualitative Data Analysis.” Forum: Qualitative Social Research [online journal, 52 paragraphs] 3(2). Retrieved April 2, 2010 (http://www.qualitative-research.net/fqs/fqs-eng.htm).

94.

Rossman

G. B.

Wilson

B. L.

1985. “Numbers and Words: Combining Quantitative and Qualitative Methods in a Single Large-Scale Evaluation Study.” Evaluation Review 9:627–43.

95.

Sale

J. E. M.

Lohfeld

L. H.

Brazil

2002. “Revisiting the Quantitative-Qualitative Debate: Implications for Mixed-Methods Research.” Quality and Quantity 36(1):43–53.

96.

Sandelowski

2000. “Combining Qualitative and Quantitative Sampling, Data Collection and Analysis Techniques in Mixed-Methods Studies.” Research in Nursing and Health 23:246–55.

97.

Sandelowski

2003. “Tables or Tableaux? The Challenges of Writing and Reading Mixed Methods Studies.” Pp. 321–50 in Handbook of Mixed Methods in Social and Behavioral Research, edited by Tashakkori

Teddlie

London: Sage.

98.

Sandelowski

Harris

B. G.

Holditch-Davis

1991. “Amniocentesis in the Context of Infertility.” Health Care for Women International 12:167–78.

99.

Sassler

2004. “The Process of Entering into Cohabiting Unions.” Journal of Marriage and Family 66:491–505.

100.

Sechrest

Sidana

1995. “Quantitative and Qualitative Methods: Is There an Alternative?” Evaluation and Program Planning 18:77–87.

101.

Sieber

S. D.

1973. “The Integration of Fieldwork and Survey Methods.” American Journal of Sociology 73:1335–59.

102.

Smith

2007. “Narrative Boundaries and the Dynamics of Ethnic Conflict and Conciliation.” Poetics 35:22–46.

103.

Stevens

P. E.

Hildebrandt

2009. “Pill Taking from the Perspective of HIV-Infected Women Who Are Vulnerable to Antiretroviral Treatment Failure.” Qualitative Health Research 19(5):593–604.

104.

Stoller

E. P.

Webster

N. J.

Blixen

C. E.

McCormick

R. A.

Hund

A. J.

Perzynski

A. T.

Kanuch

S. W.

Thomas

C. L.

Kercher

Dawson

N. V.

2009. “Alcohol Consumption Decisions Among Nonabusing Drinkers Diagnosed with Hepatitis C.” Journal of Mixed Methods Research 3(1):65–86.

105.

Surkyn

Lesthaeghe

2004. “Value Orientations and the Second Demographic Transition (SDT) in Northern, Western and Southern Europe: An Update.” Demographic Research, Special Collection 3, Article 3:45-86.

106.

Tashakkori

Creswell

J. W.

2008. “Editorial: Mixed Methodology Across Disciplines.” Journal of Mixed Methods Research 2(1):3–6.

107.

Tashakkori

Teddlie

1998. Mixed Methodology: Combining Qualitative and Quantitative Approaches. Thousand Oaks, CA: Sage.

108.

Tashakkori

Teddlie

2010. SAGE Handbook of Mixed Methods in Social and Behavioral Research, 2nd ed. Thousand Oaks, CA: Sage.

109.

Taylor

S. J.

Tashakkori

1997. “Toward an Understanding of Teachers’ Desire for Participation in Decision Making.” Journal of School Leadership 7:1–20.

110.

Teddlie

Tashakkori

2009. Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences. Thousand Oaks, CA: Sage.

111.

Twinn

2003. “Status of Mixed Methods Research in Nursing.” Pp. 541–56 in Handbook of Mixed Methods in Social and Behavioral Research, edited by Tashakkori

Teddlie

London: Sage.

112.

van de Kaa

D. J.

2001. “Postmodern Fertility Preferences: From Changing Value Orientation to New Behavior.” Population and Development Review 27 (suppl.):290–331.

113.

Watt

M. H.

Maman

Earp

J. A.

Eng

Setel

P. W.

Golin

C. E.

Jacobson

2009. “‘It’s all the time in my mind’: Facilitators of Adherence to Antiretroviral Therapy in a Tanzanian Setting.” Social Science and Medicine 68(10):1793–800.

114.

Webb

E. J.

Campbell

D. T.

Schwartz

R. D.

Sechrest

1966. Unobtrusive Measures. Chicago: Rand McNally.

115.

Weisner

T. S.

2005. Discovering Successful Pathways in Children’s Development: New Methods in the Study of Childhood and Family Life. Chicago: University of Chicago Press.