Using secondary analysis of qualitative data of patient experiences of health care to inform health services research and policy

Abstract

Qualitative research is recognized as an important method for including patients' voices and experiences in health services research and policy-making, yet the considerable potential to analyse existing qualitative data to inform health policy and practice has been little realized. This failure may partly be explained by: a lack of awareness amongst health policy makers of the increasing wealth of qualitative data available; and around 15 years of internal debates among qualitative researchers on the strengths, limitations and validity of re-use of qualitative data. Whilst acknowledging the challenges of qualitative secondary data analysis, we argue that there is a growing imperative to be pragmatic and to undertake analysis of existing qualitative data collections where they have the potential to contribute to health policy formulation. Time pressures are inherent in the policy-making process and in many circumstances it is not possible to seek funding, conduct and analyse new qualitative studies of patients' experiences in time to inform a specific policy. The danger then is that the patient voice, and the experiences of relatives and carers, is either excluded or included in a way that is easily dismissed as ‘unrepresentative’. We argue that secondary analysis of qualitative data collections may sometimes be an effective means to enable patient experiences to inform policy decision-making.

Keywords

health policy qualitative interviews secondary analysis

Introduction

Qualitative research is recognized as increasingly important in health services research, yet the potential for secondary analysis of qualitative data has been little realized. Conducting relatively unstructured, in-depth interviews with diverse samples is an established method for generating understanding of patients' experiences and perspectives,¹ but can be time-consuming to conduct, particularly when care is taken to sample the widest feasible range of perspectives. In contrast, a focussed secondary analysis of existing high quality qualitative data can be highly efficient, allowing effective use of researchers' time for analysis and rigorous testing of findings.²

Two developments in prevailing research cultures particularly favor reuse of existing data. First, synthesizing published and unpublished research before (or instead of) conducting new research is now mainstream practice.^3–5 Secondly, following the decision in 2000 by the Economic and Social Research Council (ESRC), a major funder of social science research in the UK, to require ‘award holders to offer for archiving and sharing copies of both digital and non-digital data to the Economic and Social Data Service’ and to seek the correct permission to do so when collecting data (p. 256),⁶ several funding bodies nationally^7,8 and internationally⁹ have followed suit. Many now have explicit expectations that publicly-funded research should be shared to maximize the value of data for the public good and ‘to expedite the translation of research results into knowledge, products, and procedures to improve human health’.¹⁰ A representative of one funder stated, ‘we need to ensure that research outputs are used to maximized knowledge and potential health benefits. In turn, the populations who participate in research, and the taxpayers who foot the bill, have the right to expect that every last ounce of knowledge will be wrung from the research’.¹¹

These directives, implicitly or explicitly, cover qualitative and quantitative data. It is often assumed that the case for quantitative data sharing is unproblematic and compelling, since these data are usually expensive to collect and can be reduced to variables which, theoretically, are equally useful to all analysts. Whilst this view sometimes does a disservice to the importance of circumstantial and contextual knowledge that primary data collectors have, a number of inherent features of qualitative data collection and analysis have been highlighted as particularly problematic in sharing and re-using qualitative data. Over the last 15 years^12–16 there has been a lively, and sometimes ‘moralistic and polarised’¹⁷ debate among qualitative researchers. We rehearse the main features of this debate, and present two case studies using data from one archive to illustrate the potential for judicious re-use of qualitative data.

The debate about the re-use of qualitative data

Mason summarized the polarization of the debate into ‘on the one hand a position that says that qualitative data are special and cannot be re-used by others on epistemological or ethical grounds, and on the other a pragmatic or instrumental position that says that data should be open for use by others, not least because they are expensive to produce’. The former position has been well-articulated by Parry et al.^14,15 who raise issues around ownership and copyright, the co-construction of data between respondent and researcher, confidentiality, preserving anonymity, and the problematic nature of gaining respondent consent for broader use of the data (particularly as all future potential uses cannot be exhaustively listed). Bishop, by contrast, has argued that there are good suggestive, but not prescriptive guidelines¹⁸ on how to deal with several of the legal and ethical issues raised (e.g. copyright, ownership, acquiring appropriate consent, protecting confidentiality).

An issue that has generated much heat in the debate has been the extent to which the ‘context’, the insights that a researcher has through ‘being there’ in the research setting where the data were gathered, can be adequately summarized and captured for use by analysts other than the primary researcher. An interviewer may absorb aspects of the respondent's life circumstances and biography and become aware of limitations on their willingness and ability to discuss particular issues. Such details may be captured in fieldnotes, but these are often highly individual and difficult for others to make sense of, even if they are shared. Fielding¹⁹ emphasizes the need to include as much contextual information as possible when archiving qualitative data, yet there are tensions between this need for detail and requirements to preserve anonymity. Mason argues that ‘the idea that only those involved in initial data generation can understand the context enough to interpret the data is not only anti-historical but it puts enormous epistemological weight onto the notion of “successful reflexivity”¹⁷ but Parry et al. suggest that the ‘recovery of context can only ever be partial’ (p. 340).¹⁵

Attempting to move the debate forwards, Mason has urged qualitative social scientists to be ‘leaders rather than laggards’ in thinking through the challenges of using and ‘re-using’ qualitative data.¹⁷ Others question the clarity of the distinction between ‘reusing’ and ‘using’ qualitative data, emphasising instead: that all data are constituted and reconstituted within the research process;²⁰ that qualitative data are ‘re-contextualised and co-constructed whether reading transcripts or doing an interview in real time’;²¹ continuities between ‘primary’ or ‘secondary’ data and the view that ‘secondary analysis’ can be conceptualized as ‘primary analysis of a different order of data’.²⁰ Irwin and Winterton suggest that

a more productively drawn distinction is between data and evidence ¨ both primary and secondary analysts will construct data as evidence in the service of some empirically grounded set of arguments and knowledge claims … Presence at the point of data generation is not a final arbiter [of social scientific adequacy]. (p. 8)²²

We would argue that some forms of data are probably more flexible for secondary analysis than others. Whilst not all qualitative studies will be equally rich resources for secondary analysis, the same set of video recordings of consultations could, for example, inform quite distinct analyses of topics such as the verbal and non-verbal positioning and performances of identity in clinical encounters and be approached for analysis from many different perspectives and research traditions. In contrast an ethnographer's personal field diaries might be a rich source of secondary material but the range of analytic approaches is likely to be more limited. Interviews that were conducted to address a very focused research question may have less scope for secondary analysis than those eliciting an open narrative. Fitness for purpose is thus an important consideration and the researcher might expect to be allowed access to some examples of the data before applying for funding to support their secondary research. When preparing a grant application for the ESRC the researcher Janet Heaton arranged to have access to a few narrative interviews from the Oxford HERG archive so that she could pilot her analysis and ensure that the collections would contain sufficiently rich and appropriate material for her study on Mastering Chronic Illness while Growing Up.

Collections from different eras might illuminate changed orientations to health care or initiatives over time, or in different cultures. Interviews that were collected 10 or 15 years ago may be highly relevant in some respects and very dated in others – for example the people we interviewed 10 years ago talked about using the internet for health information as a marker of their unusual and particular engagement with their health decisions, whereas in recent years people describe looking for online information in an almost off hand, unremarked manner ‘we all looked it up’ ‘as anyone would’. We would not therefore recommend using decade old interviews on views of the internet to inform current digital health policy.

We believe that with due consideration of the issues addressed by Bishop¹⁸ amongst others, problems are likely to be surmountable in all but a minority of cases.

In mixed methods studies, secondary analysis or qualitative data can offer the opportunity to help develop hypotheses quickly and efficiently, or to compare important dimensions of health care experiences (e.g. access, waiting-times, communication, information, dignity and respect) in people with different health conditions, social characteristics and biographies.

Case studies

The policy implications of reusing qualitative data are largely under explored but some of the potential can be illustrated through two examples of secondary analysis of qualitative data held by the Health Experience Research Group (HERG) in Oxford (Box 1).

Box 1.

The Health Experiences Research Group (HERG) Archive, University of Oxford: an example of a qualitative data resource for secondary analysis.

Data collection

• Interviews are conducted by experienced and well-supported qualitative social science researchers to common research protocols.

• Interview participants complete an informed consent before the interview and are then offered the opportunity to review their interview (recording or transcript) to decide if there are any sections they would prefer not to be used on a website (healthtalkonline) or in research. Participants can also choose whether they are willing for a video, audio or written version of their interview to be used.

• Interviews are copyrighted to the University of Oxford for secondary analysis (as well as teaching and broadcasting) and anonymized for use by secondary analysts.

• Maximum variation samples, including participants through the UK, aim for data saturation for each collection.

• Interviews typically start with an open narrative encouraging respondents to articulate issues of relevance to them, before a semi-structured section, which probes for more detail in specific areas.

• All interviews are audio-recorded (and most are also video-recorded).

Potential for secondary analysis

• Over 75 collections on different conditions and health-related topics, each including around 40 interviews conducted using a standardized approach.

• All interviews were collected since 2000; all collections are reviewed every two years by an expert panel to decide if new interviews are needed; most of the collections have been updated with additional interviews.

• Since inception the HERG interviews have been collected with regard to the potential for secondary analysis by external researchers.

• Possibilities for secondary analysis of topic-specific or cross-cutting issues.

• Continuity of input from the original researcher or research ‘buddy’ within the HERG team to offer internal and external users advice on the data.

• Collections are available under licence from the University of Oxford (hergadmin@phc.ox.ac.uk); the terms of the licence include a requirement that the HERG director is provided with pre-submission copies of any publication resulting from the work (this is mainly to make sure that the participants and study are referred to appropriately in the publication).

• Increasing possibilities for cross-country comparisons with development of the DIPEx International collaboration.³⁰ An established track record of secondary analysis by external users since 2004, including on individual collections,^26,27 comparison across multiple collections,²⁸ comparison of data from several collections on a similar topic.²⁹

Limitations of the HERG data sets for secondary analysis

• Specific new developments in health and social care may not be covered in depth, or in older collections.

• HERG archive interviews usually provide little information about regional practice.

• While transcripts are easy to anonymize the video and audio recordings may require extensive editing before they can be shared under licence, thus most external users of the data only use the transcripts.

• Some charges are made for the collections to cover costs of archiving and administrative support.

The first example was commissioned by the General Medical Council (GMC) when preparing the 2010 guidelines on end of life care. The GMC recognized the importance of including the perspectives of patients but were concerned that it could be unacceptably burdensome for patients close to the end of life to contribute and that those who could contribute might, for a variety of reasons, not represent a sufficiently wide range of the health and social care issues that are faced at the end of life. The GMC therefore commissioned HERG to undertake a secondary analysis of interviews with people at the end of life. The HERG data archive included 95 interviews, which were pertinent to end of life experiences, from patients with a range of conditions including advanced cancers, Motor Neurone Disease and COPD, and people's experiences of bereavement. The interview transcripts were analysed drawing on anticipated issues that were of interest to the GMC and additional, emergent themes. The final GMC report noted that the qualitative secondary analysis research

produced a rich source of first-hand views and comments from patients and carers about the care provided towards the end of life. The findings of the research report provided greater context to many of the issues in the draft guidance, and contributed to ideas about where our implementation work should focus. (General Medical Council²³)

This secondary analysis and report preparation took the (experienced and efficient) researcher, who had not been involved in the original data collection but who was familiar with the narrative methods that had been used, 18 weeks to complete. A primary researcher would have been hard pressed to identify 95 potential participants in this time, let alone gain ethical and gatekeeper approvals, conduct interviews, transcriptions and analysis. A secondary analyst who was not familiar with the interview methods might have taken a little longer to engage with the material, but narrative interviews are reasonable accessible to the secondary analyst. The example also illustrates that a focussed analysis of existing collections has the potential to contribute patients' perspectives that might otherwise have been excluded or marginalized.

Our second example is a collaboration to explore whether secondary analysis of HERG interviews could accelerate the process of experience based co-design (EBCD). EBCD uses a Participatory Action Research design which involves the ‘coupling of participation and research to action and change’. Integral to the approach, which has demonstrated positive effect²⁴ is that patient, carer and staff experiences are first collected by video-recorded qualitative interviews and then presented as ‘trigger films’ to groups of patients and staff who work together to co-design and improve local services. With funding from the NIHR HSD&R programme, a researcher from the Oxford group re-analysed two collections, one for which she had been the primary researcher and one in which she had had no previous involvement. Informed by this analysis, two ‘trigger films’ were compiled with excerpts from the interview video recordings. The aim of the study was to find out whether trigger films based on the existing collection of HERG interviews, as an accelerated form of EBCD, could achieve comparable results to EBCD, even though the data were not ‘local’. The accelerated approach proved readily acceptable to staff and patients; using films of nationally-gathered rather than local narratives did not adversely affect local NHS staff engagement, and may in some cases have made the process less threatening or challenging. The films served their purpose as a ‘trigger’ to discussion, and the resulting 48 co-design activities across the 4 pathways were similar in nature to those in EBCD but achieved at reduced cost.²⁵ Qualitative research is often thought to be highly context specific, yet in this approach to service improvement, trigger films based on existing interviews from other services and contexts were effective in stimulating change.

Discussion and Conclusions

In this article, we have illustrated the potential for qualitative secondary analysis for health policy-making, drawing on our familiarity with the HERG archive to provide two case studies. While both case study examples were projects run by HERG there are numerous examples of external researchers using HERG data for secondary analyses (see, e.g. Hunt et al.,²⁶ Hall et al.,²⁷ Charteris-Black and Seale²⁸ and Hilton et al.²⁹). The HERG archive provides just one exemplar and other archives and research collections offer a rich and highly cost-effective resource for health policy (Box 2). There are many gems in, for example, the British Library Sound Archive which includes health collections on many aspects of the history of medicine and recordings of interviews with individual doctors and nurses. The library also houses collaborative projects charting the experiences of people with ‘disabilities, ill-health or with mental health issues’. The Wellcome Trust has an extensive archives and manuscript collection including material on ‘birth control, genetics and heredity, psychiatry, psychoanalysis and psychology, public health, the colonial encounter/imperial medicine, war and medicine, and women in the healthcare professions’. The Wellcome archive also houses personal papers notebooks and diaries from doctors, nurses and scientists. See Box 3 for other resources.

Box 2.

Qualitative secondary analysis for health services research and policy.

• Allows focus on a targeted analysis to inform health policy

• Desirable to ‘stand on shoulders’ of existing research (which may not have been published)

• Efficient, especially if access and ethics for primary research raise complex issues

• Avoids burdening over-researched groups

• May provide access to perspectives of elusive groups or communities considered hard to engage

• Inform pilot studies e.g.

○ identifying item pools for questionnaire development

○ defining the parameters of a diversity sample

• Can inform a richer, comparative analysis

○ between health conditions

○ across time

○ between cultures

Box 3.

Other selected resources with potential for qualitative secondary analysis for UK health policy research.

The British Library sound archive health resources

http://www.bl.uk/reshelp/findhelprestype/sound/ohist/ohcoll/ohhealth/health.html

The British Dental Association has an oral history archive

www.bda.org > BDA Museum > The story of dentistry > Reminiscences

ESDS at the University of Essex is an extensive archive of all ESRC funded research

http://www.esds.ac.uk/qualidata/about/introduction.asp

The Timescapes project in Leeds has contemporary material on families, schooling and friendship among several projects with relevance to health and social care.

http://www.timescapes.leeds.ac.uk/archive/

http://bufvc.ac.uk/ The British Universities Film & Video Council (BUFVC)

Promotes the production, study and use of moving image, sound and related media in higher education, further education and research.

Increasingly, as with studies which contribute to the HERG data archive, the potential for secondary analysis needs to be carefully considered from the outset, to ensure respondents can give appropriate consent for re-use of the data.

The perspectives of researchers whose interviews may be used for secondary analysis have received little consideration in the literature.²¹ Researchers in HERG are aware from the outset that the data they collect will be available, under licence, to other researchers. The original researcher has an opportunity to publish key findings first, is often involved as a consultant to a secondary analysis project and may contribute to the resulting papers. While the secondary analysis process can feel somewhat exposing, those in HERG whose data has been used by other researchers emphasize the benefits of approaching the interviews through a new analytical lens, the helpful challenge of a perspective that is more distant from the context of the original data collection and the opportunity to maximize the utility of data (especially if re-use may lead to improvements in services for the patient group).

In conclusion, it is important for policy to be evidence based and patients' experiences form a vital part of the evidence about healtaposrsquo;s experiences is not always achievable within financial or time constraints yet its exclusion can silence or marginalize the patient voice. We argue for a pragmatic approach that develops appropriate archiving practices and analytic skills among qualitative researchers so that rigorously collected qualitative data sets can be used to establish patients' perspectives at the center of health policy.

Footnotes

Acknowledgement

We are very grateful to colleagues who contributed to the November 2012 workshop on qualitative secondary analysis, especially Libby Bishop, Janet Heaton, Clive Seale, Sarah Irwin, Mandy Winterton, Emma France and Ulla Raisanen.

The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HS&DR Programme, NIHR, NHS or the Department of Health.

Funding

This project was funded by the National Institute for Health Research Health Services and Delivery Research Programme (project number 10/1011/19).

References

Ziebland

Narrative interviewing. In: Ziebland

Coulter

Calabrese

Locock

(eds). Understanding and using health experiences: improving patient care, Oxford: Oxford University Press, 2013, pp. 38–48.

Heaton

. Reworking qualitative data, London: Sage, 2004.

Mays

Pope

Popay

. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy 2005; 10(Suppl 1): 6–20.

Petticrew

Roberts

. Systematic reviews in the social sciences. A practical guide, Oxford: Blackwell Publishing Ltd, 2006.

Pope

Mays

Popay

. Synthesising qualitative and quantitative health research: a guide to methods, Maidenhead: Open University Press, 2007.

Bishop

. Ethical sharing and reuse of qualitative data. Aust J Soc Issues 2009; 44(3): 255–272.

Wellcome Trust. Policy on data management and sharing, http://www.wellcome.ac.uk/About-us/Policy/Policy-and-position-statements/WTX035043.htm (2010, accessed 18 February 2014).

Medical Research Council. MRC policy on sharing of research data from population and patient studies, http://www.mrc.ac.uk/Ourresearch/Ethicsresearchguidance/datasharing/Policy/PHSPolicy/index.htm (2011, accessed 18 February 2014).

Organisation for Economic Co-operation and Development. OECD principles and guidelines for access to research data from public funding. Paris: OECD Publications, 2007.

10.

NIH. NIH data sharing policy and implementation guidance, http://grants.nih.gov/grants/policy/data_sharing/data_sharing_guidance.htm?print=yes& (2003, accessed 18 February 2014).

11.

Walport

Brest

. Sharing research data to improve public health. Lancet 2011; 377(9765): 537–539.

12.

Hinds

Vogel

Clarke-Steffen

. The possibilities and pitfalls of doing a secondary analysis of a qualitative data set. Qual Health Res 1997; 7: 408–424.

13.

Heaton J. Secondary analysis of qualitative data. Social Research Update University of Surrey. 1998; 22 (Autumn).

14.

Parry

Mauthner

. Whose data are they anyway? Practical, legal and ethical issues in archiving qualitative research data. Sociology 2004; 38: 139–152.

15.

Parry

Mauthner

. Back to basics: who re-uses qualitative data and why? Sociology 2005; 39: 337–342.

16.

Hammersley

. Can we re-use qualitative data via secondary analysis? Notes on some terminological and substantive issues. Sociol Res Online 2009; 15(1): 5–5.

17.

Mason

. ‘Re-using’ qualitative data: on the merits of an investigative epistemology. Sociol Res Online 2007; 12(3): 3–3.

18.

Bishop

. Protecting respondents and enabling data sharing: reply to Parry and Mauthner. Sociology 2005; 39: 333–336.

19.

Fielding

. Resistance and adaptation to criminal identity: using secondary analysis to evaluate classic studies of crime and deviance. Sociology 2000; 34: 671–689.

20.

Moore

. (Re)using qualitative data. Sociol Res Online 2007; 12(3): 1–1.

21.

Bishop

. A reflexive account of reusing qualitative data: beyond primary/secondary dualism. Sociol Res Online 2007; 12(3)

22.

Irwin

Winterton

. Qualitative secondary analysis and social explanation. Sociol Res Online 2012; 17(2): 4–4.

23.

General Medical Council. The development of treatment and care towards the end of life: good practice in decision making, http://www.gmc-uk.org/Story_of_the_EoL_guidance2.pdf_32510823.pdf (2010, accessed 18 February 2014).

24.

Bate

Robert

. Experience-based design: from redesigning the system around the patient to co-designing services with the patient. Qual Saf Health Care 2006; 15: 307–310.

25.

Locock L, Robert G, Boaz A, et al. Using a national archive of patient experience narratives to promote local patient-centred quality improvement: an ethnographic process evaluation of ‘accelerated’ experience-based co-design. J Health Serv Res & Pol. In press.

26.

Hunt

France

Wyke

. My brain couldn't move from planning a birth to planning a funeral: a qualitative study of parents' experiences of decisions after ending a pregnancy for fetal abnormality. Int J Nurs Studies 2009; 46: 1111–1121.

27.

Hall

Gray

Browne

. A qualitative exploration of the role of primary care in supporting colorectal cancer patients. Support Care Cancer 2012; 20: 3071–3078.

28.

Charteris-Black

Seale

. Men and emotion talk: evidence from the experience of illness. Gender and Language 2009; 3(1): 81–113.

29.

Hilton

Hunt

Emslie

. Have men been overlooked? A comparison of young men and women's experiences of chemotherapy-induced alopecia. Psycho-Oncolocy 2008; 17: 577–583.

30.

Ziebland

Kokanovic

. Emotions and chronic illness. Chronic Illn 2012; 8(3): 159–162.