Mixed methods for telehealth research

Introduction

Mixed methods research (MMR) is an emerging and evolving research methodology that requires both qualitative and quantitative approaches within the same study. The methodology is not simply two distinct investigations, but is designed to integrate the qualitative and quantitative components together to answer one’s research question. Hence, a complete definition of MMR is

an approach to research in the social, behavioural and health sciences in which the investigator gathers both quantitative and qualitative data, integrates the two, and then draws interpretations based on the combined strengths of both sets of data to understand research problems. ¹

MMR is suitable for program evaluation and translational research, and has been credited with helping move findings to policy and normal service delivery.^3,4 Within the telehealth field, there has been a rapid increase in the number of MMR studies reported in the literature (Figure 1). OPEN IN VIEWER

The axiom that quantitative methods will tell you “what” has happened whereas, qualitative methods will tell you “why” are combined in MMR to explain “what has happened and why it has happened”, hence increasing the confidence in research findings. ² For example, in a study on caregiver support ⁵ a quantitative survey showed an e-health intervention was rated significantly higher than conventional caregiver support. Qualitative semi-structured interviews found this was because it allowed caregivers to select information and services in accordance with their personal needs.

How to conduct MMR

Study design

Mixed methods proponents have identified six types of studies (Box 1) ¹ ; however, we will highlight the three main ways in which the quantitative and qualitative components are designed and sequenced to achieve the aims of one’s project.

OPEN IN VIEWER

Box 1.

Mixed methods study types.

Study types
1. Convergent parallel
2. Sequential exploratory
3. Sequential explanatory
4. Embedded
5. Transformative
6. Multiphase

Convergent parallel

Also known as concurrent triangulation, this is the most common type of MMR. The quantitative and qualitative data are gathered independently and then analysed to answer the same research question or contribute to the understanding of a particular phenomenon. The researcher needs to be open to the possibility that the sources may corroborate, complement or contradict each other. The Whole Systems Demonstrator trial in the UK is a well-known example in home telehealth, where the study was intended to be a precursor to broad-scale implementation; but the quantitative findings of no changes to quality of life ¹² or service use ¹³ did not support this goal. An ethnographic enquiry conducted throughout the trial found that the rigidity of the trial design precluded adaptation to local conditions and showed how this hindered a broader implementation. ¹⁴ This was supported by a quantitative analysis indicating that there was little impact on primary care services. ¹⁵

Sequential exploratory

In this design, the qualitative inquiry and analysis occurs first, and the findings are used to develop the quantitative research. This method is particularly useful when beginning a medium- to long-term program of research in an area that is not well understood, and fits well with the developmental approach to telehealth research described by Fatehi et al. ¹¹ The earlier stage of telehealth acceptability and feasibility testing is often qualitative, using methods such as focus groups or interviews with patients, clinicians and health service managers. These results then give researchers guidance on which outcomes should be chosen and measured in subsequent studies. For example, in the telehealth field, Normalization Process Theory was constructed from qualitative studies of telehealth services, ¹⁶ which then led into the development of an instrument to measure technology adoption readiness. ¹⁷

Sequential explanatory

By contrast, in an explanatory MMR design the quantitative data is analysed first, and then the qualitative study is planned and executed to gain further understanding of the outcomes. This can be particularly useful when the quantitative results are unexpected, such as understanding why telehealth was less efficient than usual care, or what has produced a high withdrawal rate by participants. The qualitative component can also uncover issues impacting on the validity of the quantitative data. To give one example, a mixed methods study of electronic palliative care coordination initially indicated the system was effective, but subsequent interviews of those using the system showed this result was due to selection bias. ¹⁸

Outcome measures

In considering the outcomes which may be of interest to stakeholders, the key areas are identified as: clinical outcomes (representing the provider and professional perspective); impact on health-related or general quality of life (reflecting the patient perspective); and resource utilization (reflecting the payer and societal perspective). ¹⁹ Each will be discussed from the MMR perspective.

Data collection

Survey research and qualitative interviews account for the vast majority of data collection methods used in MMR. ²⁴ A range of data collection strategies for MMR are presented in the online supplementary material. It is important that protocols are in place to ensure a rigorous approach to data collection, with a priori decisions regarding data cleaning and dealing with data conflict clearly identified. A telehealth research study developing a framework for in-home monitoring to help people with vision impairment is an example of the use of different forms of data collection. ²²

Data analysis and integration

Integration, referring to how qualitative and quantitative components are brought together, is both the most important and the most difficult aspect of MMR. ² The aim is to maximize the strengths and minimize the weaknesses of each type of data. ²⁵

Integration can occur at data collection, data analysis or the data interpretation (i.e. discussion) phase of MMR. The data collection phase may involve the collection of both open- (qualitative) and closed-ended (quantitative) questions. There are two main approaches to integration in data analysis. The first is where the quantitative and qualitative databases are analysed separately using techniques usually associated with each type of data – for example, quantitative data is analysed statistically and qualitative data is analysed using thematic analysis, preserving the integrity of each data set. ²⁶ The second and less common approach involves transforming the data sets. Qualitative data can, for example, be converted to quantitative data by numeric coding using summative content analysis. ²⁷ Lastly, integration may occur in the interpretation section of the study; a popular way to structure this is to discuss the quantitative and qualitative results separately and then consider the corroboration or contradiction of the two findings. Another approach is to develop a table or graph that illustrates results from both data sets. ¹

Hybrid approaches to integration can also be used, such as using one dataset (e.g. a quantitative survey) to inform subsequent data collection (e.g. identification of participants to interview). Another hybrid approach to integration involves a secondary data collection within a larger primary study. For example, a small qualitative study exploring the barriers to participation and the adoption of telehealth was nested with the Whole System Demonstrator cluster randomized trial. ²⁸

Reporting MMR

MMR reports are often lengthier than single-method studies, and many medical science journals have relatively restrictive word limits which may preclude the publication of complex, mixed methods studies. ² Hence, authors will need to target specific journals for the publication of their MMR study.

O’Cathain et al. ²⁹ have developed a framework to guide the structure, reporting and quality assessment of MMR studies, called good reporting of a mixed methods study (GRAMMS). The constructs of this framework are summarized in Box 2. Other quality assessment and appraisal tools that may be useful for reporting and assessing MMR include: an appraisal tool to assess the quality of a MMR study that has been the subject of reliability research ³⁰ ; the United States National Institutes of Health “Best Practices for Mixed Methods in Health Sciences” ²⁵ ; and a framework for evaluating the inclusion of required elements of both the quantitative and qualitative components of a MMR study. ³¹

OPEN IN VIEWER

Box 2.

Good reporting of a mixed method study (GRAMMS) framework.

1. Describe the justification for using a mixed methods approach to the research question.

2. Describe the design in terms of the purpose, priority and sequence of methods.

3. Describe each method in terms of sampling, data collection and analysis.

4. Describe where integration has occurred, how it has occurred and who has participated in it.

5. Describe any limitation of one method associated with the presence of the other method.

6. Describe any insights gained from mixing or integrating methods.

Common issues with MMR

Bryman ²⁴ suggested that most studies alleging to be mixed methods do not illustrate an integration of the two data sets. Hence, authors claiming to use a mixed method design should demonstrate how and where the integration has occurred. Practical tools to assist with this are the procedural flowcharts with sample studies developed by Zhang and Creswell ³² for three different methods of integration (merging, connection and embedding), and Onwuegbuzie and Teddlies’ seven-step conceptual framework for data analysis and integration (see Table 1). ³³

OPEN IN VIEWER

Table 1.

Framework for data analysis and integration in MMR.

1. Data reduction	Reducing data/finding using appropriate quantitative (e.g. descriptive statistics) or qualitative techniques (e.g. thematic analysis).
2. Data display	Visually presenting data. Tables, graphs can be used to present quantitative data, whereas lists and Venn diagrams can be used to present qualitative data.
3. Data transformation (optional)	Qualitative data converted to numeric codes (quantitized) or quantitative data is converted to narrative (qualitized).
4. Data correlation (optional)	Data correlation is used when data transformation has occurred. It involves correlating the quantitative data and quantitized data and/or qualitative data and qualitized data.
5. Data consolidation (optional)	Quantitative and qualitative data are combined to create new or consolidated variable or data sets.
6. Data comparison	Comparing data from the quantitative and qualitative data sets.
7. Data integration	Integration of both the quantitative and qualitative data into a coherent whole and two separate data sets of coherent wholes.

Researchers tend to be skilled in either quantitative or qualitative methods, hence there can be an imbalance in the quality of the quantitative and qualitative components of a mixed methods study if it is undertaken by a single researcher. Forming research teams with complementary expertise is recommended, in particular by seeking out those with qualitative expertise, rather than assuming it is a simple matter for those with little experience in conducting interviews to add these to a study as an afterthought. Bowers et al. ³⁴ give a detailed explication of how MMR teams can be created and supported. It may be faster and simpler for each researcher to take the method with which they are most familiar and conduct their analysis independently, but there are substantial benefits from interdisciplinary collaboration in which all researchers become at least somewhat familiar with each other’s methods and discuss the whole analysis in a regular, coordinated fashion.

Related methodologies

Mixed methods can be used within other methodological paradigms, such as Yin’s case study approach, ³⁵ in which a detailed, holistic investigation is undertaken of one specific organization, project, service or event, with the aim of understanding how and why the phenomenon occurs. Yin provides practical guidance for case study research, including the use of multiple data sources, including documents, interviews, observation and physical objects. The main analytic approach is concurrent triangulation, although multiple case studies can be conducted to provide comparators. For a telehealth example see the work by Singh et al. on the sustainable adoption of telehealth in a rural district. ³⁶ The theoretical approaches of ethnography and the socio-technical approach have also been combined by Greenhalgh et al. ³⁷ in researching the uptake of e-health and telehealth. Their work on the abandonment of a personal electronic health record in the UK drew from national statistics, policy documents, emails, minutes of meetings, individual interviews and the direct observation of individuals. In a complex research project such as this one, a multi-level analysis was needed, divided into patients, providers, organization, policy and system design.

Conclusion

As MMR is maturing, researchers should aim to use the methods as well as possible. We have described established study designs, data analysis methods and reporting frameworks to assist researchers in conducting a high-quality mixed-methods study. Integration of the quantitative and qualitative components of the study is considered the most important and most difficult aspect of MMR and researchers should ensure they demonstrate how and where the integration has occurred. It is recommended that MMR be conducted by a team of researchers, which will assist the goals of achieving integration within the study, and hence produce a more coherent and valuable research report. This, in turn, will help those seeking information on the uses and implementation of telehealth.