Artificial Intelligence and Qualitative Analysis of Emergency Department Telemental Health Care Implementation Survey

Abstract

Background

: Implementation of telemental health care in emergency departments (EDs) in the United States (U.S.) has been increasing. Artificial intelligence (AI) can augment traditional qualitative research methods; little is known about its efficiency and accuracy. This study sought to understand ED directors’ qualitative recommendations for improving telemental health care implementation and to understand how AI could facilitate analysis of qualitative survey responses.

Methods

: Directors at a nationally representative sample of 279 U.S. EDs that used telemental health care completed an open-ended survey question about improving telemental health care implementation between June 2022 and October 2023. Two groups of researchers completed independent qualitative coding of responses: one group used traditional qualitative methods, and one group used AI (ChatGPT 4.0) to facilitate analysis. Both groups independently developed a codebook, came to consensus on a combined codebook, and each group independently used it to code the survey responses. The two groups identified themes in ED directors’ recommendations and compared codebooks and code application across traditional and AI approaches.

Results

: Themes included (1) recommendations for improving telemental health care directly and (2) recommendations for improving mental health care systems broadly to make telehealth more effective. ED directors’ most common recommendation was enabling faster and more streamlined access to telemental health care. AI augmented human coding by identifying two valid codes not initially identified by human analysts. In codebook application, 75% of responses were coded consistently across AI and human coders.

Conclusions and Relevance

: For US EDs using telemental health care, there is a need to improve timeliness and efficiency of access to telemental health care.

Introduction

IMPLEMENTING EMERGENCY DEPARTMENT TELEMENTAL HEALTH CARE

Use of telemental health care has been increasing,¹ and developing standards for implementation remains an opportunity. Given longstanding shortages of mental health clinicians,² telehealth can expedite patients’ access to mental health clinicians in emergency departments (EDs). Most EDs lack local resources to meet patient mental health needs, and telehealth shows promise for relieving issues related to limited capacity (e.g., mental health boarding) and for improving factors that lead to ED clinician burnout.³ Existing literature focuses on perceived barriers and benefits of telehealth for patients at risk of suicide,⁴ yet little attention has been paid to opportunities for improving how telemental health care is implemented. Research to understand how EDs can improve implementation of telemental health care might provide information to augment EDs’ capacity to serve patients during mental health crises.

Within the context of a larger structured survey focusing on EDs’ telemental health care processes, our team asked ED hospital leaders to give their recommendations for improving implementation of telemental health care in their ED. Typically, open-ended survey responses are analyzed using time-intensive qualitative analysis techniques. Our research team sought to use this opportunity to understand whether artificial intelligence (AI) could introduce efficiencies or complement human qualitative analysis.

USE OF ARTIFICIAL INTELLIGENCE IN QUALITATIVE RESEARCH STUDIES

Use of AI is increasing, with OpenAI releasing ChatGPT in 2022 and companies competing to release similar software.⁵ Existing studies have found that AI recognized themes that human qualitative researchers did not, and vice versa.^5–6 Agreement between AI versus human coding data has been low in prior studies.⁷ Further, since AI is known to fabricate information,⁸ human qualitative researchers are necessary for their content expertise and to check results’ validity.⁹ Nevertheless, when appropriately guided by human researchers, AI analysis may be time- and cost-efficient, and the extent to which AI analysis can be employed in qualitative research warrants investigation.⁹

Our objectives for the present study were to (1) identify themes among ED leaders’ recommendations for improving telemental health care and (2) compare relative accuracy and efficiency of traditional qualitative analysis versus AI-assisted qualitative analysis.

Methods

This study compares use of AI (i.e., ChatGPT 4.0¹⁰) with traditional qualitative methods for analysis of responses to an open-ended survey question asking ED directors about suggested improvements to telehealth. The study evaluates agreement between methods and time required for each method of analysis (traditional vs. AI-augmented).

SETTING

The data for this study are derived from a national, cross-sectional survey of hospital ED medical or nursing directors conducted in 2023. The survey was sent to a nationally representative sample of hospital EDs stratified on U.S. geographic region and urban/rural status to ensure proportionate representation from U.S. geographic regions and rural hospitals. Survey recruitment procedures followed established recruitment methods.¹¹ At each hospital, our team obtained publicly available ED nurse manager/medical director and sent an electronic REDCap survey and paper survey packet at timed intervals until we received a response. Each ED had a unique identifying code so the ED’s responses could be linked to ED characteristics and to prevent multiple responses. ED directors were not required to provide their identifying information. Participants gave informed consent, and those who completed the survey were remunerated with a $100 gift card. The study was deemed exempt from review by the (anonymized for review) institutional review board. The funder had no role in the study’s design, implementation, or analysis.

The main survey was primarily composed of closed-ended questions describing the ED’s current suicide prevention and telehealth practices.

The focus of the present study is the survey’s final, optional, open-ended question: “If you could change one thing to improve the effective use of telemedicine for suicide prevention in your ED, what would it be?” Responses varied in length from one word to three sentences.

The survey had a total of 606 responses from the 977 eligible EDs (62% response rate overall). Of the 606 responding EDs, 413 EDs reported use of telehealth services. From the subset of 413 telehealth EDs, the present analysis includes all 279 respondents who responded to the open-ended question (68% response rate among EDs that provide telehealth). Telehealth EDs responding to the survey included 86 (31%) that had tele-social work only, 99 (35%) that had tele-psychiatry only, 74 (27%) that had both tele-psychiatry and tele-social work, and 20 (7%) that had neither tele-psychiatry nor tele-social work, only other professionals (e.g., case manager, mental health nurse). Among EDs with telehealth, 21 (8%) used telephone only and 258 (92%) used audio and video for telehealth.

DEVELOPING A CODEBOOK: TRADITIONAL QUALITATIVE ANALYSIS

One team of two researchers analyzed survey responses using ChatGPT (i.e., AI-Augmented Analysis), while a second team of two different researchers analyzed the data using traditional qualitative methods. One research team member did not participate in either analyses or organized and facilitated team decision-making. Each team independently developed a codebook to categorize the responses.

Both researchers using Traditional Qualitative Analysis developed a draft codebook by open coding¹² responses and aggregating like items into categories. After independently developing codebooks, the Traditional Qualitative Analysis team compared codes, reconciled codebooks, and identified 10 codes.

DEVELOPING A CODEBOOK: AI-AUGMENTED ANALYSIS

AI-Augmented Analysis began with a message to ChatGPT including the survey question, all survey responses, and a prompt. Specifically, we asked, “I did a research study where I conducted a large national survey of leaders in general EDs to examine the relationship between use or absence of telehealth, use of suicide prevention EBPs, and subsequent patient outcomes. We asked all respondents the following question: If you could change one thing to improve the effective use of telemedicine for suicide prevention in your ED, what would it be? Please create a limited set of thematic categories from their responses below.” All survey responses uploaded to ChatGPT were de-identified and included no information about the respondent or ED characteristics.

ChatGPT produced 10 codes with 2–3 bullet points explaining each code. We asked ChatGPT to provide a codebook with an additional command: “Please produce a codebook that provides definitions and examples for these themes that qualitative researchers can use as a guide when coding the open-ended responses.” As a check on the quality of the ChatGPT-generated codebook, the researchers reviewed ChatGPT’s outputs and examined whether any survey responses did not fit into the categories produced by ChatGPT. The researchers determined that ChatGPT’s codes did not apply to responses that indicated telehealth was working well, so the team created an additional code, No Changes Needed to Telehealth, to add to ChatGPT’s codebook. This addition resulted in 11 codes in the AI-augmented codebook.

After Traditional Qualitative and AI-Augmented codebook development was complete, the entire research team met to compare codes developed using each method and to reconcile differences across the two codebooks. The research team reached consensus on a final unified codebook with 11 total codes that fell into two broad thematic categories: improvements to telemental health care and improvements to the mental health care system. Each theme in the codebook is illustrated by an example quote from the survey responses (Table 1). Both teams used the same reconciled codebook for subsequent codebook application.

Table 1.

Codes with Example Quotes From Qualitative Analysis of Open-Ended Responses to Survey of Emergency Department Directors about Recommended Improvements to Tele-Mental Care

CODES	EXAMPLE RESPONSE
1. Need for faster and/or more streamlined access to telehealth consultation in the ED	“Faster response times for telemental health assessments to be completed.”
2. Patient Evaluation and Care Processes, including Care Delivered During Boarding	“Change the requirements for the mental health evaluation. Lab work and urine tox always required. Makes patient have to wait hours often to be evaluated. Sometimes these do not need to be done for the evaluation to take place.”
3. Health System Technology	“Crossover into EHR used in ED to avoid use of faxing.”
4. No Changes Needed	“Our system works well.”
5. Education and Training	“Encourage providers to use [telehealth] on 100% of patients that present with mental health concerns.”
6. Services for Specific Populations	“Better access to professionals for substance abuse disorder.” “Have pediatric telehealth specialists for the younger peds [patients] we get and have to ship elsewhere.”
7. Policy and Regulation Changes	“In [state], the local county behavioral health decides who can place holds on patients and our county only allows their evaluators to place or lift holds limiting the usefulness of telemedicine psychiatrists.”
8. Health System Infrastructure	“Be able to provide a more private area for interview and assessment.”
9. Mental Health Care Programs and Providers	“Expanding telehealth platform to include evaluation by psychiatrist.”
10. Collaboration and Continuity of Care	“I would like to have better outpatient support to allow for scheduling of close follow up before discharge from the ED.”
11. Bed Availability and Patient Placement	“Better access to psychiatric care after the ER experience. Often patients are days (if not weeks) awaiting a bed at a psychiatric facility.”

CODING SURVEY RESPONSES

The teams using Traditional Qualitative and AI-Augmented Analysis adopted independent procedures for codebook application. Each member of the Traditional Qualitative Analysis team coded the same one-fourth of responses using the agreed-upon codebook. They then met to review coding, reconciled minimal differences in application, and continued to code the remaining data. The goal for the Traditional Qualitative Analysis was to assign a single code to each response; however, given the open-ended question and respondents’ freedom to provide as much detail as possible, some responses reflected multiple codes. In this limited number of cases, the research team indicated a secondary and tertiary code, in addition to the primary code.

The AI-Augmented Analysis team used Chat GPT 4.0 to classify each code into one of the eleven thematic groups. GPT was provided with the codebook and the illustrative example to guide its classification. “Using the codebook I provided with the eleven categories, can you code the following survey responses into the category each response fits best? Upon completion, generate a spreadsheet with each survey ID, the responses, and the corresponding code you assign to it. Here is a list of the survey responses: …”

The teams compared agreement between codes across Traditional Qualitative Analysis and AI-Augmented Analysis. Throughout all procedures, each research team member tracked how much of their own time was spent for each approach, and we summed time spent by all team members to measure total time.

Results

CODEBOOK DEVELOPMENT

As shown in Table 2, both Traditional Qualitative Analysis and AI-Augmented Analysis produced similar codebooks, with AI-Augmented Analysis providing new insights. Overall, we organized the eleven codes in the integrated codebook into two broad themes: (1) improvements to telemental health care implementation and (2) improvements to the mental health care system.

Table 2.

Comparison of Codes and Code Descriptions Included in Codebooks Produced through Traditional Qualitative and Artificial Intelligence (AI)- Augmented Analysis

THEME	FINAL INTEGRATED CODEBOOK	INITIAL CODEBOOK INDEPENDENTLY PRODUCED THROUGH TRADITIONAL QUALITATIVE ANALYSIS	INITIAL CODEBOOK INDEPENDENTLY PRODUCED THROUGH ARTIFICIAL INTELLIGENCE (AI)-AUGMENTED ANALYSIS
I. Improvements to Telemental Health care Implementation	1. Need for faster and/or more streamlined access to telehealth consultation in the ED: Use when participant mentions barriersto accessing or comments about the need for more mental health professionals via telehealth while a patient is in the ED. Theme includes mentions of long waits for patient to see a telehealth provider, wish for quicker access for ED clinician to consult a telehealth clinician	Increase Availability/Ease: Including provider type, times, days available. Faster Telehealth: Reduce delays	Provider Availability and Responsiveness: Comments about the need for more mental health professionals available via telehealth and improvements in their responsiveness
	2. Patient Evaluation and Care Processes, including Care Delivered During Boarding: Use when participant mentions care provided to the patient in the ED, including initial evaluation, reassessment, any procedures, medication management and therapy provided in ED	Help with Boarding Issues: Including bed placement, medication management, reassessments during stay, therapy during stay etc	Patient Evaluation and Care Processes: Feedback on streamlining the evaluation process for patients, including reducing wait times and unnecessary procedures
	3. Health System Technology: Use when participant mentioned technologies, including hardware (e.g., telephone, tablet, computer, and video access), software (e.g., EHR or other software), and linkages of technology (e.g., ease of accessing telehealth information in EHR.) (Ultimately split into two codes, see also: Health System Infrastructure)	Reduce institutional barriers: Reduce costs, improve tech, incorporation of tele into EHR, etc	Technology and Infrastructure:Suggestions on improving the technological tools, privacy, and physical infrastructure necessary for telemedicine
	4. No Changes Needed: Use when participant discusses being satisfied with existing implementation of telehealth	Good	No Changes Needed to Telehealth *Added after researchers reviewed responses not categorized by AI-generated codebook
	5. Education and Training: Use when the participant mentions the need for better training and education for staff on telemedicine and mental health care practices	Education	Education and Training: Emphasizes the need for better training and education for staff on telemedicine and mental health care practices
	6. Services for Specific Populations: Use when participant mentions specific patient populations, including child/adolescent, geriatric, uninsured, individuals with substance use	Special Populations Included In Telehealth Programs: Including peds, geriatric, uninsured, OUD/SUD	Specialized Services and Integration: Recommendations for integrating specialized services or improving the coordination between telemedicine and other health services
	7. Policy and Regulation Changes: Use when participant mentions changes in policies or regulations to facilitate better telemedicine use or mental health care		Policy and Regulation Changes: Feedback calling for changes in policies or regulations to facilitate better telemedicine use or mental health care
	8. Health System Infrastructure: Used when participant discussed physical space in the ED, including beds, curtains, privacy, safety. (Ultimately split into two codes, see also: Health System Technology)	Reduce institutional barriers: Reduce costs, improve tech, incorporation of tele into EHR, etc	Technology and Infrastructure: Suggestions on improving the technological tools, privacy, and physical infrastructure necessary for telemedicine
II. Improvements to mental health care system	9. Mental Health Care Programs and Providers: Use when participant mentions availability of or funding for mental health care providers and programs in any context, including ED, hospital, community, and outpatient programs	More Resources: In the community, inpatient, and outpatient	Funding and Resources: Suggestions for securing funding or resources to support telemedicine and suicide prevention efforts
	10. Collaboration and Continuity of Care: Use when participant mentions collaboration between providers while the patient is in the ED as well as collaboration across care settings (e.g., between the ED and outpatient.)	Communication and Collaboration: Between telehealth and care team and outpatient (all providers involved in the care of a child.)	Follow-Up and Continuity of Care: Ideas for improving the follow-up care and ensuring continuity of care post-ED visit
	11. Bed Availability and Patient Placement: Use when participant mentions the availability of inpatient beds and the process of placing patients in appropriate care settings	Help with Boarding Issues: Including bed placement, medication management, reassessments during stay, therapy during stay etc	Bed Availability and Patient Placement: Concerns and suggestions regarding the availability of inpatient beds and the process of placing patients in appropriate care settings

OUD/SUD, Opioid Use Disorder/Substance Use Disorder.

Improvements to telemental health care reflected responses about increasing the efficiency of access to telehealth by increasing telehealth provider availability and removing bureaucratic barriers to enhance telehealth’s utility. By far, the most common responses focused on the timeliness of telehealth access, with one respondent noting, “It takes a long time to get a mental health practitioner available to help the patient,” and numerous respondents calling for improvements like “timely callback” and “faster response” to address concerns like “significant wait times.” Some providers wished for telehealth providers to take on a more expansive role, with the abilities to manage medication, provide therapy or peer support, coordinate disposition, and follow up with community providers. For example, one respondent wished for a more comprehensive telemental health care program, such as an “all-inclusive model that had medication recommendations, IP placement, and automatic social worker follow-up for 30 days. Also being directly affiliated with our region’s IP facilities to ensure that our patients stayed close to home instead of getting transferred across the state for IP care.” Another respondent suggested, “more consistent psychology or peerto-peer support rather than the standard ‘psychiatry/medical’ evaluation.” Several respondents noted that certain specialties were not accessible via telehealth in their ED, including social work, masters’ level clinicians, and psychiatrists.

In addition, respondents wished for technological and infrastructure improvements so that telemental health care could take place effectively and privately. Examples of specific technological improvements ED directors suggested included “integrate with Electronic Medical Record (EMR)” and “video rather than phone only.” As far as infrastructure, directors noted concerns about privacy—they wished their EDs could “provide a more private area for interview and assessment.” Several directors also noted that providing telemental health care in the main ED affected safety and patient flow.

The second theme, improvements to the mental health care system, identified pervasive issues in the mental health care system that prevent the full realization of telemental health care’s benefits. Numerous respondents simply called for “more resources.” Specific needs that ED directors mentioned included more beds in psychiatric facilities (“placement is an issue; finding an open bed”), better availability of outpatient mental health providers (“ensuring adequate follow-up and outpatient appointment availability to our patients”), and more community psychiatric resources (“access to psychiatrists in the region”). As one respondent put it, directors recognize a need for “More access to outpatient and inpatient services and expedited care to psychiatric services. When patients hold in the ED, not receiving services, time to most appropriate care is also delayed.”

CODEBOOK APPLICATION

Fig. 1 shows how frequently each code was applied in AI-Augmented analysis and Traditional Qualitative Analysis. Across all codes, AI-Augmented Analysis and Traditional Qualitative Analysis applied the same code to 70.2% of survey responses. The proportion coded the same by both methods was highest for the code: No Changes Needed (92%) and lowest for the codes: Health Systems Infrastructure (33%) and Policy and Regulation Change (33%).

Fig. 1.

Codebook Application across Traditional Qualitative Analysis and Artificial Intelligence (AI)-Augmented Analysis.

TIME SPENT

The team using traditional qualitative analysis produced a codebook in 6 h and 35 min and coded responses in 2 h and 45 min. The team using AI produced a codebook in 3 h and 30 min and coded responses in 7 h and 20 min. Fig. 2 illustrates time spent on codebook development and codebook application by both Traditional Qualitative Analysis and AI-Augmented Analysis.

Fig. 2.

Researcher Time Spent on Traditional Qualitative Analysis versus Artificial Intelligence (AI)-Augmented Analysis of Open-Ended Survey Responses.

Discussion

In their open-ended responses within a national survey, ED leaders highlighted opportunities for improving implementation of telemental health care delivery, such as the need for faster access and increased availability of providers, and highlighted the need for collaboration and coordination between telehealth and in-person care. In addition, ED leaders highlighted opportunities for improving mental health care broadly, such as the need for better follow-up upon ED discharge. Using AI to augment traditional qualitative analysis helped our team produce a more comprehensive codebook in less time; however, in application of the codebook, AI and human coding aligned for only 75% of survey responses.

Despite the promise of telehealth to improve timely access to mental health specialists,^3,4,13 our findings suggest that provider shortages and delays in access to mental health care persist, even among EDs that have telehealth. These current delays and inefficiencies represent an opportunity for public and private sector entities to meet ongoing demand for mental health services. Reducing time to mental health evaluation and disposition has the potential to help reduce ED crowding, boarding, and wait times, and EDs may be willing to make larger financial investments in telemental health care services if they can improve ED efficiency. Telemental health care programs may also benefit from adopting innovations used by other well-established remote services such as 24/7 tele-radiology programs that offer real-time reads on time-sensitive emergency radiology exams. Future research to understand what would incentivize clinicians to work in telemental health care would also be beneficial.

We found that for qualitative codebook development, AI was efficient and provided valuable insights. The initial codebook developed by AI had close alignment with the codebook developed by human coders, including identification of one novel code that humans did not identify. For our study, had we used AI to develop the initial codebook and then worked with human analysts to refine the codebook and apply the codes, we would have saved time overall and developed a valid codebook that allowed for insights into the data. However, AI-augmented analysis was less accurate and more time-consuming in codebook application when compared with traditional qualitative methods.

Using AI to apply codes to our dataset did not closely replicate human qualitative researchers. Further, researchers completing AI-Augmented analysis were less familiar with the data at the end of analysis than the traditional qualitative analysts. Data familiarization and immersion are integral parts of qualitative analysis.¹⁴ When using AI to augment analysis, these steps were essentially omitted. During thematic analysis and article development, AI Analysis team members found it more difficult to contribute thoughtfully to the results and discussion. Our findings are consistent with research on AI-assisted qualitative research in other disciplines.^5
–7

Some limitations were inherent in this study. Use of AI at the time of this study is a snapshot of ChatGPT 4.0’s capabilities in April 2024; future versions are likely to offer different capabilities. Estimates of time spent for codebook application using AI-Augmented analysis reflect time spent learning to effectively use GPT for this kind of task. Future coding exercises could proceed more quickly, though perhaps still not as efficiently as human coders. Survey responses to our open-ended question were brief, and responses were optional, so selection effects exist. Further in-depth qualitative inquiry could reveal more nuanced ideas about areas for improvement. Nevertheless, the responses in this study are derived from a nationally representative survey of U.S. ED leaders, which underscores the generalizability of these findings.

Conclusions

Leaders at EDs using telemental health care called for telemental health care services to offer faster access to mental health clinicians, adaptations to better fit telemental health care into the physical ED, such as allowing for rooms to be more private and safe, and access to a broader array of specialties and services via telehealth. AI efficiently developed a codebook to categorize survey responses but did not apply codes with high accuracy. Qualitative researchers should consider the utility of using AI software such as ChatGPT to save time during the codebook development process. However, given the time spent and relatively low agreement with Traditional Qualitative coding, AI may be less useful during the codebook application process.

Footnotes

Authors’ Contributions

C.K.: Contributed to conceptualization of the project, development of the methods, data collection, data curation and analysis, data visualization and drafting of the original article. S.C.M.: Contributed to conceptualization of the project, funding acquisition, project administration, supervision, development of the methods, data curation and analysis, and review and editing of the original article. C.F.B.: Contributed to conceptualization of the project, project administration, development of the methods, data collection, data curation and analysis, and review and editing of the original article. D.W.: Contributed to conceptualization of the project, project administration, development of the methods, data collection, data curation and analysis, and review and editing of the original article. S.K.D.: Contributed to conceptualization of the project, funding acquisition, project administration, supervision, development of the methods, data curation and analysis, and drafting, review, and editing of the original article.

Author Disclosure Statement

S.C.M. has received consulting fees from Jannsen Global Services, LLC.

Funding Information

The study was supported by the National Institute of Mental Health (R56MH124656).

References

Bowden

, Worsley

, Doupnik

. Mental health and COVID-19 in pediatric emergency departments: Perspectives from directors. J Adolesc Health, 2022; 71(3):360–363.

Thomas

, Ellis

, Konrad

, et al. County-level estimates of mental health professional shortage in the United States. Psychiatr Serv, 2009; 60(10):1323–1328; doi: 10.1176/ps.2009.60.10.1323

Reliford

, Adebanjo

. Use of telepsychiatry in pediatric emergency room to decrease length of stay for psychiatric patients, improve resident on-call burden, and reduce factors related to physician burnout. Telemed J E Health, 2019; 25(9):828–832; doi: 10.1089/tmj.2018.0124

Ward-Ciesielski

, Peros

, Conigliaro

, et al. Perceived benefits of psychotherapy via telemedicine based on suicide risk severity. Gen Hosp Psychiatry, 2018; 55:100–101.

Hamilton

, Elliott

, Quick

, et al. Exploring the use of AI in qualitative analysis: A comparative study of guaranteed income data. Int J Qual Methods, 2023:22; doi: 10.1177/16094069231201504

Bastiaansen

, Veldhuizen

, De Schepper

, et al. Experiences of siblings of children with neurodevelopmental disorders: Comparing qualitative analysis and machine learning to study narratives. Front Psychiatry, 2022; 13:719598.

Bano

, Zowghi

, Whittle

. AI and human reasoning: Qualitative research in the age of large language models. AIEJ, 2024; 4(2).

Verma

. How AI Fake News Is Creating a ‘Misinformation Superspreader.’ 2023. Available from: https://www.washingtonpost.com/technology/2023/12/17/ai-fake-news-misinformation/ [Last accessed: December 6, 2024 ].

Weissler

, Naumann

, Andersson

, et al. The role of machine learning in clinical research: Transforming the future of evidence generation. Trials, 2021; 22(1):537; doi: 10.1186/s13063-021-05489-x

10.

ChatGPT Version 4.0. Available from: https://openai.com/index/chatgpt/ [Last accessed: March 25, 2024 ].

11.

Dillman

, Smyth

, Christian

. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. John Wiley & Sons; 2014.

12.

Strauss

, Corbin

. Open Coding. In: Social Research Methods: A Reader Routledge: London, UK; 2004; pp. 303–306.

13.

Roberts

, Hu

, Axas

, et al. Child and adolescent emergency and urgent mental health delivery through telepsychiatry: 12-Month prospective study. Telemed J E Health, 2017; 23(10):842–846; doi: 10.1089/tmj.2016.0269

14.

Lacey

, Luff

. Qualitative Data Analysis. Trent Focus Group UK; 2001.