A Mixed-Methods Quick Strike Research Protocol to Learn About Children With Complex Health Conditions and Their Families

Abstract

Advances have been made to improve health care for children with complex health conditions (CCHCs); however, little is known of the needs of these children and their families in the Canadian context. In this article, we describe our Canadian Institutes of Health Research funded Quick Strike protocol, a mixed-methods multisite research project that explored CCHC and their families in two Canadian provinces. The aims were (a) to describe and define CCHC, (b) to understand the needs of CCHC and their families, (c) to identify gaps and barriers to services for this population, and (d) to adapt and test the application of a computerized algorithm to yield information on CCHC. The mixed-methods design was comprised of four components: three qualitative and one quantitative. We describe the components of this project and outline the methods and procedures of data collection and analysis for each component. One of the main sources of data was interviews from 121 stakeholders, which included CCHC and family members, as well as health, social, and education professionals. This Quick Strike project was designed to engage stakeholders and the public with integrated knowledge translation threaded as a core element throughout the research process. Multiple strategies were used to validate and disseminate early findings from the research. As we outline in this article, this research project provided the foundation for one innovative service model of care, NaviCare/SoinsNavi, and spawned a number of additional outcomes such as a secondary analysis of the data to describe interprofessional collaboration for CCHC.

Keywords

mixed method qualitative methodology qualitative analysis quantitative analysis computerized algorithm protocol pediatrics

What Is Known

Research is limited for children with complex health conditions (CCHCs), largely because of the complexity of this population and their needs. Complex clinical research questions require a multi-method approach that involves engagement for diverse stakeholders throughout the research process; however, there are few examples available of protocols for multi-method approaches similar to our study.

What This Paper Adds

This protocol provides a guide for other researchers who are interested in conducting a complex multi-method research study that uses integrated knowledge translation (iKT) methods. In addition, this protocol can provide a starting point for researchers wishing to further explore CCHCs or wish to study other populations in similar contexts.

Introduction

Childhood is a time of happiness and good health for most children. Unfortunately, this is not the reality for all children; approximately 15–18% of children in North American have a chronic condition that impacts their health and causes limitations in their lives (Arim et al., 2015; Berry et al., 2011; Kaufman, Pinzon, & Canadian Paediatric Society Adolescent Health Committee, 2007). Advances in neonatal, medical, and nutritional care have improved the survival rate of vulnerable infants who are often left with complex health conditions (Burns et al., 2010). A limitation of research for this population is that experts do not agree on a common definition of CCHCs, thus various terms have been used in the literature to describe this cohort of children. The range of conditions and complexity varies, but this population includes a small but growing subset of children with medical complexity, who tend to have functional difficulties, reliance on technology for activities of daily living, and high health-care resource utilization (Cohen et al., 2011).

CCHC and their families require a multitude of services across a range of settings, including health, education, and social services, yet parents report significant unmet needs. Many studies are conducted in the United States and less is known about the needs of CCHC and their families in Canada when faced with navigating the health-care system and other sectors to access the services they need. Thus, this article outlines our mixed-methods design for a Canadian Institutes of Health Research (CIHR) funded Quick Strike research project that explored the nature of CCHC in two Canadian provinces with a particular view (a) to describe and define CCHC, (b) to gain an understanding of the needs of CCHC and their families, (c) to identify gaps and barriers to services for this population, and (d) to explore what information is gained on CCHC by adapting and applying a computerized algorithm to administrative data sets.

Background

Although there is no single definition for complex health conditions, for this protocol we considered CCHC as those children ranging in age from 0 to 19 with one or more chronic physical, mental, developmental, neurological, or behavioral condition(s) who require health and other services from multiple care providers, often across multiple locations (Cohen et al., 2012; McPherson et al., 1998). These children and their families, particularly children with medical complexity, comprise a cohort of high volume utilization of the health-care system and related services, requiring a coordinated effort by the family and the care team to meet their complex needs (Berry, 2015; Cohen et al., 2012; Kaufman et al., 2007).

As noted by families of CCHC, the health-care system is fragmented and difficult to navigate (Kuo, Cohen, Agrawal, Berry, & Casey, 2011). In addition to frequent illnesses, CCHC often have educational and social issues that need to be addressed (Burns et al., 2010). Caring for CCHC often involves direct home care, hospitalizations, emergency room visits, and countless appointments with specialists in different fields (e.g., family physicians, audiologists, physiotherapists, psychologists, and social workers). Based on a national sample of children with special health-care needs in the United States, Kuo et al. (2011) examined the national prevalence of more complex children with special health-care needs and the challenges faced by caregivers. Children with more complex needs included those with medical technology dependence and care by more than two subspecialists. Approximately 3% of children with special health-care needs met the complex criteria. Families of these more medically complex children spent on average 11–15 hr/wk on direct home care and a median of 2 hr per week on care coordination for these children. Similar research examined the characteristics of hospitalizations for medically complex children and found that these children experienced on average approximately three hospital admissions within a 2-year span, with an average stay of about 12 days per admission (Berry et al., 2011). Berry et al. (2011) also reported that medically complex children experienced roughly a 25% readmission rate to hospital within 30 days. More recent data from Canada showed that CCHC had seen on average four specialists within the previous 6 months of the study (Cohen et al., 2012). These children were medically complex and had largely relied on technological assistance, such as a tracheostomy tube, a feeding tube, or a wheelchair for activities of daily living (Cohen et al., 2012). Caring for CCHC places considerable financial and time commitment stresses on families as care-providers for CCHC. Often families are required to travel long distances for specialized care, particularly when they live in rural or semirural areas (Colver et al., 2013).

In Canada, it is widely recognized that there is a need for integrated primary health care (PHC) delivery models for CCHC that are community-based with linkages to secondary and tertiary care, as well as relevant sectors outside of the health-care system (Berry, 2015; Cohen et al., 2012; Kingsnorth, Lacombe-Duncan, Keilty, Bruce-Barrett, & Cohen, 2015). Shi (2012) describes secondary care as usually short term and based on sporadic consultation (e.g., specialist referral for expert opinion), whereas tertiary care usually reflects the most complex levels of care that typically are institution based and highly specialized (e.g., neonatal intensive care). Providing comprehensive and integrated health-care services for CCHC and their families is challenging across Canada, but particularly so in the provinces of New Brunswick (NB) and Prince Edward Island (PEI), due to limited resources, the diversity of communities, and the rural geography in these provinces. Provincial ministries of health in both NB and PEI are making significant investments in PHC, including investing in programs and services for CCHC. For example, in 2014 the Government of PEI announced an investment of US$3.75 million over 5 years to increase and enhance services for CCHC (Health PEI, 2015). Similarly, in 2009 the Government of NB committed to providing better services and programs for at-risk children, which resulted in a new integrated service delivery (ISD) strategy (Government of New Brunswick, 2010), enabling departments to work together to more efficiently meet the needs of at-risk children.

While the provinces of NB and PEI have been working to develop and implement innovative initiatives for CCHC, the availability of epidemiological data to evaluate these initiatives has been limited. Further complicating matters, there are various ways to identify CCHC using different methodologies and with varying definitions. Having a theoretical and operational definition for a concept, such as for CCHC, is important to both classify and measure phenomenon being studied (Fawcett & Garity, 2009; Walker & Avant, 2011). Given this rationale, we accepted that the definition of CCHC needed to be stable and consistent in identifying children who depend on the integration of programs and services to receive optimal health-care services. A clear operational definition and understanding of CCHC is also needed to inform practice, research, and policy. Finally, very little is known about the health-care needs of CCHC and their families in the Canadian context, and of the gaps and barriers in service for this population.

Goals and Objectives of Research Project

The overall goal of this research project was to improve care for CCHC and their families by generating knowledge that could inform practices of health care and other professionals, future research, and policy development. Four research objectives were established to enable the generation of knowledge in this regard. The first objective was to develop an operational definition for children (ranging in age from 0 to 19) with complex health conditions. The second objective was to explore the medical–physical, mental, social, educational, and spiritual needs of CCHC and their families. Once these needs were identified, the third objective was to identify the services and programs available within the health-care continuum and across sectors to address the needs of CCHC and their families, including resource gaps and barriers to accessing care in NB and PEI. The fourth and final objective was to develop and test a standardized algorithm (Simon et al., 2014) to identify and classify CCHC in NB and PEI into three cohorts of medical complexity. In addition to informing practice, research, and policy, the results from this project were intended to provide a foundation for developing integrated and innovative service delivery models for CCHC and their families.

Methods

For this protocol, we selected a mixed-methods design that involved both separate and integrated use of qualitative and quantitative approaches for data collection and analysis to address four research objectives (Figure 1). Creswell and Plano Clark (2011) described mixed methods as a research design whereby combining and integrating qualitative and quantitative methods is used to gain understanding of complex phenomena through multiple research phases and/or complementary data sources. A mixed-methods design was appropriate to address the four study objectives. Our research project involved: (a) a diverse research team from several sites across NB and PEI, (b) data collection from various sources and settings, (c) iKT and end-of-grant knowledge translation (KT) activities to engage participants and stakeholders in a dynamic and iterative research process (CIHR, 2012), and (d) four research components that corresponded to each of the four study objectives. The research components and timeline are outlined in Figure 2.

Figure 1.

Overview of the mixed-methods design for the research study components.

Figure 2.

Research components and timeline (September, 2015–December, 2016).

Research Team

Our project was initially led by a steering committee that provided oversight to the planning and initiation of the research process. The steering committee included representation from each province and consisted of three principal investigators (PIs), two clinicians, two decision-makers, and patient/family advisors in NB; one advisor represented children from 0 to 9 years old and the other children aged 10–19 years. Inclusion of clinicians, decision-makers, and users of the health system (i.e., patient/family advisors) was considered an instrumental part of iKT and KT activities early on in the research process (CIHR, 2012). The PIs (SD, RA, WM) assembled a team of research associates (e.g., DN), post-doctoral fellows (PC, AL), and independent researchers (e.g., NH) who represented NB and PEI. The team composition facilitated successful completion of the four research objectives. Expertise within the research team included qualitative research (SD, NH, AL, DN) and quantitative research (RA, WM, PC), and involved librarians and research assistants for literature reviews and interviews. The PIs facilitated regular meetings of the team, as well as collaborations and engagement of diverse stakeholders that included patient representatives (e.g., RS) in each jurisdiction, and attended to issues that arose throughout the research process. One such collaboration was with the School of Mathematics and Computational Sciences and the School of Nursing at UPEI for programming applications and incorporating elements of data science structures for large database development.

Research Context and Data Sources

Both NB and PEI are sparsely populated provinces with a large percentage (about 50%) of the population located in rural areas. In addition, NB is a bilingual province with both English and French-speaking residents; which was considered as a potential barrier for access to services. A goal at the outset of the project was to ensure representation in the data from across the geographical regions and population diversity of NB and PEI to better understand CCHC and their families across each province.

Qualitative data sources

For the qualitative components of this project, the main source of data was 97 individual interviews and 2 focus group interviews of participants. Total participants included: (a) CCHC (n = 5), family members of CCHC (n = 32), health professionals (n = 40), education professionals (n = 17), social professionals (n = 26), and one administrator. Other sources of qualitative data included the literature, health-care charts, policies, and sources within the realm of gray literature.

Quantitative data sources

Two sources of data were used for the quantitative phase of this project: (a) health administrative data (i.e., physician billing/claims) were used for the application of a validated algorithm to identify CCHC in PEI and (b) individual case reports collected from chart audit reviews completed in four paediatric health-care clinics in NB and PEI were used to describe the demographic and clinical characteristics of complex conditions and health-care utilization patterns. Transcription of chart information was based on a chart audit process that was guided by a nine-step approach to chart review research methodology outlined by Gearing, Mian, Barber, and Ickowicz (2006).

Qualitative Research Components

There were three qualitative research components within the project, beginning with the establishment of a clear and comprehensive definition of CCHC, as this was fundamental to all other elements of the project. The other two qualitative research components included: (a) interviews and focus groups with clients (i.e., CCHC and family members) and stakeholders (e.g., professional health, social, and educational stakeholders) to determine the needs of CCHC and families and (b) an environmental scan to identify services and determine gaps and barriers to accessing services for CCHC and their families.

Concept definition of CCHC

An essential first step in this project was to conduct a concept analysis of CCHC and to develop an operational definition for this concept. As noted previously, there are multiple and inconsistent definitions that describe CCHC, causing lack of clarity and misunderstandings for clinicians, researchers, and decision-makers. In addition to informing the other parts of the research project, we anticipate our concept analysis could provide an important contribution to the literature in regards to establishing a comprehensive definition of CCHC. The number of primary studies and the broad range of definitions in the literature for CCHC exemplify the need to establish a comprehensive operational definition that considers elements such as physical, mental, social, spiritual, and educational complex care needs. The concept analysis process and development of an operational definition was guided by the first research question stated in objective one as “How is CCHC defined in the literature?”

A concept analysis involves the process of unfolding, exploring, and understanding a concept for the purposes of concept development, delineation, comparison, clarification, correction, identification, refinement, and validation (Morse, Hupcey, Mitcham, & Lenz, 1996). A concept analysis is also considered to be a critical step to creating a clear definition and would enable understanding of CCHC. While there are different concept analysis methods that can be used in health research, we selected Walker and Avant’s (2011) widely used methodology because the approach includes a comprehensive review of related literature from across disciplines and information sources. This broad approach is essential within the various contexts of CCHC, given the complex and inter-sectoral nature of this concept. Walker and Avant’s eight-step inductive approach includes: (1) selecting a concept; (2) determining the purpose of the analysis; (3) identifying the uses of the concept; (4) determining the defining attributes; (5) identifying the model case; (6) identifying any borderline, related, contrary, or invented cases; (7) identifying any antecedents and consequences; and (8) defining empirical referents.

We completed Steps 1 and 2 of Walker and Avant’s (2011) approach when first developing this background for our protocol. Given the need to arrive at a clear definition and understanding of the population to inform other components of the research project, the concept CCHC was selected in the first step. In the second step we determined the purpose of our analysis, which was to clarify the meaning of CCHC and develop an operational definition of this concept. Consistent with Walker and Avant’s approach, the research team identified various uses of the concept in multiple sources, including dictionaries, thesauruses, stakeholder interviews, and the published, as well as in the gray literature. As part of this process, the published literature was searched systematically through a variety of databases, such as MEDLINE, PsycINFO, PubMed, and CINAHL. An example of a keyword search used in these databases to identify articles is provided in Table 1. We included papers published in English or French that focused on children from 0 to 19 years with complex health conditions.

Table 1.

Sample keyword search for concept definition of CCHC.

(“children with complex health needs” or “children with medical complexity” or “children with complex chronic conditions”) or (“complex health conditions” or “complex health needs” or “complex chronic conditions” or “complex health-care needs” or “special health needs” or “special health-care needs” or “mental health needs” or “physical health needs” or “comorbidity” or “multimorbidity”)

And

(“infant” or “neonate” or “newborn” or “toddler” or “children” or “child” or “teen” or “youth” or “adolescent” or “pediatric”)

And

(“define” or “definition” or “defined as”)

Needs of CCHC and families

We used a cross-sectional qualitative research design to conduct semistructured interviews and focus groups with clients and stakeholders in NB and PEI to address the second research question stated in objective two as “What are the needs of CCHC and their families in NB and PEI?” The goal of this research component was to explore the complex needs of CCHC and their families.

We used a purposeful sampling technique to achieve maximum variation in a representative sample to capture a wide range of informant perspectives (Creswell & Plano Clark, 2011; Patton, 2002). Both electronic and traditional mailing systems were used to distribute invitations to eligible participants. The invitation included a letter that explained the proposed study. Interested participants were contacted by members of the research team who explained the study and emphasized the voluntary nature of the participants’ involvement in the study. Sample size reflected the number of participants required for data saturation and identification of repetitive patterns in participant responses (Patton, 2002; Polit & Beck, 2012).

Consistent with best practices in qualitative research methods and to facilitate participant-driven data, a flexible topic guide was developed using open-ended questions to encourage participants to tell their own stories in their own words (Milne & Oberle, 2005). Accordingly, semistructured interviews that included both focused and open-ended questions, as well as prompts, were used to address the study objectives. Examples of the interview guides for caregivers of CCHC (i.e., family) and professional providers are provided in Appendices A and B, respectively. Research team members recorded field notes documenting insights and/or reflections that arose during and following the interview (Patton, 2002; Polit & Beck, 2012).

All interviews were audio-recorded and transcribed verbatim. The data were analyzed using inductive thematic analysis, which is a research method for identifying, analyzing, and reporting themes within the data (Braun & Clarke, 2006; Patton, 2002). The analysis was guided by Braun and Clarke’s six phases for thematic analysis, which included (1) familiarize self with data, (2) generate initial codes, (3) search for themes, (4) review themes, (5) define and name themes, and (6) provide the report. Members of the research team, under the guidance of PIs, coded the transcripts from the first three interviews to generate preliminary codes and working definitions; this guided subsequent analysis of remaining interview transcripts. Data management was done utilizing NVivo 10™ software.

Environmental scan of services

We conducted an environmental scan to explore the availability of programs and services in NB and PEI with respect to CCHC and their families to address the third research objective, and work toward to the goal of identifying any resource gaps and barriers in accessing care for CCHC and their families.

Typically, an environmental scan requires that information about the external environment and the interconnections of its various sectors is obtained to inform strategic directions and recommendations for future research, practice, and decision-making (Morrison, 1992; Naumann, Reynolds, McColl, & Smith, 2013; Rowel, Moore, Nowrojee, Memiah, & Bronner, 2005). An environmental scan is essential for strategic planning to identify gaps in services and thereby help to identify the need for new and innovative service delivery models. The environmental scan included data from the semistructured interviews and focus groups with clients and stakeholders, using a cross-sectional descriptive design. In addition to these sources, data for the environmental scan included:

NB and PEI government reports and websites that reflected health, hospitals, and education;

NB and PEI community-based organizations websites and reports;

Pediatric networks and associations in NB and PEI.

Data collected from the review of the gray literature (e.g., reports and websites) was analyzed using content analysis techniques (Graneheim & Lundman, 2004; Patton, 2002). The semistructured interview and focus group data in the second qualitative research component of this protocol, Needs of CCHC and Families, was analyzed using inductive thematic analysis (Braun & Clarke, 2006) to identify programs and services, including resource gaps and barriers to accessing care in NB and PEI.

Quantitative Research Component

Building on the qualitative components of this project, the research team applied Simon et al.’s (2014) pediatric medical complexity algorithm (PMCA) to 3 years of data from health administrative databases in PEI. This provided epidemiological information that included prevalence and crude estimates of the number and percentage of children with complex and noncomplex chronic conditions. This quantitative component was guided by the fourth research objective to determine the information gained by applying a comprehensive algorithm about CCHC to administrative data sets.

Test custom algorithm

Given the intricacies of the needs of CCHC that emerged from the concept analysis and qualitative interviews, it was expected that an algorithm design not be restricted to a single condition; rather, the algorithm was intended to enable consideration of multiple issues related to CCHC and their families. By applying the algorithm, we classified children ages 0–19 into three cohorts of medical complexity: (a) children with complex chronic disease, (b) children with noncomplex chronic disease, and (c) children without chronic disease. The cohorts will be described according to various characteristics including demographics, clinical features, and health-care utilization patterns. The characteristics of the cohort of complex, high system users could serve as a future target sample for research teams in NB and PEI to conduct cross-jurisdictional research projects and develop corresponding ISD models.

The pediatric algorithm applied here was based on earlier work by Simon et al. (2014) and was intended to identify the cohort of CCHC within a target population based on physician billing claims data—an administrative database which is accessible in restricted form from the Provincial health authorities. Although the original algorithm was developed for an evaluation of a U.S.-based Medicare data set, it was applicable in the Maritime Provinces because cases can be identified, for example, by applying the algorithm to the physician billing claims database which has medical conditions recorded using International Classification of Diseases Ninth Revision (ICD-9) code formats. These data are available under specific controls and legislative agreements in both NB and PEI and are included in current data sharing agreements in the respective Provincial strategy for patient-oriented research (SPOR) support units.¹

Consistent with the approach by Simon et al. (2014), the development of the algorithm component of this study used a three-stage approach that built on the principles of the SPOR program. That is, the research began with creation of a working group with representation from both NB and PEI that consisted of researchers, clinicians, and decision-makers. The working group collaborated with the research team to evaluate and integrate the information developed in the first three qualitative components of this project.

In the second stage of the quantitative application, the working group created a prioritization schema that informed the basis of the algorithm. Inclusion and exclusion criteria were developed and used to determine the epidemiology of CCHC who use the hospital systems or who have contributed information to the selected administrative databases in PEI. In PEI, the research team worked collaboratively with health PEI to access the data according to established policies and procedures. Although the algorithm component of the study has only been applied to PEI to date, it is anticipated that this component will be similarly applied to NB data sets in future.

In a second step of the quantitative evaluation of CCHC, we conducted a chart audit of pediatricians’ medical records of children ages 0–19 who had been referred to a pediatrician over a 3-year period. In PEI, the chart audits took place in one hospital pediatric clinic and one pediatrician office in two different cities in the province. Similarly, in NB, the chart audits were conducted in two pediatrician’s offices in two cities. The purpose of the chart audit was (a) to identify disease conditions and comorbidities and compare/validate with the administrative database results and (b) to document referral patterns for in and out-of-province referrals for CCHCs.

Additional data were obtained on health-care utilization of the chart audit cases. These variables included (a) total physician visits, (b) total emergency room visits, and (c) total hospital visits/discharges. More than 700 patients were included in the chart audit. The population proportion estimate for the final set of records were randomly selected from the database, with a minimum selection of 400 charts to ensure a 95% confidence interval and a 10% error of estimation that the sample of charts selected described the true population of children when there is at least a 10% prevalence of the complex health conditions.

Finally, by applying the algorithm developed from the administrative database analysis to the set of representative chart audits, the research team compared the generalizability of the algorithm outcomes to hospital records to establish the applicability and appropriateness (e.g., predictive capacity—identification of false negatives and false positives) of the computerized algorithm. The outcome of this final application was specifically intended to enable the researchers to provide epidemiological assessments of CCHC and econometric forecasting of future programs and services to meet identified needs.

Methodological Rigor

To promote rigor in the execution of this project, we respected the application of principles deemed appropriate to each qualitative and quantitative methodology. For the three qualitative components, we adopted the principles of trustworthiness first outlined by Lincoln and Guba (1985): credibility, dependability, confirmability, and transferability. For the quantitative component, we applied the widely accepted principles of validity, reliability and generalizability (Polit & Beck, 2012).

Methodological Rigor for Qualitative Components

Credibility is regarded as the truth value and believability of the study’s findings, (Lincoln & Guba, 1985; Schwandt, 2007). Two specific aspects of credibility in qualitative research are the degree to which rigor is applied in execution of methods and confidence in the research (Patton, 2002; Polit & Beck, 2012). With respect to execution of methods, adherence to qualitative methodological principles was maintained for each of the three qualitative components. For instance, the steps outlined by Walker and Avant (2011) were followed for the concept analysis, as was the process described by Braun and Clarke (2006) for coding the needs of CCHC and their families.

Findings were shared with stakeholders in meetings, and feedback was gathered to refine the final results. As well, where appropriate for analysis, in vivo codes were used for category names to bring participant’s voice to the theoretical development and articulation of research findings. The composition of the research team, particularly the backgrounds and experience of the members, brought together individuals with “diverse methodological and context expertise” necessary for a successful collaboration and execution of this research enterprise (Creswell & Plano Clark, 2011, p. 15).

Dependability reflects how reliable the research findings are and also demonstrates whether same or similar findings would be achieved if the study were repeated (Lincoln & Guba, 1985; Polit & Beck, 2012). One strategy used to enhance dependability was having RAs, trainees, and PIs work together in developing the approach to coding interview transcripts; this helped the team to be consistent in coding data across the diverse participant groups and various settings (Graneheim & Lundman, 2004). In addition, we maintained an audit trail to facilitate evaluation of the consistency of the research process, demonstrate how conceptualizations were formulated, and provide guidance in replication of the study (Polit & Beck, 2012; Sandelowski, 1986). The audit trail for this project consists of all documentation related to the qualitative components (e.g., transcripts, coding records, and field notes), notes from research team meetings and e-mails (Carnevale, 2002; Polit & Beck, 2012). As well, data saturation was achieved with thoroughness of data collection and analysis (Carnevale, 2002; Polit & Beck, 2012).

Confirmability “refers to objectivity or the potential congruence between two or more independent people about the data’s accuracy, relevance or meaning” (Polit & Beck, 2012, p. 539), and data and interpretations must represent the information provided by participants. Confirmability was enhanced in data coding through consensus on content and thematic codes (e.g., objective two focused on the Needs of CCHC and Families) and utilization of participant narratives to enhance reporting of the study findings. The audit trail for our project also supports confirmability in allowing the reader to draw conclusions about the veracity of our interpretations of data from the participants (Carnevale, 2002).

Transferability reflects the extent to which findings can be used in other contexts (Carnevale, 2002; Polit & Beck, 2012). Transferability will be enhanced by recorded demographic information of the participants, descriptions of the context for each study component and our audit trail of data sources that contextualize the findings when reporting on each qualitative component of the project.

Rigor for Quantitative Component

During the development of the quantitative component of the study, we addressed several issues using standardized practices to ensure that the methods and outcomes were valid and reliable. For example, the algorithm used to assess the prevalence of CCHC in NB and PEI was a standardized peer-reviewed and published tool that was initially developed using a similar process to that which was used in the present study. The PMCA was developed by Simon et al. (2014) based on a SAS® based program. The algorithm was applied to ICD-9 codes of children aged 0–18 years old that were insured by Washington State Medicaid (WA-Medicaid) visiting the Seattle Children’s Hospital. In the present study, the algorithm was restricted to administrative data and was applied to a sufficient sample size of records drawn from the physician billing claims data set to enable comparison to the original application. In its original form, the PMCA was considered to demonstrate good sensitivity and good to excellent specificity when applied to hospital discharge or medical claims data, and was thereby deemed applicable to large data sets representing hospital and health plan utilization by children.

The next step in the application of the algorithm was to evaluate the efficacy of the filtering tool in identifying the prevalence of children with complex and chronic conditions based on the SAS coding for administrative data to data that were collected by direct chart audits. The chart auditing process began with the development of a standard chart audit abstraction procedure that was vetted amongst nurse administrators and data clerks to ensure that the process for data collection by a member of the research team followed a consistent and appropriate process. Ensuring that the chart auditor met regularly with the hospital-based data clerk and/or the nurse administrator responsible for patient charts was intended to mitigate problems associated with recording and interpretation, and ensure accuracy, reliability, and consistency of coding. The sample size for the chart audits was large (n = 718), and the charts were randomly selected, which helped to minimize sampling bias and allow for generalizability of the results to the population from which the sample was drawn (Vassar & Holzmann, 2013).

Ethical Considerations

Prior to beginning participant recruitment and data collection, ethics approval was obtained by each PI from their respective institutions and jointly through each of the affiliated health authorities. Ethics approval for this project was received from (a) University of New Brunswick—Saint John (#023-2015), (b) Mount Alison University (#2016-044), (c) University of Prince Edward Island (#6006412), (d) Horizon Health Network in NB (#20162329), and (e) the PEI Research Ethics Board (no file #).

To facilitate informed consent for inclusion to the study, information on the research project was provided to the interview and focus group participants upon recruitment, and participants read and signed a consent form prior to data collection. Given the small numbers of CCHC and their families in NB and PEI, and the highly visible nature of some of the health conditions, special care has been taken to ensure privacy and confidentiality of all participants. This included (a) using anonymized study codes to mask identities of participants and patient charts, (b) removing all identifiers from interview transcripts, (c) keeping identifying participant information (e.g., consent forms and study code register) separate from the collected data (e.g., transcripts and field notes), and (d) restricting access to participant information and data to members of the research team.

Security of participant information was maintained through erasing the audio recordings once interviews were transcribed and securing all participant information separate from data in locked offices designated to each PI on her or his respective campus. Interview transcripts will be kept for 5 years after the completion of this project, and will then be destroyed. When reporting findings of the data, results will be presented in an aggregate manner where possible; for qualitative findings, quotes will be attributed to generic sources (i.e., “client” to represent CCHC and family participants; “stakeholder” to represent health care, education, and social work professionals).

Significance and Implications of This Research

It was anticipated that our firsthand exploration of the needs of CCHC and their families in NB and PEI, and examining current services and health system gaps to meet these complex needs, would be essential to inform practice, research, and policy development. Adoption of iKT and end-of-grant KT strategies in our project led to a significant engagement of participants (i.e., clients and stakeholders), knowledge users, and decision-makers in the research process; it also facilitated dissemination of early findings and envisioning directions for work with CCHC and their families.

Stakeholder Engagement in the Research Process

From the beginning of the proposal development for this research project, the researchers engaged families of CCHC. For example, a town hall meeting with stakeholders, including patient advisors, was facilitated by one PI (WM) to describe the opportunities related for the CIHR SPOR initiative, which included such specialized projects as the present study. This town hall was met with great enthusiasm for the proposed project, and became an early vehicle for both engagement later in the research process and for recruitment. CCHC or their parents of CCHC in both NB and PEI were involved in the development of this protocol and informing aspects of the research design throughout.

At the outset of our research project, we anticipated our proposed concept analysis would fill a gap in the literature concerning the substantive question on the definition of CCHC. In addition to informing the quantitative component of our project, defining CCHC provided a common understanding of this concept for the research team and participants when progressing through the research and in early dissemination of research results. In fact, the concept analysis was informed by diverse stakeholders, including family members of CCHC, as the research team worked through the concept analysis process.

Finally, one of the more powerful means of stakeholder engagement and iKT will occur through the pan-Canadian SPOR Network in Primary and Integrated Health Care Innovation Network (PIHcI Network). This pan-Canadian Network will provide opportunities to share ideas, early outcomes, and lessons learned through this Quick Strike project. The pan-Canadian network and its activities present a tremendous opportunity for small provinces like NB and PEI as we embark on our project; fostering the creation of responsive learning health networks within and across our jurisdictions.

Dissemination of Early Findings

At the beginning of this project, a research dissemination plan as part of the iKT and KT strategies was developed with goals, target audiences, strategies, expertise/leadership, resources, and deadlines to maximize project outputs. Dissemination of early findings has included both traditional and innovative KT activities. For instance, traditional dissemination of findings has included presentations at conferences such as the CIHR SPOR Summit (October, 2016) and the SPOR Primary and Integrated Health Care Innovation Network Research Day (May, 2017). An example of an innovative and more integrated form of KT included an event entitled “Mount A Breakfast Chat: Growing up with a Complex Health Condition”. This breakfast chat event at Mount Alison University was funded by the Atlantic Chapter of the Royal Society of Canada and brought together a diverse group of CCHC, family members, researchers, care providers, and other knowledge users (1) to discuss daily stress issues faced by CCHC and their families and (2) to learn about coping strategies and professional programs that can offer support to families of CCHC. Our research team had the opportunity to share and discuss our research findings with this diverse group. In addition to these activities, the research team continues to be actively engaged in dissemination of findings to targeted practitioners, academics, decision-makers, and the general public (e.g., several manuscripts being published and in progress in peer-reviewed, open access journals, presentations at national and international conferences, and site visits to children’s hospital-based programs for CCHC, etc.).

Secondary Outcomes from the Research Process

During implementation of our research project, we identified a number of additional opportunities for knowledge development as secondary outcomes to the objectives of this protocol and that are currently in progress. First, the interviews of professional stakeholders revealed facilitators and challenges for collaborative efforts to support CCHC and their families due to health system designs and individual competencies for interprofessional collaboration. To explore these issues more fully, a secondary analysis of the interview data is being conducted to explore the barriers and facilitators to interprofessional collaboration and communication in the care of context of CCHC in rural and low-population geographic areas. Second, when addressing the third objective of this project around the environmental scan, members of the research team discovered that there is no common approach or consensus on a methodology for conducting an environmental scan. In response, we are undertaking a scoping review to explore the methodology and procedures used for conducting an environmental scan. A scoping review is useful in knowledge synthesis on activities, such as environmental scans, because they help to coordinate and assess the related literature in identifying the extent and types of research available on a particular topic (Colquhoun et al., 2014; Khalil et al., 2016; Peters et al., 2015).

Future Directions for Research and Planning

With the conclusion of data collection and current work on outcomes from this Quick Strike research project, our research team has become well positioned to propose innovative service delivery models and programs to address the complex needs of CCHC and their families. In addition, stakeholders from the ministry in NB have also expressed an interest in research exploring innovative interprofessional models of care to inform PHC reform in their respective Provinces (e.g., Government of New Brunswick, 2012), which is research that will be carried out as a result of this project. The outcomes of the project will also allow us to conduct cross-jurisdictional research on models of care for children with complex conditions that may prevent avoidable hospital admissions and unneeded travel out-of-province. Data are urgently needed on upstream predictors of complex health-care needs for CCHC to develop and implement preventive interventions for this population. An upstream preventative approach could have a significant impact in reducing complications that can result from complex health conditions, which could contribute to a better patient and family experience and health outcomes. An upstream approach also has the potential to reduce avoidable acute care utilization and high health-care costs. Furthermore, the algorithm that results from this work will be further developed to fit cohorts of CCHC across the lifespan, as they transition to adulthood and progress through the aging process.

With the development and testing of a computerized algorithm to evaluate CCHC, as part of our fourth objective, an important deliverable of our research project is the hosting of a future workshop on the research findings, including how to access data to conduct research that focuses on CCHC. The workshop will be guided by the findings from the concept analysis, stakeholder interviews, and environmental scan. This workshop will be offered in both NB and PEI at the completion of our project.

Finally, as mentioned previously, through the pan-Canadian SPOR PIHCI Network, we will have opportunities to disseminate the findings and replicate/scale up this project across jurisdictions, reaching an audience much broader than the academic and local community. This will ensure that information about successful and unsuccessful features of our project is shared. This will also allow for the examination of comparative data on CCHC across provinces and territories, which has not occurred to date. It is widely known that cross-jurisdictional research and dissemination is needed to transform Canada’s many successful pilot projects into sustained initiatives. This project’s data about family needs and existing services (i.e., objectives 2 and 3) jointly highlighted the complexity of (a) CCHC and family needs, (b) the geography in NB and PEI (i.e., rural and semirural contexts), and (c) existing services, which are fragmented and difficult to navigate, especially in the context of the geography and absence of children’s hospitals in NB and PEI. These emerging findings informed the needs assessment phase of the newly launched bilingual NaviCare/SoinsNavi for CCHC in NB on January 10, 2017 (http://navicare-soinsnavi.ca/). This center has the potential to significantly help families of CCHC navigate not only the health but also the education and social systems. An evaluation of NaviCare/SoinsNavi is currently in progress; once complete, outcomes of the evaluation will be shared and, if demonstrated to be successful, such a center can be replicated in PEI and other jurisdictions across Canada.

Conclusion

At the outset of this project, it was anticipated our research outcomes would help improve the effectiveness, efficiency, scalability, and short-term policy and program impact of primary and integrated health-care innovations for CCHC and their families. The results from this project continue to build a strong foundation for ongoing research and policy development to support the complex needs of CCHC and their families. Furthermore, our project has brought together a diverse collaborative of stakeholders, knowledge users, and decision-makers to build capacity for sustainable initiatives within NB and PEI. Finally, the results and template of our project are transferable and scalable to other provinces/territories across Canada and beyond in the goal of promoting integrated and innovative service delivery models to improve the lives of CCHC and their families.

Footnotes

Appendix A

Appendix B

Acknowledgments

The principal investigators gratefully thank the Canadian of Institutes of Health Research (CIHR) and the New Brunswick Children’s Foundation for their financial support of this project. We also thank all participants for their contributions to our research study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article. Canadian Institutes of Health Research Quick Strike and New Brunswick Children’s Foundation.

Note

References

Arim

R. G.

Kohen

D. E.

Brehaut

J. C.

Guevremont

Garner

R. E.

Miller

… Rosenbaum

P. L

. (2015). Developing a non-categorical measure of child health using administrative data. Health Reports, 26, 9–16.

Berry

. (2015). What children with medical complexity, their families, and healthcare providers deserve from an ideal healthcare system [Briefing paper]. Lucille Packard Foundation for Children’s Health. Retrieved from http://www.lpfch.org/publication/what-children-medical-complexity-their-families-and-healthcare-providers-deserve-ideal

Berry

J. G.

Agrawal

Kuo

D. Z.

Cohen

Risko

Hall

… Srivastava

. (2011). Characteristics of hospitalizations for patients who utilize a structured clinical-care program for children with medical complexity. Journal of Pediatrics, 159, 284–290.

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101.

Burns

K. H.

Casey

P. H.

Lyle

R. E.

Mac Bird

Fussel

J. J.

Robbins

J. M.

(2010). Increasing prevalence of medically complex children in US hospitals. Pediatrics, 126, 638–646.

Canadian Institutes of Health Research. (2012). Guide to knowledge translation planning at CIHR: Integrated and end-of-grant approaches. Retrieved from http://www.cihr-irsc.gc.ca/e/45321.html

Carnevale

F. A.

(2002). Authentic qualitative research and the quest for methodological rigor. Canadian Journal of Nursing Research, 34, 121–128.

Cohen

Kuo

D. Z.

Agrawal

Berry

J. G.

Bhagat

S. K.

Simon

T. D.

Srivastava

(2011). Children with medical complexity: An emerging population for clinical and research initiatives. Pediatrics, 127, 529–538.

Cohen

Lacombe-Duncan

Spalding

MacInnis

Nicholas

Narayanan

U. G.

… Friedman

J. N

. (2012). Integrated complex care coordination for children with medical complexity: A mixed-methods evaluation of tertiary care-community collaboration. BMC Health Services Research, 12, 1–11.

10.

Colquhoun

Levac

O’Brien

Straus

Tricco

Perrier

Kastner

Moher

(2014). Scoping reviews: Time for clarity in definition, methods and reporting. Journal of Clinical Epidemiology, 67, 1291–1294.

11.

Colver

A. F.

Merrick

Deverill

Le Couteur

Parr

Pearce

M. S.

… McConachie

. (2013). Study protocol: Longitudinal study of the transition of young people with complex health needs from child to adult health services. BMC Central Public, 13, 675–686.

12.

Creswell

J. W.

Plano Clark

V. L.

(2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: SAGE Publications Inc.

13.

Fawcett

Garity

(2009). Evaluating research for evidence-based nursing practice. Philadelphia, PA; F. A. Davis Company.

14.

Gearing

R. E.

Mian

I. A.

Barber

Ickowicz

(2006). A methodology for conducting retrospective chart review research in child and adolescent psychiatry. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 15, 126–134.

15.

Government of New Brunswick. (2010). Integrated service delivery demonstration sites underway [News release]. Retrieved from http://www2.gnb.ca/content/gnb/en/news/news_release.2010.06.0990.html

16.

Government of New Brunswick. (2012). A primary health care framework for New Brunswick. Retrieved from http://www2.gnb.ca/content/dam/gnb/Departments/h-s/pdf/en/PHC/A-Primary-Health-Care-Framework.pdf

17.

Graneheim

U. H.

Lundman

(2004). Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Education Today, 24, 105–112.

18.

Health

PEI

. (2015). Government investing $3.7 million over five years for children with complex needs [News release]. Retrieved from http://www.healthpei.ca/index.php3?number=news&dept=&newsnumber=9839&lang=E%3EGovernment%20of%20P.E.I.%20Web%20site%C3%A2%E2%82%AC%E2%80%9DParent%20Education%20Program%3C/a%3E%20%3Cbr%3E%20Parent%20Education%20Program%20information%20brochure.

19.

Kaufman

Pinzon

, & Canadian Paediatric Society Adolescent Health Committee. (2007). Transitions to adult care for youth with special health care need [Position statement]. Paediatric Child Health, 12, 785–788. Retrieved from http://www.cps.ca/documents/position/transition-youth-special-needs

20.

Khalil

Peters

Godfrey

C. M.

McInerney

Soares

C. B.

Parker

(2016). An evidence-based approach to scoping reviews. Worldviews on Evidence-based Nursing, 13, 118–123.

21.

Kingsnorth

Lacombe-Duncan

Kielty

Bruce-Barrett

Cohen

(2015). Inter-organizational partnership for children with medical complexity: The integrated complex care model. Child Care Health and Development, 41, 57–66.

22.

Kuo

D. Z.

Cohen

Agrawal

Berry

J. G.

Casey

P. H.

(2011). A national profile of caregiver challenges among more medically complex children with special health care needs. Archives of Pediatric Adolescent Medicine, 165, 1020–1026.

23.

Lincoln

Y. S.

Guba

E. G.

(1985). Naturalistic inquiry. Newbury Park, CA: Sage.

24.

McPherson

Arango

Fox

Lauver

McManus

Newacheck

… Strickland

. (1998). A new definition of children with special health care needs. Pediatrics, 102, 137–140.

25.

Milne

Oberle

(2005). Enhancing rigor in qualitative description. Journal of Wound, Ostomy and Continence Nursing, 32, 413–420.

26.

Morrison

J. L.

(1992). Environmental scanning. In Whitley

M. A.

Porter

J. D.

Fenske

R. H.

(Eds.), A primer for institutional researchers (pp. 86–99). Tallahassee, FL: The Association for Institutional Research.

27.

Morse

J. M.

Hupcey

J. E.

Mitcham

Lenz

E. R.

(1996). Concept analysis in nursing research: A critical appraisal. Scholarly Inquiry for Nursing Practice: An International Journal, 10, 253–277.

28.

Naumann

Reynolds

McColl

Smith

(2013). Environmental scan of programs for fetal alcohol spectrum disorder in Eastern Ontario. Journal of Developmental Disabilities, 19, 29–49.

29.

Patton

M. Q.

(2002). Qualitative evaluation and research methods (3rd ed.). Thousands Oaks, CA: SAGE.

30.

Peters

Godfrey

Khalil

McInerney

Parker

Baldini Soares

(2015). Guidance for conducting systematic scoping reviews. International Journal of Evidence-based Healthcare, 13, 141–146.

31.

Polit

D. F.

Beck

C. T.

(2012). Nursing research: Generating and assessing evidence for nursing practice (9th ed.). Philadelphia, PA: Wolters Kluwer—Lippincott, Williams & Wilkins.

32.

Rowel

Moore

Nowrojee

Memiah

Bronner

(2005). The utility of the environmental scan for public health practice: Lessons learned from an urban program to increase cancer screening. Journal of the National Medical Association, 97, 527–534.

33.

Sandelowski

(1986). The problem of rigor in qualitative research. Advances in Nursing Science, 8, 27–37.

34.

Schwandt

T. A.

(2007). The SAGE dictionary of qualitative inquiry (3rd ed.). Thousand Oaks, CA: SAGE Publications.

35.

Shi

(2012). The impact of primary care: A focused review. Scientifica, 2012, 1–22. Retrieved from https://www.hindawi.com/journals/scientifica/2012/432892/

36.

Simon

T. D.

Cawthon

M. L.

Stanford

Popalisky

Lyons

Woodcox

… Mangione-Smith

. (2014). Pediatric medical complexity algorithm: A new method to stratify children by medical complexity. Pediatrics, 133, e1647–e1654.

37.

Vassar

Holzmann

(2013). The retrospective chart review: Important methodological considerations. Journal of Educational Evaluation for Health Professions, 10, 1–7.

38.

Walker

L. O.

Avant

K. C.

(2011). Strategies for theory construction in nursing (5th ed.). Upper Saddle River, NJ: Pearson Prentice Hall.