Abstract
Introduction
The 2019
In reflecting on this, we became concerned that COVID-19 will pose many difficulties for researchers who are investigating psychiatric and related variables in youth. This editorial is an effort to conceptualize the ongoing impact of COVID-19 on child and adolescent psychiatric research in the Western world. The goals of this editorial are (1) to identify factors capable of confounding current research; (2) to recognize sources of “noise” (experimental error) in COVID-19-era studies; (3) to classify variables that newly affect children, adolescents, and their families' health before, during, and after COVID-19; and (4) to propose quantification of those variables. Our overall objective is to create a structure for understanding the influences that COVID-19 has had on child mental health research and to assist researchers as they attempt to deal with major life events that have accompanied COVID-19.
COVID-19 Factors Influencing “Mechanics” of Child and Adolescent Psychiatric Research
COVID-19 almost certainly has increased variability (“noise”) in current child and adolescent psychiatric research. This will be seen in many of the observations to follow and it will be an issue that we return to when addressing overall implications.
Characteristically, child and adolescent psychiatric research activities take place within office settings. However, as a consequence of the epidemic, such face-to-face encounters have been greatly diminished because of stay-at-home orders for both families and research staff.
Standardized rating scales, completed by parents and teachers, comprise an important part of psychiatric assessment. Historically, parent ratings have often been done interactively with research team members during office visits. Unfortunately, COVID-19 has disrupted this form of feedback in many clinical and naturalistic studies, with most parent rating forms being completed at home. Some investigators might have concerns about intrinsic differences between face-to-face ratings done in research settings and those done at home. However, the existing research indicates that results from both forms of completion (office and home) are more or less equivalent (Gwaltney et al. 2008; Bjorner et al. 2014; Muehlhausen et al. 2015). Nevertheless, this apparent equivalency between in-person ratings in clinics and distant ratings through paper or other format is not grounds for complacency.
In our experience, research staff, even when on-site, often inadvertently leave out vital instructions. The problem of insufficient guidance of raters probably becomes more consequential as ratings are done “farther” from the clinical site. Thus, researchers must be absolutely consistent when instructing raters. This translates to giving thorough and complete directions each and every time they instruct informants. Explicit guidance should be given before each rating regarding (1) behavioral frequency, (2) severity, (3) temporal period covered, and (4) who comprises the normative group.
Another COVID-19-related issue concerns which informants conduct study behavior ratings. Ideally, these are the same designated persons over the entire study, not shared across roles, such as mother, father, and/or grandparent. To state the obvious, as the rating enterprise has moved from clinic setting to home settings, investigators lose much of their ability to dictate with confidence who completes ratings. A special situation also introduced by COVID-19 concerns parents working in health care who have been pressed into working extraordinary hours and conditions because of coronavirus. Such raters likely would be forced to give up their roles as raters because of diminished access to their children and because of overwork. A second form of rater consistency concerns when the informant completes ratings (ideally at the same time of the week and day, and in reference to a consistent set of subject activities). A third form concerns the context and location of in which informant ratings are completed. An example of a nonideal context/location might be informants completing their ratings in what might often be a highly distracting environment. All of these will seem obvious factors to this journal's readership. Nevertheless, if caregivers are left to their own devices, many will inevitably violate the usual assumptions regarding standardized behavior ratings.
Pharmacological research in children and adolescents faces its own problems secondary to reduced physical study visitations. Examples include difficulties documenting medication adherence, problems getting medication to participating families, and monitoring physical and behavioral effects of study drugs. Accommodations might include shipping medications to participants through priority mail, having families post used medication containers to the clinic for pill counts, and doing “med checks” through telehealth interviews.
Another issue for many research clinics has been the move to technologies such as Zoom and telemedicine platforms that may be unfamiliar both to some researchers and to families. Some study families may have limited ability to access technology, and research staff may find themselves doing “Zoom meetings” with some families and phone calls with others. This, of course, is another form of procedural variability. Under these circumstances, data may be compromised by factors such as limited ability by research team members to “read” nonverbal cues in the children and parents, parents inadvertently “helping” their children to complete study measures, and extraneous distractions in the environment (e.g., a screaming sibling in the next room).
Furthermore, the new virtual environment may not be as rewarding on an interpersonal level for the children and their families, given that it lacks the “personal touch” that families feel when interacting in person with the research team during in-clinic visits. Conversely, the telehealth environment newly associated with COVID-19 may be perceived as being beneficial to some families. Some parents may be relieved when they do not need to travel to research centers. Children, especially those with autism spectrum disorder (ASD), who associate clinic visits with anxiety-provoking procedures (e.g., blood draws, injections, and ECG), may feel calmer completing study procedures at home. Some families may be reluctant to revert to office visits after the COVID crisis recedes—and thus pre-COVID study procedures may be difficult to reinstate. Indeed, some families may simply chose to discontinue participation if the study does not allow them to continue with telehealth visits. These factors may impact multisite studies to an even greater extent than single-site studies, given that different sites may be differentially impacted—and these losses may be additive across multiple study sites.
The Impact of COVID-19 on Family Systems
One thesis of this article is that COVID-19 has caused significant disruptions to the lives of children, adolescents, and families living in the Western world. All researchers experience unanticipated disruptions in the context of psychiatric research with youth. Some of these (e.g., parent divorce and death of a sibling) can be significant and—if such events are frequent enough—they can potentially influence interpretation of a study. Nevertheless, in most studies of the past, these disruptions were relatively uncommon, and it was reasonable to assume that the impacts of such events would “even out” over the course of the study and affect all experimental conditions equally. COVID-19 is different because its impacts on child and adolescent life have been marked and because it has influenced nearly all youth involved in psychiatric research at approximately the same time. Furthermore, many of the youth who are enrolled in psychiatric research are initially embedded in highly enriched clinical environments. Hence, major environmental influences, such as ushered in by COVID-19, have far greater disruptive potential for many children in psychiatric research.
Although not a “family event” per se, some survey research from China suggests that COVID-19 has had a negative psychological impact on many youth. One survey suggests that COVID-19 has been associated with increased number of children who experienced significant anxiety and depression (Xie et al. 2020). A total of 1784 children attending regular schools in grades 2–6 were surveyed with standardized anxiety and depression scales. Some 23% reported depressive symptoms and 19% reported anxiety symptoms, which are higher than figures reported before the emergence of COVID-19. Another survey of 320 Chinese children, ages 3–18 years, reported high levels of behavioral and emotional problems: >15%, fatigue and poor appetite; >20%, sleeping disorders and fear for health of relatives; and 25%–30%, obsessive requests for updates, irritability, and excessive inattention (Jiao et al. 2020). We were unable to locate similar data from North America and Europe, but it is likely that the COVID-19 pandemic is having similar effects in North America, Europe, and Australasia.
Let us consider potential ways that COVID-19 has affected youth with significant psychiatric issues in Western society.
COVID-19 impacts on children and adolescents
For illustrative purposes, let us consider children with ASD in the context of COVID-19. In the course of their educational services, it would not be unusual for such children to receive occupational therapy, speech therapy, applied behavior analysis, and social skills training—all delivered through their school program. In the early stages of COVID-19, most schools suspended teaching and greatly reduced many other therapeutic services being offered through the schools. Instead, teacher surrogates, usually parents, were asked to assume supervision. So, one immediate consequence of COVID-19 was loss of contact with highly trained and experienced special education teachers and the structure that they offered.
Other services, such as speech therapy, occupational therapy, and social skills training, were greatly pared back or suspended for many children. Another consequence was the loss of social contact and social opportunities afforded by classmates. This could have profound consequences for children/adolescents with a variety of clinical conditions such as ASD, conduct disorder, posttraumatic stress disorder, and other anxiety disorders. Many youth were monitored for medication adherence (including study medication adherence) by school nurses. With school classes suspended, such monitoring is impossible.
Suspension of school also eliminated travel to/from school and the opportunity to interact with other commuting students on a regular basis. Shelter-in-place orders and social distancing have been the COVID-19 norms in most states, and the potential influence of these for youth with emotional disorders should be obvious. Many interventions (behavior therapy, social skills training, remedial tutoring, applied behavior analysis, cognitive therapy, dance and movement therapy, music therapy, interpersonal psychotherapy, and neurofeedback) were greatly reduced or discontinued. Many children and adolescents, including those in psychiatric research, received free school meal (breakfast and lunch) programs, and COVID-19 usually disrupted these as well. Conversely, during this time, families may have initiated online “self-help” programs at home that could confound study procedures.
Clearly, the disruption to school programs imposed by COVID-19 was profound. As noted elsewhere in this article, COVID-19 has led to suspension of regular clinic assessments of most study participants. Some youth, such as those with ASD, find it very anxiety provoking to attend traditional clinics, and this procedural alteration alone could influence behavioral outcomes in psychiatric studies.
COVID-19 impacts on family life
In addition to the impact of school loss, COVID-19 has often had a profound influence on family life. Much of this has been broadcast into our homes on a regular basis since the onslaught of COVID-19. One effect is that parents and other significant family members (e.g., grandparents) have often been asked to assume the role of teacher. Some parents are ill equipped to assume this burden, and many parents may not have the time to supervise and participate in their children's teaching. COVID-19 has severely stressed many families financially. Numbers of adults, not seen since the Great Depression, are unemployed. Many parents who have not lost jobs feel threatened by the potential loss of jobs. Others, namely those with health care occupations, are struggling with the exact opposite: battling to fulfill their work obligations and the worry of staying virus free for the well-being of their families.
Newscasts have reported that large numbers of families are unable to manage rent and mortgage payments. With the rapidly changing economy, large numbers of parents have had to change jobs. In addition, COVID-19 has even influenced family composition: for safety reasons, grandparents may make the decision to separate from the remainder of the family or, conversely, grandparents may have been recruited because they were needed for their caregiving roles. Shelter-in-place orders have been in effect in most states. The impact of these on people's mental health is unknown at this time, but one hears complaints that adults are “stir-crazy” from the relative isolation. In the midst of COVID-19, many adults complain of a common feeling of malaise. Finally, although ill documented thus far, there are concerns that domestic violence (including child abuse) has increased during COVID-19 (Abramson 2020). Workers speculate that reduced contact with extended family (such as grandparents), coaching through schools, reduced contact with religious groups, and stress on child protective organizations have increased child abuse.
Added to this list are occasional deaths of extended family members and close family friends. These lost relatives may have provided substantial support to the family (e.g., child care, emotional support, and financial support) that could be destabilizing to the family left behind.
Naturally, children and adolescents are not insulated from the stresses experienced by their parents and other responsible family decision makers. Thus, COVID-19 likely has impaired quality of family life for many children and adolescents with psychiatric issues. Conversely, some youth may even find certain changes to their liking, such as school closures and elimination/reduction of in-person assessments. Nevertheless, all of these disruptions, whether welcomed or perceived as aversive, are the sources of increased variability. Therefore, we strongly recommend that they be documented systematically.
Conceptualizing Stages of Psychiatric Research in Waves, Stages, or Cohorts
If readers accept our assertion that COVID-19 has had a big impact on the behavior and development of children and adolescents with psychiatric problems, then it becomes necessary to document phases of research projects conducted before, during, and after COVID-19. The assumption here is that there are many studies underway whose participants had mixed experiences both in terms of their exposures to coronavirus and in terms of when they passed through their respective studies in real time.
Figure 1 depicts the temporal impacts of COVID-19 on child and adolescent psychiatric studies. Some participants, characterized as Cohort A, completed their full protocol before arrival of COVID-19. It is relatively easy to understand group changes within this cohort, as only planned study protocols were in effect during their participation. Other participants, comprising Cohort B, were enrolled before COVID-19 emerged, but part way into the protocol, their experiences were marred substantially by the arrival of coronavirus. In looking at the figure, it is important to realize that the left arrow (“some procedures before COVID”) reflects highly variable periods, ranging from a few days to most of a youth's study participation. Likewise, the right part of the Cohort B arrow (“some procedures during COVID”) indicates that variable numbers of this group were exposed to some study procedures after the arrival of COVID-19. Great complexity lies in the fact that highly variable proportions of subjects were enrolled pre-COVID-19 and, likewise, highly variable proportions were enrolled para-COVID-19.
Real-time cohorts based on when study participants are enrolled relative to COVID-19. Diagram characterizing subject cohorts,
Cohort C comprises participants whose complete study experience occurred during the course of COVID-19. It would be a mistake to assume that their experience was entirely homogeneous, but historically these individuals were exposed to the same sociocultural environment throughout their participation. Cohort D encompasses individuals whose initial participation occurred during COVID-19 but who finished study participation after COVID-19 was eradicated. Presumably, this would occur after an effective vaccine is developed for the coronavirus. Similar to Cohort C, this cohort (D) would contain highly variable proportions of individuals receiving para-COVID-19 and post-COVID-19 exposures. Finally, the last Cohort E comprises only individuals who are enrolled after eradication of COVID-19. Hopefully, investigators will be enrolling such individuals in the near future. There is one possible cohort that does not appear in Figure 1, and this might occur if there is a marked decline (effectively zero new cases, as in Cohort E) followed by a pronounced resurgence in coronavirus cases. Although one does not want to contemplate such a scenario, we have to admit to the possibility of COVID-19 rearing its ugly head a second time.
The Need to Probe and Document
In this section, we argue that all investigators should probe for and document major disruptive events that occur during the COVID-19 experience. These events should be as exhaustive as possible and should reflect, as thoroughly as possible, events that clearly impact child and adolescent participants and their families. We provide a sample table listing onsets and suspensions of interventions and variables that may influence participating youth.
Most established laboratories will have good methods for capturing significant child, adolescent, and family events that may impact research findings. We hope that research teams begin to develop a shared culture during and after COVID-19 to determine important environmental events that should be consistently reported. We do not believe that Table 1 captures everything that deserves documentation, and we look forward to an emerging consensus within the field as we come to grips with this task.
Documentation of Disruptions Affecting Youth Participants and Families Before, During, and After COVID-19
Specify behavior therapy, social skills, applied behavior analysis, occupational therapy, physical therapy, family therapy, cognitive therapy, music therapy, dance, movement therapy, neurofeedback, aroma therapy, etc.
Implications for Researchers, Reviewers, and Editors
It seems that there are some clear implications of the issues outlined here for researchers, reviewers, and journal editors. The first, of course, is to acknowledge the problem that we likely will be dealing with inflated noise (experimental error) when analyzing the findings of many COVID-19-era studies. The consequence of this could be smaller effect sizes and a heightened number of negative studies in the foreseeable future. The second implication is the desirability of disclosing possible COVID-19 contamination of study protocols. This means that investigators will report the types of events described in Table 1, as thoroughly as possible, within their reports. This needs to include not only the types of events summarized in the table, but also a characterization of where the study was in real time in relation to those events and the number of participants impacted by those events. The third implication, self-evident by now, is that much of child and adolescent psychiatric research in the COVID-19 era will be intrinsically more complex than similar research of the past. There is a marked need for investigators and statisticians to form strong alliances as early as possible to begin strategies for dealing with any study confounds that are laid bare by examination of study events and data. A fourth implication is that there is likely to be more missing data in COVID-19-era research because of changes to methods of data capture in the midst of the virus.
A fifth implication is that perhaps there needs to be more flexibility by reviewers and editors in dealing with COVID-19-era study reports. In general, we as reviewers should be rewarding candor by researchers. The events of COVID-19 were the fault of no researcher! In general, we should be tolerant of negative results, and this might be particularly so in the COVID-19 era. Sixth, it may be appropriate to probe our data/findings in greater depth than normally done in the past. For instance, it may be appropriate to block study participants in terms of enrollment to reduce error. It may also be appropriate to explore rational subgroups, perhaps as suggested by the cohorts described in Figure 1, for more coherent treatment effects. Finally, there may be a role for amalgamation by separate study teams to pool their data in an effort to “merge” data sets and to reveal trends that otherwise might not be apparent. If feasible, supplemental funding from research sponsors might be made available to researchers to run more study participants—especially if statistical procedures prove inadequate to control for differences in data collected before and after COVID-19. Journal editors and other central figures, such as program officers, may have a role to play in encouraging this type of enterprise.
There is an enormous amount of psychiatric research underway in this COVID-19 era. And, although we all hope for early eradication of the coronavirus, its ultimate duration is unknown but could be an unwelcome presence for some time to come. Our job now is to extract the maximum amount of information possible from that research.
Footnotes
Acknowledgments
We thank the following for critical suggestions on earlier drafts of this article: Drs. Rebecca Andridge, L. Eugene Arnold, Luc Lecavalier (Ohio State University), Cristan Farmer (NIH Intramural Program), and Cynthia Laurie-Rose (Otterbein University). We thank Savannah Brand, MA, and associates at Syneos Health for sharing published discussions about the impact of COVID-19 on research practices and, in particular, for sharing literature with us on equivalence of electronically presented and paper-and-pencil rating forms. Finally, we thank Marsha Aman for her very helpful editorial assistance with the article.
Disclosures
M.G.A. has received research contracts, consulted with, served on advisory boards, or done investigator training for Bracket Global; CogState Clinical Trials, Ltd.; J & J Pharmaceuticals; MedAvante-Prophase; Otsuka Pharmaceutical Development & Commercialization Inc.; Ovid Therapeutics; Hoffmann-La Roche; Supernus Pharmaceuticals; and Zynerba Pharmaceuticals. He receives royalties from Slosson Educational Publications. D.A.P. has received travel reimbursement and research support from Curemark LLC, research support from Biomarin and Novartis, and has served as a consultant to Curemark LLC.