How Pinteresting: An Analysis of Social Media Resources for Internalizing Disorders

Abstract

Many efforts have been made to understand social media and the resources existing online. However, prior studies have not thoroughly assessed specific platforms and the content being shared. The present study examined Pinterest content sharing as a proxy for interest among school personnel. Using Hall, Breeden, and Giacobe’s coding scheme, 657 pins from 499 randomly selected pinners following the National Association of School Psychologists’ Pinterest account were coded by content area and assessed for level of evidence base. Significant associations were found in chi-square analyses between category of internalizing disorders, evidence base, and types of pins shared. In addition, the category of internalizing disorder and level of evidence base were found to have a significant interaction with the ease of implementation. Assessing the content shared on Pinterest may inform future evidence-based implementation difficulties in schools.

Keywords

social media implementation internalizing disorders evidence-based intervention

Introduction

Internalizing disorders are the most common mental disorder for children and adolescents (Bluhm, Covin, Chow, Wrath, & Osuch, 2014). Reports from the Anxiety and Depression Association of America (ADAA) indicate anxiety disorders affect one in four adolescents and one in eight children, and as many as 12.5% of adolescents have depressive disorders (ADAA, 2016a, 2016b; Merikangas et al., 2010). According to the National Institute of Mental Health (NIMH) and National Alliance on Mental Illness (NAMI), 4% of adolescents are affected by trauma and stressor disorders, 2.7% of adolescents are affected by eating disorders, 2% of adults are affected by dissociative disorders, and 1% of adults in the United States are affected by obsessive-compulsive disorders (Merikangas et al., 2010; NAMI, 2017; NIMH, 2016a, 2016b, 2016c, 2016d, 2016e). This study aims to analyze information seeking among school personnel in regard to internalizing disorders using the social media platform Pinterest.

Despite the alarming prevalence of internalizing disorders, the recognition and identification of anxiety and depression has been difficult due to their covert nature, which results in frequent underreporting in the school setting (Bluhm et al., 2014; In-Albon, 2012; Miller & Jome, 2008). Due to this inward directedness and resulting underreporting, it is estimated that as little as 20% of children with internalizing disorders worldwide receive treatment, and even fewer receive evidence-based treatment (Bekker, Griffiths, & Barrett, 2016; Cobham, 2012; Elkins, Mchugh, Santucci, & Barlow, 2011; Essau, 2005; James, James, Cowdrey, Soler, & Choke, 2015). Furthermore, Miller and Jome (2008) found the majority of educational professionals felt they needed some or significant training in regard to internalizing disorders (p. 508). Because of the apparent lack of understanding surrounding internalizing disorders, educational professionals may be turning to other sources to obtain more information.

Although school psychologists can access helpful information in multiple ways including online continuing education, peer-reviewed journal articles, conferences, and workshop attendance, school psychologists are also turning to the Internet to find ways to support their students (Mittal, Gupta, Dewan, & Kumaraguru, 2014; Pham, 2014; Seaman & Tinti-Kane, 2013). Specifically, one avenue professionals may use is the use of extant social networking websites such as Pinterest. With the rise in social media use as well as the growing presence of educational professionals on social networking sites (Mittal et al., 2014), understanding the content type and quality of material being shared online concerning internalizing disorders is vital as little research has been conducted on what content is actually being shared on these sites and if those resources stem from quality sources. Moreover, school psychologists are not well informed about evidence-based interventions, reporting less than one third of respondents in the study by McKevitt (2012) indicated that they used journal articles to learn about interventions.

Not only do school psychologists lack knowledge about interventions to implement but also little is known about the relative quality of services provided within school settings (Langley, Nadeem, Kataoka, Stein, & Jaycox, 2010). Despite policies on a national level urging professionals to use evidence-based prevention and early intervention services in the schools, evidence-based programs are not successfully being implemented or are implemented with poor fidelity (Langley et al., 2010). Langley et al. (2010) identified four main barriers to the successful implementation of services: competing responsibilities, parent engagement, logistics, and support from administration and teachers. Funding has also been noted as a critical issue preventing implementation (Forman, Olin, Hoagwood, Crowe, & Saka, 2008). Without funding, it is difficult to acquire the training, staff, equipment, and materials necessary for successful implementation (Forman et al., 2008). Because of limiting factors in the use of evidence-based practices, the implementation of treatments not supported by the current evidence base is concerning as effects of the treatments are unknown and may be ineffective (Heier, 2016).

As school psychologists search for more information surrounding internalizing disorders, they turn toward other methods of gaining needed information to fill the need for interventions that do not rely on fiscal resources. Educators have often used Pinterest as an avenue for low- or no-cost strategies and are frequent users of other social media platforms such as Facebook, Twitter, Instagram, and Pinterest (Pham, 2014; Seaman & Tinti-Kane, 2013).

Internalizing Disorders

Internalizing disorders have been described as a “secret illness” given their inwardly directed nature (Miller & Jome, 2008, p. 501). Characteristics of internalizing disorders often include subjective distress and overcontrolled symptoms (Miller & Jome, 2008). According to the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, internalizing disorders include depressive disorders, anxiety disorders, obsessive-compulsive and related disorders, trauma and stressor-related disorders, and dissociative disorders (Regier, Kuhl, & Kupfer, 2013). While externalizing disorders cause distress in other individuals (e.g., tantrums, aggression, and other disruptive behaviors), internalizing disorders are developed, maintained, experienced and exhibited within the individual (Miller & Jome, 2008, p. 500). Furthermore, the understanding of childhood internalizing disorders has not progressed as quickly as other areas of psychopathology in the same age-group (Tandon, Cardeli, & Luby, 2009). The inward directedness of internalizing disorders decreases the ease of detection in children but increases the view from professionals that internalizing disorders are less problematic than others, as the symptoms are not typically overtly disruptive (Tandon et al., 2009).

Best practices

Currently, the research indicates that the gold-standard treatment for internalizing disorders is child- and family-focused cognitive behavioral therapy (CBT; Bekker et al., 2016; Elkins et al., 2011). Generally, the approach for children receiving CBT is similar to the approach for adults where CBT is administered over a 10- to 12-week period on a weekly or biweekly basis by a qualified clinician in a one-on-one clinic setting (Bekker et al., 2016). When implemented in this format, research has supported moderate to large treatment effects for the efficacy and effectiveness of CBT (Bekker et al., 2016; Elkins et al., 2011).

However, CBT is inaccessible to the majority of individuals requiring therapy for internalizing disorders as it is a high-cost treatment that requires transportation to a licensed clinician’s office. Evidence-based services provided by trained professionals are not able to meet the demand resulting in children not accessing professional care (Collins, Westra, Dozois, & Burns, 2004). To increase access to evidence-based treatments for children with internalizing disorders, creative modifications aim to offer CBT in a more portable format (Elkins et al., 2011). Low-intensity CBT methods have been discussed with a reduction in cost and employing staff only trained in the protocol being delivered as well as reducing the burden to the individual by implementing briefer interventions (Bekker et al., 2016). Yet, formats that would increase transportability of gold-standard treatments for internalizing disorders have not been the focus of recent empirical investigation (Elkins et al., 2011; James et al., 2015).

School psychologists and training needs

In a 2003 study, Farmer, Burns, Phillips, Angold, and Costello concluded that of the 54% of children nationwide who have used mental health services, the educational setting was the most common care provider. As the schools fill a central role in access to the mental health system, it is important for school psychologists to increase their knowledge on internalizing disorders to provide appropriate services to students in need. The school setting is paramount in prevention and early intervention because of the unparalleled access to children (Elkins et al., 2011).

In some cases, select teachers may not believe internalizing disorders are a significant problem for their students and would benefit from education around the identification of children at risk for internalizing disorders (Elkins et al., 2011). When surveyed, nationally certified school psychologists and education professionals consistently agree they should have knowledge of all internalizing disorders and conditions as 72% reported being not familiar with well-researched treatments and 89% rarely or never using evidence-based treatments in their daily practice (Hicks, Shahidullah, Carlson, & Palejwala, 2014; Miller & Jome, 2008).

In response to the underidentification of internalizing disorders, school psychologists in the United States are being called upon by the National Association of School Psychologists (NASP) to focus on social-emotional issues, including internalizing disorders (Miller & Jome, 2010). Although emerging research exists about the practices of school psychologists in relation to the assessment of internalizing disorders, little research exists in the area of perceived knowledge of internalizing disorders, role preferences, and training needs of school psychologists (Miller & Jome, 2008, 2010). Miller and Jome’s (2010) study was the first to research the perceived knowledge, role preferences, and training needs of school psychologists for internalizing disorders. The authors concluded school psychologists perceive a significant gap between received training and necessary skills required to respond to students with internalizing disorders.

Barriers to implementation of interventions

Furthermore, it is not solely that school psychologists lack training in the area of internalizing disorders. Previous research demonstrates best practices are not implemented due to various concerns. The most primarily cited barriers included financial concerns and access to resources to conduct an intervention (Elkins et al., 2011; Forman et al., 2008; Hicks et al., 2014; Miller, 2010; Whitney, Renner, Pate, & Jacobs, 2011) as well as time for intervention in the school day (Bekker et al., 2016; Elkins et al., 2011; Forman et al., 2008; Miller, 2010; Whitney et al., 2011). Other commonly cited barriers include the demands of additional training for school personnel (Elkins et al., 2011; Hicks et al., 2014; Langley et al., 2010; Miller, 2010) and the standardization and ease of implementation of materials (Miller, 2010; Whitney et al., 2011). Moreover, requiring participation on an individual basis in areas outside of the classroom prevents programs from being implemented as students begin missing academic instruction. In addition, government policies such as the Every Student Succeeds Act continue to place a high emphasis on results of standardized assessments while additionally requesting focus on social emotional learning and wellness, adding to mounting pressures for schools (Bekker et al., 2016; McKevitt, 2012; Miller, 2010).

Online Information Seeking

To fill the perceived training gap, address barriers to implementation, and effectively respond to students with internalizing disorders, school psychologists may look to the Internet as a potential source for more information. The Internet provides convenient, low- or no-cost opportunities for the dissemination of information as well as the sharing of ideas between individuals. School psychologists could turn to published articles, websites, personal blogs, Facebook groups, Pinterest pins, and more for ideas in the areas in which they are lacking information. Despite the wide availability of information online, concerns remain regarding the relative quality of resources online.

Much of the research on the content quality of online resources relates to parents and parents of children with disabilities. Subsequently, the content found on websites generally is neither evidence-based nor of high quality (Di Pietro, Whiteley, Mizgalewicz, & Illes, 2013). This is exemplified in the use of personal blogs, posts on Facebook, and pins on Pinterest, where individuals are able to use personal experiences instead of consulting evidence-based strategies to create their own content. The resulting content that is available for view by the public may not have evidence-based strategies from research but sometimes do contain theory-based strategies that may not be part of programs. Having personal blogs with no references but some content that can be shown to be effective can promote the promotion of strategies that are of mixed effectiveness but lacking in scope, sequence, and intentions. Furthermore, online websites with information on developmental disabilities were found to be differing in content and terminology, lacking consistency necessary for clarity (Reichow, Gelbar, Mouradjian, Shefcyk, & Smith, 2014). Even websites with resources that appear to be of high quality such as .gov or .org websites have been found to be avoided by some parents as they can be perceived as more complicated or confusing than helpful (Hall, Culler, & Frank-Webb, 2016).

Research on online information has focused primarily on resources pertaining to autism, parent information, and platforms other than Pinterest, and a dearth of research exists on online resource utilization for educational professionals with the exception of instructional use of social media (Mittal et al., 2014; Seaman & Tinti-Kane, 2013) and ethics of social media use (Segool, Goforth, Bowman, & Pham, 2016). Currently, limited research regarding internalizing disorders is reflected in the content on Pinterest. Investigation into this area began with a 2016 study where Guidry, Zhang, Jin, and Parrish (2016) examined the portrayal of depression on Pinterest and found a high perception of severity of depression in Pinterest users in the form of 20% of pins referring to dysfunctional coping strategies such as behavioral disengagement, self-blame, or denial and 10% referring to thoughts of suicide or committing suicide. Guidry et al. (2016) looked at overall search results of depression to all content on Pinterest. The present study investigates content being pinned by the followers of NASP, which promotes student welfare, and this may drastically change the picture from the results found by Guidry et al. (2016).

Due to the pervasive gaps in the literature for internalizing and social-emotional student issues and the noted dissonance between received training and necessary skills for response, school psychologists are driven to search for more information to respond to their students, even if the content may not be ideal. The present study aims to address Pinterest as a newer platform providing resources and to further investigate the content on Pinterest reflecting internalizing disorders more broadly than the current literature contains.

Social media

Presently, 65% of American adults take part in social networking, increasing from 2005 where it was found only 7% of American adults used social media (Perrin, 2015). With the evident surge in social media users, faculty and school personnel have not been exempt from the increase of those using social networking sites (Seaman & Tinti-Kane, 2013). In fact, educators have become frequent users of social media such as Facebook, Twitter, Instagram, and Pinterest (Pham, 2014). Using these platforms, educators look for ideas to inform their practice as social media websites provide opportunities for school personnel to communicate and consult in ways that have not previously been available (McKevitt, 2012; Pham, 2014). The rise in social media usage allows the free, widespread exchange of professional content to an audience of millions (Pham, 2014).

One platform of increasing attention for educators is Pinterest (Mittal et al., 2014; Pham, 2014; Seaman & Tinti-Kane, 2013). In September 2015, Pinterest surpassed 100 million unique monthly active users (MAUs), valuing the company at $11 billion after its inception in 2010 (Granger & Reiter, 2015). For comparison, the social networking giants of Facebook and Twitter took 4 and 5 years, respectively, to reach 100 MAUs (Eldon, 2008; Yeung, 2015). As of June 8, 2016, Pinterest’s 100 million MAUs had saved 75 billion total ideas on the website (Gavini, 2016). With a monthly average of 2 billion searches, Pinterest’s guided search offers not only the most relevant answer but also ideas around the term being searched (Gavini, 2016). Therefore, Pinterest’s Engineering Manager, Naveen Gavini (2016), likens the site more toward the search engine of Google rather than social networking sites such as Facebook or Twitter. Being able to search for various terms and gain information about similar ideas makes Pinterest a prime platform for educators and school personnel looking for information to assist their students with internalizing disorders.

Users on Pinterest, commonly called pinners, are able to share ideas by creating a public collection of images. To do so, users pin images to a board, much like a virtual bulletin board. Shared images, referred to as pins, consist of a screen capture from other websites, a description, and a hyperlink to an associated website (Feng, Cong, Chen, & Yu, 2013). Looking at all pins present on Pinterest, education has the second highest number of pinners per pin with an average of 10.34 followers per pin posted, showing the exchange of professional ideas is extremely high in this area (Mittal, Gupta, Dewan, & Kumaraguru, 2013).

Pinterest and educators

The rising popularity of Pinterest shows a new opportunity for educators and other school personnel to collaborate and share information. Currently, teachers have used Pinterest to save ideas related to new classroom activities as well as share information with their colleagues and support student research (Taylor, 2015). Teachers, however, are not the only school personnel using this social media site. School counselors and school psychologists are also noted as users of Pinterest. Even national organizations in psychology such as the American Psychological Association, the NASP, and the American School Counselors Association have Pinterest accounts created to share information with professionals in these fields (Taylor, 2015).

As of November 2017, the NASP Pinterest account had more than 5,700 followers and 1,250 pinned images. Many of NASP’s followers on Pinterest have created accounts that contain a pinboard where they share pins related to the areas of education and school psychology (Hall, Breeden, & Giacobe, 2018). The present research in school psychology and related disciplines has lagged in the investigation of Pinterest as a tool for educators who post content on Pinterest as resources for their own practices and to provide information for others searching for help.

Research does exist about the overall, general behaviors of pinning on Pinterest (Feng et al., 2013; Mittal et al., 2013, 2014; Wang, Yang, Zheng, & Sundar, 2016; Wei & Zhang, 2016). Posting content on Pinterest in the area of school psychology and related disciplines provides the dissemination of content to the education community. Content on Pinterest can include providing information, assessment tools, intervention ideas, and more that pinners can access by simply searching for a term. Therefore, Pinterest could be a valuable tool to fill the gaps identified by school psychologists (Miller & Jome, 2010).

Despite the possibility that existing social media platforms like Pinterest could be ways to disseminate evidence-based strategies and low- and no-cost solutions for practitioners, the content shared on Pinterest may have corresponding weblinks that have the possibility to link back to websites that may not be verified as being based in sound research. Therefore, it is a concern that the information provided on pins is not consistent with best practices. The present study aims to address the lack of research specifically investigating the quality of the resources for internalizing disorders shared by school psychologists on Pinterest.

Present Study

Schools are aware that significant numbers of children currently experience symptoms of emotional or behavioral problems or are at risk of developing a disorder (Eklund et al., 2009). Of these emotional and behavioral problems, the most common are internalizing disorders, including depressive disorders, anxiety disorders, obsessive-compulsive and related disorders, trauma and stressor-related disorders, and dissociative disorders (Miller & Jome, 2008; Regier et al., 2013). Despite the prevalence of internalizing disorders in the school setting, a majority of educational professionals still feel unprepared to respond to or support students experiencing these conditions (Miller & Jome, 2008, 2010). To help fill the need for more information, school psychologists have been turning to social networking sites such as Pinterest (Mittal et al., 2014; Seaman & Tinti-Kane, 2013). However, whether the content being shared on Pinterest is supported by research and evidence-based practices is unclear (Di Pietro et al., 2013; Hall et al., 2016). To better understand the state of knowledge surrounding internalizing disorders on social media, the following two research aims will guide the present study.

The first research aim concerns the types of pins and internalizing disorders shared by school personnel and whether these pins are supported by the current evidence base. The study hypothesized that associations will exist between the category of internalizing disorders, level of evidence base, and types of pins shared. The nature of association is anticipated to have more observed instances than expected of informational pins in the nonevidence-based category given that Pinterest content is largely not vetted by any specific body such as the What Works Clearinghouse or other entity. More observed instances than expected of anxiety-related content are anticipated in the informational category because the current literature suggests school psychologists and related school personnel view anxiety as paramount to other internalizing disorders (Miller & Jome, 2008). Furthermore, an overrepresentation of anxiety pins, underrepresentation of intervention pins, and underrepresentation of pins supported by the current evidence base were hypothesized. These representations are expected due to the current literature stating school psychologists demonstrate training needs for evidence-based interventions for internalizing disorders (Miller & Jome, 2008). If this hypothesis is supported, it would provide insight to inform future directions of information disseminated and searched for online.

The second research aim investigates the implications of school personnel implementing the activities identified in the pins coded as interventions. Previous literature identifies common barriers to implementation for educators (Bekker et al., 2016; Collins et al., 2004; Elkins et al., 2011; Forman et al., 2008; Heier, 2016; Hicks et al., 2014; Langley et al., 2010; Whitney et al., 2011). Each intervention will be scored on how easily it may surpass these barriers. Coding for ease of implementation in relation to barriers will result in a value for implementation, the higher value being the more easily the intervention can be implemented by school personnel. Therefore, due to gold-standard practices often having barriers such as high-cost, inaccessible resources, and additional training demands, it is hypothesized that there will be a main effect of the level of evidence base on how easy the intervention is to implement. Support of this hypothesis would imply school personnel will generally use nonevidence-based interventions because they are more accessible and easier to implement instead of using the evidence-based activities suggested by the current research.

Method

Unit of Analysis—Boards and Pins

The current study focused on content concerning internalizing disorders that is being shared on Pinterest by school personnel. As the contents (pins) are the unit of analysis, the presumed school psychologists (pinners) following NASP’s Pinterest account were not considered research participants. All information was obtained from the public domain; therefore, consent was not collected as the research did not involve collection from human subjects. Pinterest as a social media platform allows its users to maintain a profile on their site by signing up with an e-mail and password combination or through another social media network, such as Facebook or Twitter. However, what differentiates Pinterest from other social media networks is that Pinterest does not permit users to keep their accounts entirely private (Mittal et al., 2013). Instead, pinners have a profile hosted by the public domain, and the specific boards created to categorize the pinners’ pins have the ability to be made private by the pinner. If a board is made private, only the people the pinner selects to provide access are able to view the pins. For this reason, private boards are not included for analysis in the present study.

Procedures

Sampling

To find pins related to internalizing disorders from the 5,730 followers of NASP on Pinterest, 499 followers of the NASP Pinterest account were acquired through a random sample using randomizer.org, an online resource that was designed to provide series of random numbers to help obtain random samples for research purposes (Urbaniak & Plous, 2013). Screenshots were taken on 1 day of NASP’s Pinterest followers to obtain a consistent sample. Without these screenshots, the sample of the study would have been subjected to variability because users follow and unfollow the NASP account every day. Contents such as the user boards and pins were obtained from these randomly selected accounts and were examined. Boards containing school psychology content were identified, resulting in a total of 1,163 boards as some of the randomly selected users had more than one board on their profile matching content for school psychology. Selenium WebDriver, a browser automation tool commonly used for automated website testing while under development, was used to capture images of the boards containing material related to school psychology. The images (pins) were then stored offline for further analysis. To identify exact matches of pins, the MD5 hash was calculated for each image (Rivest, 1992).

Coding

The database of pins was compiled and qualitatively coded using the coding scheme adapted from Breeden, Hall, and Giacobe (2018). The MD5 hash enabled the analysis to investigate the spread of unique images to show the impact of sharing various images and associated content. Coding for the initial content was completed by the lead author and another graduate student in school psychology. Reliability of coding was estimated by both coders independently coding 30% of the initial pins for pin content. Any disagreement in coding was resolved through discussion. To assess the variety of content for each pin, component analysis was used by the author. Internalizing related pins were coded according to a predetermined set of codes. To assess the implementation value for pins coded as intervention, a coding scheme was developed and used by the author.

Network impact

Network impact of an individual pin on Pinterest is measured through analysis of content sharing. Although some identical pins were able to be collapsed, some pins were identical but had cryptographic hashes. Therefore, pins that were similar were noted during coding. Similar pins were considered to be sharing the same content using a different image or URL attached. Similar pins were also counted as a measure of network impact of the underlying concept. These three methods combined provided a more in-depth analysis of individual pin and concept network impact.

Measures

Content coding

The coding scheme adapted from Hall et al. (2018) included qualitative coding in two forms: content-related area and component analysis. Pins with a description, title, or associated weblink mentioning a coding term were used to code each pin. For example, if a pin’s description included a diagnosis considered to be internalizing, such as anxiety or depression, that pin would be coded as internalizing disorders. The full list of content codes can be found in the original study by Hall et al. (2018). Pins coded into the internalizing disorders category were used for analysis in the present study, resulting in a total of 657 pins. Using this coding scheme, the overall interrater reliability was found to be .80 for 30% of the sample which totaled 197 pins. For each set of coding, which consisted of 100 images, a kappa coefficient of .60 was considered adequate for interrater reliability.

Component analysis

To analyze the dissemination of information and if the techniques and strategies associated with the pins were based on the current evidence from research, the pins were analyzed according to component categories. These predetermined categories of codes were based on the possible types of pins, including informational, identification, and intervention. Subsequently, each pin was coded into predetermined categories of evidence level: no resource offered, nonevidence-based, limited evidence base, and evidence-based. See Table 1 for the component coding scheme.

Table 1.

Component Codes.

Overall category	Code category	Definition
Evidence base	No resource offered	Led back to an error page and did not contain any information on the image itself
	Nonevidence-based	Offered strategies or information regarding internalizing disorders that were incorrect according to the current evidence base
	Limited evidence base	Included theories of evidence but lacked citations or labels of the programs
	Evidence-based	Cross-referenced to the Evidence-Based Programs Registry: Blueprints for Healthy Youth Development (Blueprints) or the National Registry of Evidence-based Programs and Practices
Type of pin shared	Informational	Provided a description of a specific internalizing disorder or general information about internalizing disorders
	Identification	Had information related to screeners, warning signs, or symptoms
	Intervention	Contained programs or strategies to combat internalizing disorders

Due to the nature of the Internet, some associated weblinks were found to be broken, meaning the weblink lead back to an error page or the site no longer existed. In these instances, the information written in text on the image of pin itself or the written caption was used to code the pin for level of evidence. If the pin itself or the caption did not contain any information at all, it was considered as no resource offered. As this study is examining the dissemination of content on Pinterest by and for educational professionals, it was necessary to consider these pins with no text, no caption, and no working link as no resource offered because pinners are still able to see these pins when they search for terms such as internalizing disorders in the search bar. When a pinner clicks on the pin that is just an image without text, no resource is attached to it, and it offers nothing to the educator in that moment. This differs from the nonevidence-based label because the nonevidence-based pins offered resources, strategies, or information that was contrary to the current literature and had no sources to cite the effects being claimed.

Network impact

The network impact of internalizing related pins was measured in three ways: (a) through the number of repins and comments associated with each pin; (b) analysis of the pin’s MD5 hash, unique URL, and image file; and (c) the MD5 hash, image file name, and unique URLs were used to examine the number of times a pin appears on Pinterest. This can be analyzed as occurring across pinners. This method allowed multiple pins with the same MD5 hash, image file, or URL to be collapsed. If a pin was duplicated and collapsed, it was recorded as a measure of network impact.

Implementation value

Once the pins had been coded for type and level of evidence, the pins coded as intervention were further analyzed. Each pin was assigned a value based on a coding scheme developed by the author from previous literature that had identified barriers to implementation (Bekker et al., 2016; Collins et al., 2004; Elkins et al., 2011; Forman et al., 2008; Heier, 2016; Hicks et al., 2014; Langley et al., 2010; Whitney et al., 2011). Ten opportunities were created where an intervention could earn up to three points each for a total implementation value of 30. These opportunities are listed in the following.

How much does the intervention cost in total?

Is the intervention easy to access?

How long does the intervention take?

Is the intervention recurring?

Does the intervention have instructions?

Who can implement the intervention?

How much training is required?

How many students can participate at once?

Where can the intervention take place?

Can the intervention be modified while maintaining core principles?

The full coding scheme can be found in Table 2. Implementation values were assigned for each statement and totaled for each intervention pin.

Table 2.

Implementation Value Coding Scheme.

	P oint values
Statements	3	2	1	0
How much does it cost?	Free	<$100	$100–$300	>$300
Is it easy to access?	No materials necessary	Print from browser or use available	Buy from a store	Order from manufacturer
How long does it take?	10 minutes	30 minutes	1 hour	2+ hours
Is it recurring?	No	1/week	2/week	>2/week
Does it include instructions?	Detailed step-by-step	Basic steps	Can recreate	No
Who can do it?	Anyone	Special education teacher/counselor	SPSY or administrator	Licensed psychologist or interventionist
How much training is required?	None	1 hour	1–5 hours	>5 hours
How many people can participate at once?	Whole class	Large groups	Small groups	Individual
Where can it take place?	Classroom	Special education classroom	Individual counseling	Outside school
Can it be modified while maintaining core principles?	Modified as needed	General modifications allowed	Specific modifications allowed	No modifications allowed

Note. SPSY = school psychologist.

Results

A total of 657 pins were analyzed. The majority (55.4%) were coded as anxiety-related with depressive-related content accounting for the second-most pins (27.2%) in the sample. The rest of the sample was distributed between dissociative-related content (5.2%), trauma and stressor-related content (4.7%), multiple categories of content (2.9%), obsessive-compulsive content (2.3%), and eating disorder-related content (2.3%). The type of pin shared was most often informational in nature (49.3% of the sample), closely followed by activities for intervention (42.9% of the sample). Pins concerning the identification of internalizing disorders were the least shared (7.8%). By a large margin, the pins shared were mostly nonevidence based (39.7%), followed by pins offering no resources, accounting for 21.5% of the sample. Pins with a limited basis in evidence were found to represent 20.7% of the sample, while pins supported by the current literature represented only 18.0% of the sample. In 33.5% of the sample, or 220 pins, when the associated weblink was clicked on, it led back to an error page or a broken link. Refer to Table 3 for a full description of frequency counts of pins in the sample. Finally, of the 282 intervention pins, each were assigned values based on the ease of implementation. These scores ranged from a minimum implementation value of 4 to a maximum implementation value of 30. The mean value was 23.51 with a standard deviation of 6.28.

Table 3.

Frequencies of Pins in the Sample.

Category	Total count	Percentage of sample
Category of internalizing disorder (N = 657)
Anxiety	364	55.4
Depressive	179	27.2
Dissociative	34	5.2
Trauma/stressor	31	4.7
Multiple categories	19	2.9
Obsessive-compulsive	15	2.3
Eating	15	2.3
Type of pin shared (N = 657)
Informational	324	49.3
Intervention	282	42.9
Identification	51	7.8
Level of evidence base (N = 657)
Nonevidence-based	261	39.7
No resource offered	141	21.5
Limited evidence base	136	20.7
Evidence-based	118	18.0
Broken links (n = 220)
Organizational	73	33.2
Parent/personal blog	58	26.4
School personnel blog	48	21.8
General website	41	18.6

Research Question 1: Pins Shared by School Personnel

Chi-square and single-cell contingency analyses through SLEIPNER version 2.1 were used to determine if the mentioned associations and representations were present (Bergman & El-Khouri, 2002). The criteria threshold used to determine if these analyses were consistent with the hypotheses was an alpha level of .05.

An initial chi-square test of independence was conducted between category of internalizing disorder and level of evidence base. Due to too many cells having expected counts less than 5 in the internalizing disorder variable, categories with expected counts less than 5 were collapsed into the label other. There was a statistically significant association between internalizing disorder and evidence base, χ²(6) = 14.839, p =.022. There were significantly more anxiety-related pins in the nonevidence-based category than any other categories. This association has a small effect size, Cramer’s V = .106. These results, including indications of which cells significantly deviated from independence, can be found in Table 4.

Table 4.

Crosstabulation of Internalizing Disorder and Evidence Base.

	Level of evidence base
Internalizing disorder	No resource offered	Nonevidence-based	Limited evidence base	Evidence-based
Depressive (n = 179)	47⁺26.3%	6737.4%	28^–15.6%	3720.7%
Anxiety (n = 364)	6919.0%	160⁺61.3%	7958.1%	56^–15.4%
Other (n = 114)	2521.9%	34^–29.8%	2925.4%	2622.8%
Total (n = 657)	14121.5%	26139.7%	13620.7%	11918.1%

Note. The overall relationship was χ ² (6, n = 657) = 14.839, p < .05. +/– indicates significantly more/fewer than expected by chance in single-cell contingency analysis (p < .05).

Single-cell contingency analyses, using SLEIPNER version 2.1 (Bergmann & El-Khouri, 2002), indicated depressive disorders were overrepresented in the pins with no resource offered while underrepresented in pins with limited evidence base. Anxiety disorders were overrepresented in the nonevidence-based category and underrepresented in the evidence-based category. Other internalizing disorders were underrepresented in the nonevidence-based pins.

A second chi-square test of independence was conducted between category of internalizing disorder and type of pin shared. All expected cell frequencies were greater than five. There was a statistically significant association between internalizing disorder and type of pin shared, χ²(6) = 33.318, p < .0001. There were significantly more anxiety-related pins in the intervention and informational categories than any others. This association has a small to moderate effect size, Cramer’s V = .159. These results, including indications of which cells significantly deviated from independence, can be found in Table 5.

Table 5.

Crosstabulation of Internalizing Disorder and Pin Type.

	Pin type
Internalizing disorder	Informational	Identification	Intervention
Depressive (n = 179)	8648.0%	23⁺12.8%	7039.1%
Anxiety (n = 364)	163^–44.8%	17^–4.7%	184⁺50.5%
Other (n = 114)	75⁺65.8%	119.6%	28^–24.6%
Total (n = 657)	32449.3%	517.8%	28242.9%

Note. The overall relationship was χ ² (4, n = 657) = 33.318, p < .001. +/– indicates significantly more/fewer than expected by chance in single-cell contingency analysis (p < .05).

Single-cell contingency analyses indicated depressive pins were overrepresented in the identification type pins. Anxiety-related content was found as underrepresented in the informational type and the identification type and overrepresented in the intervention type. All other internalizing disorders were found to be overrepresented in the informational type and underrepresented in the intervention type.

A third chi-square test of independence was conducted between type of pin shared and level of evidence base. All expected cell frequencies were greater than five. There was a statistically significant association between the type of pin shared and level of evidence base, χ²(6) = 39.016, p < .0001. There were significantly more intervention pins in the nonevidence-based category than any others. This association has a small to moderate effect size, Cramer’s V = .172. These results, including indications of which cells significantly deviated from independence, can be found in Table 6.

Table 6.

Crosstabulation of Pin Type and Evidence Base.

	Level of evidence base
Pin type	No resource offered	Nonevidence-based	Limited evidence base	Evidence-based
Informational (n = 324)	7723.8%	117^–36.1%	80⁺24.7%	50^–15.4%
Identification (n = 51)	1019.6%	7^–13.7%	19⁺37.3%	15⁺29.4%
Intervention (n = 282)	5419.1%	137⁺48.6%	37^–13.1%	5419.1%
Total (n = 657)	14121.5%	26139.7%	13620.7%	11918.1%

Note. The overall relationship was χ ² (6, n = 657) = 39.016, p < .001. +/– indicates significantly more/fewer than expected by chance in single-cell contingency analysis (p < .05).

Single-cell contingency analyses indicated informational and identification pins were underrepresented in the nonevidence-based category, while intervention pins were overrepresented. In the limited evidence base category, informational and identification type pins were overrepresented, and intervention pins were underrepresented. In the pins that were supported by the current literature, informational pins were underrepresented, and identification pins were overrepresented.

Research Question 2: Implementation Value of Shared Interventions

A two-way factorial analysis of variance (ANOVA) was used to examine the dependent variable of implementation value, how easy the intervention is to implement for school personnel, and the effects of two independent variables: category of internalizing disorder and level of evidence base. Residual analysis was performed to test for the assumptions of the two-way ANOVA. There were seven outliers and eight extreme outliers as visually assessed with a box plot. A second analysis was conducted without the outliers in the sample and found no appreciable difference in the level of significance and values of the confidence intervals in the analysis. Therefore, the outliers were deemed appropriate to include in the sample for the analysis.

As assessed by Shapiro–Wilk’s test (p > .05), data were not normally distributed. Because ANOVAs are considered to be fairly robust to these deviations from normality, it was determined the analysis could still be conducted as the groups were similarly skewed (Maxwell & Delaney, 2004). Keeping the original data for clarity rather than transforming the dependent variable was considered a more representative picture of the content present on Pinterest.

The homogeneity of variances assumption was violated, as visually assessed using a scatterplot of predicted values for implementation value and the residual and studentized residual for implementation value. A square transformation of the data did not result in a change in the significance of the overall two-way ANOVA nor did it result in a change for the assumption of normality and the homogeneity of variances. Thus, continuation with the original data for clarity was determined as the best option, using caution when interpreting the conclusions of the ANOVA.

A statistically significant disordinal interaction was found between the category of internalizing disorder and the level of evidence base for ease of implementation score, F(6, 270) = 4.924, p < .001, partial η² = .099. On closer examination, it appears that for anxiety- and depressive-related content, the interventions with a higher value for implementation were less evidence-based. This relationship reverses for other categories of internalizing disorders, where nonevidence-based activities were found to have lower implementation values than the limited evidence- or evidence-based interventions. A summary of the two-way ANOVA can be found in Table 7, and the plots of the interaction and bar graph of differences can be found in Figures 1 and 2.

Figure 1.

Plot of two-way ANOVA interaction between category of internalizing disorder and level of evidence base on the ease of intervention implementation.

Figure 2.

Differences between categories of internalizing disorders and level of evidence base on the ease of intervention implementation.

Table 7.

Summary of Two-Way Factorial Analysis of Variance.

	df	F	Partial η²	p
Internalizing disorder	2	30.522	.000	.184
Evidence base	3	1.296	.276	.014
Internalizing Disorder ×Evidence Base	6	4.924	.099	.000

Discussion

The present study has focused on Pinterest as a newer platform for providing resources to school personnel in the area of internalizing disorders, where a dearth of knowledge seems to exist as demonstrated by pervasive gaps in the literature and the dissonance between received training and necessary skills for response in school personnel. This study examined the content being shared on Pinterest to see if it was supported by research- and evidence-based practices. This provides a better understanding for the state of knowledge in the field of school psychology surrounding internalizing disorders on social media and to inform future directions of information disseminated online. This study also investigated the implications of school personnel implementing the activities found online to inform future directions of research regarding transportability of evidence supported interventions.

Current Content Present on Pinterest

Categories of internalizing disorders

This study found anxiety and depressive disorders dominated 82.6% of the sample, leaving little numbers of pins concerning obsessive-compulsive, trauma/stressor, dissociative, and eating disorders. This mirrors previous research that found professionals stated, of all classified internalizing disorders, school phobia or refusal and suicidal threats or acts were of the utmost importance to be knowledgeable about, while eating disorders such as anorexia and bulimia were rated as the least important (Miller & Jome, 2008). Furthermore, the focus on anxiety and depression is reflective of the current prevalence rates of internalizing disorders in the United States where 25% and 12.5% of adolescents experience anxiety- and depressive-related disorders, respectively, whereas only 4% of adolescents, 2.7% of adolescents, 2% of adults, and 1% of adults experience trauma/stressor-related, eating, dissociative, or obsessive-compulsive-related disorders, respectively (ADAA, 2016a; Merikangas et al., 2010; NAMI, 2017; NIMH, 2016a, 2016b, 2016c, 2016d, 2016e). The large focus on anxiety and depression and concurrent lack of information concerning the majority of internalizing disorders was demonstrated in the first and second chi-square analyses that found significant associations between internalizing disorders and evidence base as well as between internalizing disorders and type of pin shared.

Types of pins shared

It was also found only 7.8% or 51 pins of the total sample were tools for identification of internalizing disorders. Essentially, this indicates materials such as screeners are not being shared online between professionals trying to find children at risk for internalizing disorders. Despite internalizing disorders being coined as the “secret illness” and the covert nature of the disorder being mentioned across various studies in the literature, it is not reflected in the content present on Pinterest that anything is being done to improve the number of children being identified.

Analysis of the category of internalizing disorder and type of pin shared by the chi-square procedure showed that anxiety-related content was present more often in the intervention category, while other internalizing disorders were significantly underrepresented in the intervention category. This indicates that the content on Pinterest that offers resources and activities to mitigate symptoms of internalizing disorders most often focuses solely on anxiety, and little exists on dissociative, eating, trauma/stressor, or obsessive-compulsive disorders. Having the majority of internalizing disorders content on Pinterest being related to anxiety disorders is potentially due to practitioners viewing anxiety as paramount to other disorders. School personnel in prior literature have rated school/phobia or refusal and generalized anxiety disorder as the foremost disorder that is appropriate for school psychologists to assess (Miller & Jome, 2008, p. 504). Because the effects of anxiety are more visible, such as school refusal or anxiety before taking tests, education professionals may consider anxiety more important than other internalizing disorders.

Moreover, within the identification type of pins, there was an overrepresentation of depressive disorders and an underrepresentation of anxiety disorders. This implies, within the pins examined, there were more depressive pins and less anxiety pins than expected to be present, indicating the tools the identification pins offer are largely based on the identification of depressive disorders. Pinning in this area suggests school psychologists and related practitioners may be actively looking for a way to bridge the gap in identification of depressive disorders.

In addition, anxiety- and depressive-related disorders accounted for more than half of the identification pins, again mirroring the previous literature that these disorders trump the others as their symptoms are perceived to be more important than other internalizing disorders (Miller & Jome, 2008). It is encouraging to note while the identification tools offered mostly concern depressive disorders, identification pins were overrepresented in the evidence-based category as well as the limited evidence base category, demonstrating these available tools are effective methods of identifying these depressive disorders.

Evidence base

While the identification pins offered tools supported by research, it was disheartening to find that only 18.0% of the sample were pins supported by the current research. Ideally, significant associations would be expected in the first and third chi-square analyses between the category of internalizing disorder and level of evidence base as well as the type of pin shared and the level of evidence base due to a high level of evidence shared. While significant associations were found between the category of internalizing disorder, type of pin shared, and level of evidence base, it seems to be the result of the large amount of nonevidence-based pins accounting for 39.7% of the sample.

This high prevalence of pins not supported by research can be exemplified by the anxiety-related pins, which accounted for more than half of the total sample, was underrepresented in the evidence-based category. These pins included pictures or text with suggestions such as using essential oils to mitigate anxiety, suggested prayers to ease stress and anxiety, or various arts and crafts activities to use to discuss anxiety with your students or children. These activities did not mention or refer to any cited evidence or gold-standard practices that are available online. Therefore, this suggests that of the 364 pins examined, there were significantly less pins in the evidence-based category than expected.

Having significantly less pins in the evidence-based category than expected is concerning as this speaks to the large number of resources that will be present when educators search for materials and information to respond to students with internalizing disorders in their classrooms or schools. The consequences could include educators accessing incorrect information about these disorders or seeing activities that have not been studied or shown to be effective by trusted literature.

Intervention activities

Specifically, in the 282 pins identified as activities for interventions, this lack of evidence is clear by overrepresentation in the nonevidence-based category. This displays there were more intervention pins present in the nonevidence-based category than expected. This is possibly the result of activities for interventions often being found on one of the 73 links to blogs created by parents of children with these disorders who use personal experiences to often advise other parents in how to “cure” their child’s disorder. Blogs created by parents account for 11.1% of the total number of links. There were also 75 links to personal blogs created by school personnel that offer information and suggestions that may include components of evidence but, more often than not, do not provide additional resources or mention evidence-based resources by name or peer-reviewed reference such as journal article citations. Personal blogs created by educators account for 11.4% of the total number of links. Having information disseminated from personal blogs created by parents and school personnel is concerning as the sites often did not use research to validate their claims, instead relying on personal experiences.

The personal experiences that create the content of these blogs illustrate practitioners coming up with their own solution to meet the demands of their students with these internalizing disorders. Sharing answers and solutions from individual professionals demonstrates a vast online community attempting to bridge the gaps that have been identified by practitioners. To mitigate some of these experiences lacking evidence, the field of school psychology and closely related professions could consider providing guidance and resources on these platforms for the individual professionals and parents can find supports that are based in the current literature.

Furthermore, there was an overrepresentation of anxiety disorders and an underrepresentation of other internalizing disorders within the pins offering intervention activities. This signifies there were more interventions for anxiety disorders than expected and less intervention activities directed at the majority of other internalizing disorders. Therefore, most of the intervention activities shared by pinners focused on anxiety, leaving very little resources for all other classifications of internalizing disorders.

Dissemination of information from organizational websites

In 220 pins, 33.5% of the total overall sample of 657 pins, the associated weblink brought up an error page. Seventy-three cases (33.2%) of broken links were error pages on organizational websites such as NASP. This shows these pins were once evidence-based pins offering quality resources to school personnel. However, because organizational websites often restructure their layout by way of periodic website redesign and server changes, this causes the old links associated with pins to become broken as the links can only be manually updated on pins. This is compared with the personal parent and school personnel blogs that are simple point-and-click formats and are often available since their creation, as they are not reformatted on such a large scale as organizational websites. Broken links occurred only in 58 links to parent and personal blogs and in 48 links to school personnel blogs. Broken links occurred the least in 41 links to general websites. Refer to Table 3 for a complete breakdown of the types of websites containing the broken links.

This loss of evidence-based information and quality resources available on Pinterest is concerning as many educational professionals are turning to online sources to gain information (Mittal et al., 2014; Pham, 2014; Seaman & Tinti-Kane, 2013), not just on internalizing disorders. If the major organizations continue to restructure their websites without concern for these new platforms that are linking back to these gold-standard resources, many evidence-based resources will continue to be lost, and the activities suggested by parents that are not well researched will be the only activities remaining for use.

Moreover, it is also common for research-based information and activities for interventions to require a membership to access, as was present in 27 links or 4.1% of the sample. This is often the case when looking for journal articles about interventions and any resources on large organizational websites such as NASP. If this log in is required to view the information, school personnel are not inclined to purchase an annual membership to access the information as financial concerns are already a stated barrier to implementation. Therefore, dissemination of information backed by evidence is lower due to the required payment to access it. If national associations like NASP want professionals to use research-based practices, making these resources more accessible would be the first step to increasing use.

Quality of the Content Present on Pinterest

According to prior literature, school personnel are more likely to use interventions with fewer implementation barriers such as high cost, limited resources, lengthy time requirements, or additional training needs (Bekker et al., 2016; Elkins et al., 2011; Forman et al., 2008; Heier, 2016; Hicks et al., 2014; Langley et al., 2010; Miller, 2010; Whitney et al., 2011). This study found that the category of internalizing disorder and the level of evidence base had a significant interaction on the ease of implementation of intervention activities. In the depressive category, nonevidence-based activities had the highest implementation value, indicating they were the easiest activities for educators to use in their classroom. The highest implementation value interventions often included detailed, step-by-step instructions to lead exercises in the classroom; worksheets that a practitioner could simply print out from a website; or lists of strategies to combat symptoms of disorders. Having the nonevidence-based activities come first in how easy they are to conduct is concerning, as the nonevidence-based pins offered activities that were unverified by the current literature and could be ineffective.

The second easiest activities in the depressive category were those without any resources offered on the pins. This signifies the image had text and suggested activities without a website to link back to. This little amount of information about the activity is concerning because the effects on students are unknown. The activities with a limited evidence base or complete support from the current literature were found very similarly as the hardest activities to implement, indicating school personnel generally would not use these activities due to their inaccessibility.

In the category of anxiety disorders, it was encouraging to find that the limited evidence base activities were as easy to implement as the activities with no basis in evidence. With activities that are based on theories of evidence, even if no specific references are there, at least the potential of adverse effects on the anxiety of students is low. Similarly to the previous category, evidence-based interventions were found to be the lowest form of activities to use in schools, further suggesting less use of these activities due to their inaccessibility.

In all other internalizing disorders, this relationship was interestingly reversed. Nonevidence-based activities were found to be the least accessible, followed by activities suggested without a resource. Limited evidence-based activities and evidence-based interventions were found to be the first- and second-most accessible, respectively. Despite the low numbers of interventions and content related to all other internalizing disorders, this is promising to begin the implementation of activities that are significantly supported by the current literature to have positive effects on students’ dissociative, obsessive-compulsive, trauma/stressor, and eating disorders.

Contrary to the findings from the ease of implementation of depressive- and anxiety-related interventions in this study, research regarding the transportability of CBT indicates that school-based CBT has the potential to be a highly competitive form of intervention surpassing the common barriers of implementation (Elkins et al., 2011). Since various research studies (Bekker et al., 2016; Elkins et al., 2011; Heier, 2016; James et al., 2015) have found gold-standard, evidence-based interventions such as CBT are able to be adapted for more accessible use without significantly compromising the treatment effects, this should be reflected in the content present on Pinterest.

Limitations

The present study had several limitations that could potentially be addressed in future research. This study solely examined content present on the social media platform of Pinterest. Therefore, the findings are not representative of the content shared on other social media platforms. Future research might investigate other platforms such as Twitter, Facebook, YouTube, or Teachers Pay Teachers.

Furthermore, due to nature of the Internet and Pinterest being an active form of social media, content is updated daily. Any study concerning information on the Internet must take into account the daily, even hourly, variability. Therefore, there is a necessity to take caution when interpreting the findings of studies researching Internet platforms. In the original study by Hall et al. (2018), the sample was taken in 2016, and it included 49,627 pins to get a snapshot of what the NASP account was sharing to its followers and what its followers were pinning. As a cross-sectional sample containing no identified users, this snapshot could potentially represent about a 10% picture of the pins shared by the people who had some relationship or interest in school psychology-related content. The variability of the Internet did have an impact, including the corresponding links on individual pins that are no longer actively functional. The high level of nonfunctional links speaks to the consequences of this impact on educators such as how much time is spent on these searches because a third of pins lead to an error page. Further research may provide information on this sample in the future, to speak to the number of links that are still actively working or have been manually updated and the implications and could also take a new sample of the additional pins added to the same boards to see if pinning behavior is consistent with current trends being pushed by national associations.

In addition, features of Pinterest are often updated and can interfere with the interpretation of the findings. In late-2016, when this study began, Pinterest had a separate like and repin feature. This was eventually combined into one feature. Another update to the features on Pinterest included the addition of an I Tried It button to each pin. In mid-2017, as the study was concluding, Pinterest eliminated the visual that displayed the number of times a pin had been repinned. This resulted in only limited ability to interpret network impact and was omitted from the analysis.

In the present study, a high occurrence of anxiety and depressive-related content but very low prevalence of all other internalizing disorders was found. The high prevalence of anxiety- and depressive-related content caused difficulty with chi-square analyses and resulted in categories having to be collapsed, so expected counts were more than 5 (Kroonenberg & Verbeek, 2018). Future research may consider investigating the content present when searching for terms such as internalizing disorders in addition to examining the content present on personal boards of school personnel. While the attitudes and practices of school personnel interacting with content present on Pinterest were critical for this study, it would be interesting to see what materials and resources are offered in a general search of all existing content on Pinterest. This could provide a more targeted approach to a topic found in 1.43% of the total 49,627 pins coded in the larger sample of this study, showing that internalizing disorders content was not as highly shared as externalizing disorders, coping skills, or classroom management strategies.

Future Directions

As long as school personnel continue to turn to online sources to gain information, a need will exist for well-researched information that is accessible. Further research on the topic of online dissemination of information would benefit from focus on the research regarding transportability of evidence-based interventions and using the findings to create information that can be disseminated on platforms such as Pinterest. By doing so, this would provide school personnel with the information to implement the gold standard of treatments for internalizing disorders without the common barriers of implementation. Furthermore, if national associations such as NASP continue to urge school personnel to use evidence-based practices, the information needs to be more easily accessible. As previously stated, this could be achieved by eliminating the need for paid memberships to access valuable information (e.g., position statements, resources on interventions, etc.).

Conclusion

The present study examined current content present on the social media platform of Pinterest regarding internalizing disorders. This study also investigated the likelihood of implementation of pins identified as activities for intervention. The initial hypothesis that associations will exist between the category of internalizing disorders, level of evidence base, and types of pins shared was supported with three significant associations. Findings indicated anxiety and depressive disorders dominated the sample, leaving little numbers of pins concerning the majority of other classified internalizing disorders. This is in line with previous research that states that educational professionals view anxiety and depression as more important than other internalizing disorders. Very little was offered in terms of identification of internalizing disorders, despite previous research recognizing internalizing disorders are covert in nature. Finally, more than half of the sample of pins was determined to be from sources supported by evidence, potentially due to the higher frequency of organizational websites restructuring than personal blogs by parents with children with internalizing disorders creating content based on personal experience or because of paid memberships required to access content.

The second hypothesis that there would be a main effect of the level of evidence base on the ease of implementation was partially supported. Findings indicated a significant interaction between the category of internalizing disorder and the level of evidence base on the ease of implementation of intervention activities present on Pinterest. Despite findings from research regarding the transportability of gold-standard interventions, this is not reflected on Pinterest.

Therefore, evidence-based information must be made more accessible to school personnel by considering links that will be affected if organizational sites restructure their layouts and opportunities to make paid content available to school personal, as more and more education professionals are turning to online sources to fill gaps in information and training needs. Further research on this topic would benefit from focusing on the research regarding transportability of gold-standard interventions to create information to disseminate on online platforms such as Pinterest.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Alyssa San Jose, MEd, is a doctoral candidate in school psychology at the Pennsylvania State University. Her research interests focus on school-based prevention for internalizing disorders and substance use.

Cristin Hall is an assistant professor in school psychology at the Pennsylvania State University. Her research includes technology applications for dissemination of evidence-based practice, school readiness, and the social-emotional adjustment of children in schools.

Barbara Schaefer is an associate professor in school psychology at the Pennsylvania State University. Her research interests are in educational and behavioral assessment, learning behaviors, gender and cultural differences, and instrument development/scaling.

Nicole Breeden is an assistant research professor at Rutgers the State University of New Jersey, Graduate School of Applied and Professional Psychology. Her research includes implementation science and technology-based applications for evidence-based practice.

References

Anxiety and Depression Association of America. (2016a). Children and teens. Retrieved from https://adaa.org/living-with-anxiety/children

Anxiety and Depression Association of America. (2016b). Anxiety and depression in children. Retrieved from https://adaa.org/living-with-anxiety/children/anxiety-and-depression

Bekker

M. J.

Griffiths

K. M.

Barrett

P. M.

(2016). Improving accessibility of cognitive behavioural therapy for children and adolescents: Review of evidence and future directions. Clinical Psychologist, 21(3), 1–8. doi:10.1111/cp.12099

Bergman

L. R.

El-Khouri

B. M.

(2002). Sleipner. A statistical package for pattern-oriented analyses (Version 2.1). Stockholm, Sweden: Department of Psychology, Stockholm University.

Bluhm

R. L.

Covin

Chow

Wrath

Osuch

E. A.

(2014). I just have to stick with it and it’ll work”: Experiences of adolescents and young adults with mental health concerns. Community Mental Health Journal, 50, 778–786.

Breeden

N. C.

Hall

C. M.

Giacobe

(2018). Component analysis and evidentiary basis of coping-skills content on Pinterest. Psychology in the Schools, 55(7), 783–800.

Cobham

V. E.

(2012). Do anxiety-disordered children need to come into the clinic for efficacious treatment? Journal of Consulting and Clinical Psychology, 80(3), 465–476.

Collins

K. A.

Westra

H. A.

Dozois

D. J. A.

Burns

D. D.

(2004). Gaps in accessing treatment for anxiety and depression: Challenges for the delivery of care. Clinical Psychology Review, 24, 583–616.

Di Pietro

N. C.

Whiteley

Mizgalewicz

Illes

(2013). Treatments for neurodevelopmental disorders: Evidence, advocacy, and the internet. Journal of Autism and Developmental Disorders, 43, 122–133.

10.

Eklund

Renshaw

T. L.

Dowdy

Jimerson

S. R.

Hart

S. R.

Jones

C. N.

Earhart

(2009). Early identification of behavioral and emotional problems in youth: Universal screening versus teacher-referral identification. California School Psychologist, 14, 89–95.

11.

Eldon

(2008). As Facebook hits 100 million user mark, twenty percent have already opted in to the redesign. Venture Beat. Retrieved from: http://venturebeat.com/2008/08/25/as-facebook-hits-100-million-user-mark-twenty-percent-have-already-opted-in-to-the-redesign/

12.

Elkins

R. M.

Mchugh

R. K.

Santucci

L. C.

Barlow

D. H.

(2011). Improving the transportability of CBT for internalizing disorders in children. Clinical Child Family Psychology Review, 14, 161–173.

13.

Essau

C. A.

(2005). Frequency and patterns of mental health services utilization among adolescents with anxiety and depressive disorders. Depression and Anxiety, 22, 130–137.

14.

Farmer

E. M. Z.

Burns

B. J.

Phillips

S. D.

Angold

Costello

E. J.

(2003). Pathways into and through mental health services for children and adolescents. Psychiatric Services, 54(1), 60–66. doi:10.1176/appi.ps.54.1.60

15.

Feng

Cong

Chen

(2013). An empirical study of user behaviors on pinterest social network. Proceedings – 2013 IEEE/WIC/ACM International Conference on Web Intelligence, WI 2013, 1, 402–409.

16.

Forman

S. G.

Olin

S. S.

Hoagwood

K. E.

Crowe

Saka

(2008). Evidence-based interventions in schools: Developers’ views of implementation barriers and facilitators. School Mental Health, 1, 26–36.

17.

Gavini

(2016). 2 billion monthly idea searches and counting. Oh, How Pinteresting! Retrieved from https://blog.pinterest.com/en/2-billion-monthly-idea-searches-and-counting

18.

Guidry

Zhang

Jin

Parrish

(2016). Portrayals of depression on Pinterest and why public relations practitioners should care. Public Relations Review, 42, 232–236.

19.

Granger

M. M.

Reiter

L. R.

(2015). An exploratory analysis of entrepreneurs’ social media usage in small business practices. Journal of Marketing Management, 3(2), 11–26.

20.

Hall

C. M.

Breeden

N. C.

Giacobe

(2018). I found it on Pinterest: An exploration of Pinterest content for followers of the National Association of School Psychologists. Contemporary School Psychology, 22(4), 413–423. doi:10.1007/s40688-018-0174-1

21.

Hall

C. M.

Culler

E. D.

Frank-Webb

(2016). Online dissemination of resources and services for parents of children with autism spectrum disorders (ASDs): A systematic review of evidence. Review Journal of Autism and Developmental Disorders, 3, 273–285.

22.

Heier

J. E.

(2016). Implementation support: Improving transportability of evidence-based treatments from laboratories to schools. Austin, TX: The University of Texas at Austin.

23.

Hicks

T. B.

Shahidullah

J. D.

Carlson

J. S.

Palejwala

M. H.

(2014). Nationally certified school psychologists’ use and reported barriers to using evidence-based interventions in schools: The influence of graduate program training and education. School Psychology Quarterly, 29, 469–487.

24.

In-Albon

(2012). State of research on internalizing disorders in children and adolescents: Is it still in its infancy? Verhaltenstherapie, 22, 246–257.

25.

James

A. C.

James

Cowdrey

F. A.

Soler

Choke

(2015). Cognitive behavioural therapy for anxiety disorders in children and adolescents (Review). The Cochrane Database of Systematic Reviews, 2(2), 1–108.

26.

Kroonenberg

P. M.

Verbeek

(2018). The tale of Cochran’s rule: My contingency table has so many expected values smaller than 5, what am I to do? The American Statistician, 72(2), 175–183.

27.

Langley

A. K.

Nadeem

Kataoka

S. H.

Stein

B. D.

Jaycox

L. H.

(2010). Evidence-based mental health programs in schools: Barriers and facilitators of successful implementation. School Mental Health, 2, 105–113.

28.

Maxwell

S. E.

Delaney

H. D.

(2004). Designing experiments and analyzing data: A model comparison perspective (2nd ed.). New York, NY: Psychology Press.

29.

McKevitt

B. C.

(2012). School psychologists’ knowledge and use of evidence-based, social-emotional learning interventions. Contemporary School Psychology, 16, 33–46.

30.

Merikangas

K. R.

Burstein

Swanson

S. A.

Avenevoli

Cui

Swendsen

(2010). Lifetime prevalence of mental disorders in U.S. adolescents: Results from the national comorbidity study-adolescent supplement (NCS-A). Journal of American Academic Child Adolescent Psychiatry, 49, 980–989.

31.

Miller

D. L.

(2010). Mental health practices of school psychologists. Iowa City, IA: University of Iowa. Retrieved from http://ir.uiowa.edu/etd/858

32.

Miller

D. N.

Jome

L. M.

(2008). School psychologists and the assessment of childhood internalizing disorders. School Psychology International, 29, 500–510.

33.

Miller

D. N.

Jome

L. M.

(2010). School psychologists and the secret illness: Perceived knowledge, role preferences, and training needs regarding the prevention and treatment of internalizing disorders. School Psychology International, 31, 509–520.

34.

Mittal

Gupta

Dewan

Kumaraguru

(2013). The pin-bang theory: Discovering the pinterest world. New Delhi, India: Indraprastha Institute of Information Technology.

35.

Mittal

Gupta

Dewan

Kumaraguru

(2014). Pinned it! A large scale study of the Pinterest network. New Delhi, India: Indraprastha Institute of Information Technology.

36.

National Alliance on Mental Illness. (2017). Dissociative disorders. Retrieved from https://www.nami.org/Learn-More/Mental-Health-Conditions/Dissociative-Disorders

37.

National Institute of Mental Health. (2016a). Any anxiety disorder among children. Retrieved from https://www.nimh.nih.gov/health/statistics/prevalence/any-anxiety-disorder-among-children.shtml

38.

National Institute of Mental Health. (2016b). Eating disorders among children. Retrieved from https://www.nimh.nih.gov/health/statistics/prevalence/eating-disorders-among-children.shtml

39.

National Institute of Mental Health. (2016c). Major depression among adolescents. Retrieved from https://www.nimh.nih.gov/health/statistics/prevalence/major-depression-among-adolescents.shtml

40.

National Institute of Mental Health. (2016d). Obsessive compulsive disorder among adults. Retrieved from https://www.nimh.nih.gov/health/statistics/prevalence/obsessive-compulsive-disorder-among-adults.shtml

41.

National Institute of Mental Health. (2016e). Post-traumatic stress disorder among children. Retrieved from https://www.nimh.nih.gov/health/statistics/prevalence/post-traumatic-stress-disorder-among-children.shtml

42.

Perrin

(2015). Social networking usage: 2005-2015. Washington, DC: Pew Research Center. Retrieved from www.pewresearch.org

43.

Pham

(2014). Navigating social networking and social media in school psychology: Ethical and professional considerations in training programs. Psychology in the Schools, 51, 767–778.

44.

Regier

D. A.

Kuhl

E. A.

Kupfer

D. J.

(2013). The DSM-5: Classification and criteria changes. World Psychiatry, 12, 92–98.

45.

Reichow

Gelbar

N. W.

Mouradjian

Shefcyk

Smith

I. C.

(2014). Characteristics of international websites with information on developmental disabilities. Research in Developmental Disabilities, 35, 2293–2298.

46.

Rivest

(1992). The MD5 message-digest algorithm (RFC 1321, Informational). Fremont, CA: Internet Engineering Task Force. Retrieved from http://www.ietf.org/rfc/rfc1321.txt

47.

Seaman

Tinti-Kane

(2013). Social media for teaching and learning. Boston, MA: Pearson Learning Solutions. Retrieved from http://www.onlinelearningsurvey.com./

48.

Segool

N. K.

Goforth

A. N.

Bowman

Pham

(2016). Social networking practices in school psychology: Have moral panic concerns been overstated? Journal of Applied School Psychology, 32, 66–81.

49.

Tandon

Cardeli

Luby

(2009). Internalizing disorders in early childhood: A review of depressive and anxiety disorders. Child and Adolescent Psychiatric Clinics of North America, 18, 593–610.

50.

Taylor

(2015). Flipping great or flipping useless? A review of the flipped classroom experiment at Coventry University London Campus. Journal of Pedagogic Development, 5(3), 57–65.

51.

Urbaniak

G. C.

Plous

(2013). Research randomizer (Version 4.0) [Computer software]. Retrieved from http://www.randomizer.org/

52.

Wang

Yang

Zheng

Sundar

S. S.

(2016). Why do we pin? New gratifications explain unique activities in Pinterest. Social Media and Society, 2(3), 1–9. doi:10.1177/2056305116662173

53.

Wei

Zhang

(2016). Understanding user behavior on Pinterest: Findings and insights. Proceedings of the Conference on Information Systems Applied Research, 9, 1–11.

54.

Whitney

S. D.

Renner

L. M.

Pate

C. M.

Jacobs

K. A.

(2011). Children and youth services review principals’ perceptions of benefits and barriers to school-based suicide prevention programs. Children and Youth Services Review, 33, 869–877. doi:10.1016/j.childyouth.2010.12.015

55.

Yeung

(2015). Pinterest finally shares its size: 100M monthly active users and counting. Venture Beat. Retrieved from http://venturebeat.com/2015/09/16/pinterest-finally-shares-its-size-100m-monthly-active-users-and-counting