Helping Youth Navigate Privacy Protection: Developing and Testing the Children's Online Privacy Scale

Importance of Protecting Children's Personal Information Online

Many individuals struggle to protect their personal information online and often surrender to the convenience of technology, facing potential harm and uncertain long-term risk (Walker 2016). As noted in the Children's Online Privacy Protection Act (COPPA; 1998), the term “personal information” (also referred to as “personally identifiable information” or PII), means “individually identifiable information about an individual collected online.” This includes a first and last name; a home or other physical address including street name and name of a city or town; an email address; a telephone number; a social security number; any other identifier that permits the physical or online contacting of a specific individual, as determined by the Federal Trade Commission (FTC); or information concerning the child and/or the parents of that child that discloses the child's identity.

Although young children are found to be more tech-savvy than their parents (60 Minutes 2022; SWNS 2020), they still take risks online (White, Gummerum, and Hanoch 2015) and are often dependent on others for protection of their personal information (i.e., they are a vulnerable population). Thus, children's online privacy is an important issue for parents, consumer groups, privacy researchers, and companies as well as federal (FTC 2019a, b), state, and international agencies. As an example, the increasing “datafication” of children, or an organization's collection of information about children as they interact online, may shape their future opportunities as the data are used for education, career, insurance, and other screening purposes (Lupton and Williamson 2021).

Children's ability to consent to the exchange of information is a growing issue for regulation and policy (Livingstone, Stoilova, and Nandagiri 2019). This is demonstrated by attempts to update the COPPA and to expand protection and limit access to children's personal information (e.g., Protecting the Information of our Vulnerable Children and Youth Act, H.R. 4801 [2021]). Newly proposed legislation to protect children online involves extending protection to age 16 from age 12 and a ban on advertising without consent (e.g., Children and Teens’ Online Privacy Protection Act, S. 1628 [2022]). It also imposes responsibility for online platforms to act in the best interest of minors, as well as equipping parents and minors under age 16 with more safeguards (e.g., Kids Online Safety Act, S. 3663 [2022]).

In recent years, children’s and young teens’ online use has reached record highs (American Academy of Child & Adolescent Psychiatry 2020; Anderson and Jiang 2018). Although parents often claim they check what their teens and children do online (Anderson 2016), reviews of the most popular video websites (e.g., YouTube) indicate that child protection mechanisms are breaking down (Wendling 2017). Research shows that over half of parents with children between ages 5 and 11 indicate that their child uses a smartphone (Auxier et al. 2020), and the ability of parents and children to understand the complexity of the digital environment is becoming increasingly challenging (Livingstone, Stoilova, and Nandagiri 2019). Even devices intended to help parents monitor their children's online activities (e.g., smartwatches) are found to be a “privacy and security nightmare” (Greenberg 2020) and child/teen “workarounds” to get around protections are common (60 Minutes 2022). Some parents are even actively involved with public online oversharing of their children’s private content (Fox and Hoy 2019). Research also shows potential privacy issues with children's apps, noting that there may be a “financial incentive to ignore privacy violations” in these apps and that “limiting (data) collection efforts” may reduce revenue (Reyes et al. 2018, p. 77).

Other examples abound. YouTube's volunteer “Trusted Flaggers” report that for the 526 complaints made to YouTube's abuse page, only 15 responses were received from the service (Wendling 2017). The reports were made primarily against accounts that left objectionable comments (often sexually explicit) on videos made by young teens or children. The FTC and the New York attorney general levied a $170 million fine on YouTube for collecting personal information from children without their parents’ consent (FTC 2019a). That same year, the FTC fined Musical.ly (TikTok) $5.7 million for violating both COPPA and FTC rules by collecting PII from children without consent (FTC 2019c).

Musical.ly (TikTok) recently agreed to pay $92 million for collecting personal data without consent, mostly from minors, with some as young as age six (Allyn 2021). The data collection included the sharing of personal information with third parties, some of which were based in other countries. Other investigations have revealed how Musical.ly (TikTok) (Wall Street Journal 2021) and Meta (aka Facebook/Instagram) (Horowitz and Wells 2021) have used their algorithms to serve increasingly harmful content to children. One study showed that 20% of videos watched on YouTube by children age eight and younger contained ads deemed inappropriate for that age (i.e., violence, sex, or drugs/alcohol; Radesky et al. 2020). Live streaming platforms and apps with children's content (e.g., YouTube Gaming), which allow interactions between users, also face compliance issues with regard to the collection of personal information and potential COPPA violations. Since 2000, the FTC has filed almost 30 cases involving violations of COPPA (FTC 2020b).

Adding to the problem is the lack of children’s engagement in safe privacy behaviors. Only 61% of 10-to-18-year-olds have enabled privacy settings on their social network profiles; 52% do not turn off their location or GPS across apps, leaving their locations visible to strangers; and 14% have posted their home addresses online, a 27% increase from the previous year's results (McAfee 2014). Even with the FTC's Children's Online Privacy Protection Rule (2000), a study of online children's apps showed that more than 50% failed to protect data (Egelman 2017). The evolving technical aspects of online privacy and data protection complicate the practices of institutions and online service providers and attempts to update privacy laws protecting children. This continuous evolution also exacerbates potential privacy threats or risks, makes consistent strategy for individual privacy control challenging, and complicates efforts to educate children about privacy.

Online Literacy and Privacy Knowledge Among Children and Young Teens

Online literacy refers to “a variety of skills associated with using information and communication technologies to find, evaluate, create, and communicate information” (Clark and Visser 2011, p. 39). The focus of our research is on the skills and information aspects of this definition with respect to objective privacy knowledge/literacy of children and young teens. What children and young teens objectively know about online privacy may affect their sharing of information online and the associated risks of doing so (Walker 2016). For example, studies show that when children as young as age seven know the actual risks of sharing personal information online (objective knowledge), they become more diligent on websites and are more likely to deny requests for personal information or check with parents first before sharing information (Kennedy, Baig, and Chiasson 2017). Others also note that children's online objective knowledge plays an important role in understanding, managing, and safeguarding their privacy (Stoilova, Nandagiri, and Livingstone 2021). This includes placing restrictions on their information by using privacy settings, creating different email accounts, viewing online content with private mode(s), and so forth. Work by Brough and Kelly (2020) further emphasizes the importance of accounting for such factual/practical knowledge in assessing privacy actions.

From previous privacy literacy research (e.g., Blank, Bolsover, and Dubois 2014; Trepte et al. 2015), we conceptualize objective knowledge of online privacy as what children and young teens age 6 to 15 actually know across five privacy content domains identified by public policy and academic scholars (e.g., Children’s Online Privacy Protection Rule 2000; Trepte et al. 2015): (1) knowledge about practices of institutions and online service providers, (2) knowledge about technical aspects of online privacy and data protection, (3) knowledge about potential privacy threats and risks, (4) knowledge about basic laws/legal aspects of data protection in the United States, and (5) knowledge about strategies for individual privacy control. Our research develops and tests the COPS, which taps these five verifiable content domains.

Validity-Related Hypotheses

We conceptualize COPS in a broader nomological framework that offers testable hypotheses to establish the scale's validity (see Figure 1). We predict that (1) COPS scores are affected by cognitive defense conditions; (2) child and young teen age groups will show predicted differences in COPS scores; (3) COPS scores will be predictive of children’s and young teens’ intent to share personal information online and online privacy behaviors; and (4) COPS scores will offset (moderate) the effect of key personality traits and social media usage on intent to share personal information online. We expand on these key hypotheses (H₁ through H₄) as the article progresses.

OPEN IN VIEWER

Figure 1.

Youth Objective Online Privacy Knowledge via COPS: Effects on Intent and Behaviors.

Cognitive Defense Conditions and COPS

We predict that objective privacy knowledge/literacy (COPS score) will be strengthened via cognitive defense conditions (i.e., quiz with feedback and educational video vs. a control nonexposure group; Figure 1). In comparison with those not exposed to a defense condition, children and teens viewing an online privacy quiz with feedback and an educational video about online safety (identical in content with the quiz) showed higher scores on agreement with online safety beliefs and less willingness to share a YouTube video watched (Andrews, Walker, and Kees 2020). This suggests that carefully developed online quizzes and educational videos providing important objective privacy safety information should be reflected in higher COPS scores. Thus, we predict that objective knowledge/literacy with regard to online privacy measured via the COPS will be stronger for those exposed to such educational efforts. Experimental support for such a prediction would offer causal evidence for the “known-group validity” of the COPS. In this type of validity, the understanding of a construct leads us to expect groups to differ with respect to mean scores on the construct's measure based on a theoretical premise (Cronbach and Meehl 1955), in this case educational interventions. We test this hypothesis in Study 1.

H₁: Mean COPS scores in the cognitive defense conditions (e.g., educational video, quiz with feedback) will be higher than the mean COPS score in a control condition.

COPS and Child or Young Teen Age Groups

According to child cognitive development and information processing research, older children are likely to possess greater online privacy knowledge than younger children. Research suggests three stages of cognitive development for children to separate central and incidental material: limited (under age 8), cued (age 8–12), and strategic (over age 12) (Gregan-Paxton and Roedder John 1997; Roedder 1981). Brucks, Armstrong, and Goldberg (1988) suggest that children must reach the strategic processing stage (age 13) before they can generate spontaneous cognitive responses, whereas children age 8–12 may need a defense strategy cue or prompt to focus their attention and counterargue information that may be harmful. Andrews, Walker, and Kees (2020) show that older children/teens (age 13–15) had stronger online safety beliefs than other age groups. Thus, as further evidence of known-group validity, we predict that age groups will show mean-level differences in COPS scores, with the expectation that the older groups will exhibit higher COPS scores than the youngest group. We test this prediction in Studies 1 through 4.

H₂: The older age groups (8–12 and 13–15 years) will show higher mean COPS scores than the youngest age group (6–7 years).

COPS, Personality Traits, and Social Media Usage

In addition to a COPS → intent to share personal information online effect, we examine the COPS's ability to moderate the effect of three key antecedents of intent to share private information online: the personality traits of (1) needing to be popular and (2) thrill seeking; and (3) social media usage. First, in support of the predictive validity of the COPS, the greater the objective knowledge in a given domain, the more one is likely to perform positive within-domain behaviors or avoid negative ones in that domain (Alba and Hutchinson 2000). This premise has found support in several application areas, including nutrition (Andrews, Netemeyer, and Burton 2009), physical health (Waters et al. 2018), and personal finance (Ward and Lynch 2019). Thus, we expect COPS scores to be negatively related to intent to share private information online (tested in Studies 1 through 4). We also predict that COPS scores will be positively related to an index of “safe” online privacy behaviors (tested in Study 4).

H₃: COPS scores will be (a) negatively related to intent to share personal information online and (b) positively related to an index of “safe” online privacy behaviors.

An important element of construct validity is external validity, which includes the degree to which the construct may interact with key background factors (e.g., values, personality traits, behaviors) in affecting an outcome (Lynch 1999). Through an examination of interaction effects, the external validity of the COPS is enhanced by a more thorough understanding of how and when COPS scores might interact with such background factors operationalized as moderator variables (Cook and Campbell 1979). We expand on the reasoning for these moderated predictions in Studies 2 through 4.

H₄: COPS scores will moderate (offset) the effects of children’s/young teens’ (a) need to be popular, (b) thrill-seeking tendencies, and (c) social media usage on intent to share privacy information online.

COPS Item Generation and Initial Filtering: Pretest 1

We first generated an initial pool of items to tap the five content domains of children's online privacy knowledge in Pretest 1. All items were adapted or written for children/young teens age 6–15, and average grade-level readability scores were assessed (e.g., Flesch-Kincaid, Lexile: fourth grade). The items were based on our literature review, several sources of external information on children's online privacy (e.g., FTC 2020a), and an open-ended online survey of children for their thoughts regarding their online behavior and beliefs toward internet privacy. For the open-ended survey, we screened for child presence, age, parental or guardian consent, child/young teen assent, online usage, and reading ability. We surveyed 100 children/teens in Pretest 1, equally split across gender and the three age categories (6–7 years; 8–12 years; 13–15 years). The initial pool consisted of 50 online personal information privacy items, with 10 items for each of the five content domains identified.

Kantar Lightspeed, a vendor specializing in collecting research data with children, conducted the study with its online panel of parents with children and teens. Two trained independent coders matched the responses with thought categories to provide intercoder reliability (Ir) indices for each question. Following screener and consent questions, children were asked: “What are some of your favorite things that you do online?” Top response categories (in order) included “play video games,” “watch videos,” “social media,” “chat with friends,” and “listen to music” (agreement = .86, Ir = .85). Specific social media sites and apps visited included YouTube (general and YouTube Kids), Facebook, Instagram, Snapchat, TikTok, and game sites.

Sample questions then tapped the five content domains of children's online privacy objective knowledge/literacy. For knowledge about company privacy practices (Content Domain 1) and technical aspects of privacy (Content Domain 2), we asked: “What types of information do you think companies collect from children and teens?” Top response categories included age, address/location, likes/interests, online activity, names, and other personal information (agreement = .963, Ir = .962). To also assess these content domains, we asked: “What do you think companies do with the information they collect from children and teens?” Leading response categories included “to help sell/make new products,” “create targeted ads,” “sell the information,” “track you/your activity,” and “for rewards/gifts” (agreement = .966, Ir = .965). For Content Domain 3 (privacy threats/risks), we asked: “Are there any risks in providing personal information online?” The top response categories were “getting stalked/harassed,” “access to personal information,” “identity theft,” “getting hacked,” “bad people can find you,” and “too many ads” (agreement = .901, Ir = .896).

Content Domain 4 addressed basic knowledge of any privacy laws or regulations in the United States. The major response categories mentioned were “none/not sure,” “specific information collected,” and “yes—there are laws” (agreement = 1.00, Ir = 1.00), indicating a need for greater literacy in this domain (e.g., COPPA), especially for teens. Finally, Content Domain 5 addressed strategies for privacy control with the question “If you try to keep your information private while online, explain how you do this” (87% of respondents answered this question by first saying that they try to keep their personal information private). The leading response categories for privacy control were “not sharing personal information,” “changing privacy settings,” “using fake information/fake names,” “parental controls,” and “not talking to strangers” (agreement = .883, Ir = .875).

In general, children and young teens strongly agreed on a seven-point scale (1 = “not very important at all,” and 7 = “extremely important”) that it was important to keep their personal information private online (M = 6.01, SD = 1.28). Also, an attention check question asked respondents if they saw a picture of a nonrelevant character in any of the information presented, and 91% correctly indicated that they did not. At least 60% of the children/teens participants spent between 7 and 14 hours a week online. Finally, 75% of the children indicated that their parents restricted what they watched online. The 50 items were then reviewed by expert judges to further reduce the pool of items for the empirical studies that followed.

Ratings, Trimming of Items, and Content Validity: Pretest 2

To reduce the initial pool of 50 items to approximately 20 (i.e., four items for each of the five content domains) in Pretest 2 and help establish the items’ content validity in their respective domains, we asked 12 experts on children’s and teens’ online/social media habits and privacy issues to rate the items as “very representative,” “representative,” or “not representative” of each online privacy domain. The experts included five people with PhDs in marketing or information technology, one writing expert and nonprofit educator, and six young adults with online/social media expertise. Four nonredundant items with the top-rated scores in each of the five domains were retained for further testing in Studies 1 and 2. All items were multiple-choice questions with a “don’t know” option. In all analyses that follow, correct answers were scored as 1, and incorrect and “don’t know” answers were scored as 0.

Finalizing the COPS Items: Factor Structure and Reliability via Studies 1 and 2

Procedures, Measures, and Predictions

Recall that the sample for Study 1 consisted of 150 participants (80 female, 70 male); 46 were age 6–7, 53 were age 8–12, and 51 were age 13–15. In addition to developing the COPS, another major purpose of Study 1 was to initially test the construct's role in the conceptual framework shown in Figure 1, and thus its validity. In this regard, Study 1 was actually set up as an experiment in which children or young teens were randomly assigned to one of three “cognitive defense” conditions: an online privacy quiz with feedback (n = 54), an online privacy video (“Be a Smart Cookie”; n = 50), or a control condition without a quiz or video (n = 46). As the quiz and video have been shown to be successful in helping children and teens build defenses in restricting their sharing of personal information online (Andrews, Walker, and Kees 2020; Desimpelaere, Hudders, and Van de Sompel 2020), H₁ predicted there would be a significant difference in the mean COPS scores of those exposed to these educational defenses and those in the control condition. The quiz and video contained identical information on privacy protection with the difference being the delivery of the content.

Study 1 also assessed other measures for validity purposes. We assessed a single, seven-point item measure of intent to share personal information online as the primary dependent variable. As control variables, we also gathered a single-item measure of child/young teen perception of their parents restricting their online behavior (“yes” or “no”); the demographic variables of age group, gender, and race; and a five-item measure of prior (initial) online privacy beliefs. The rationale for these measures is as follows.

Both theory and empirical research suggest that parental influence may have an effect on a child's intentions and behavior, particularly with respect to online activities (Livingstone and Smith 2014; Stoilova, Nandagiri, and Livingstone 2021), and our item generation study revealed that 75% of children/teens interviewed stated that their parents restricted their online activities. Thus, we use perception of parents restricting online behavior as a control variable in predicting intent. Likewise, the demographics of child/teen gender, age, and race have been shown to be related to risky behaviors in general, and specifically online behaviors and intent (Helsper and Smahel 2020; Livingstone and Smith 2014). Thus, as a more rigorous test of the validity of COPS to predict intent to share privacy information online (H_3a), we control for demographics and for parents restricting their child’s/teen's online viewing.

Prior beliefs have been shown to be a significant predictor of behavioral intentions in numerous social and consumer behavior contexts (Ajzen 1991; Sheppard, Hartwick, and Warshaw 1988), but such beliefs may be biased in that they are not always predictive of intent and behaviors, including those for privacy (Trepte et al. 2015), and particularly for children and young teens (Blank, Bolsover, and Dubois 2014). Children and young teens have been shown to meander from simple inferencing and initial beliefs due to a lack of actual knowledge and experiences (Bem 1972; Bonawitz 2010). Thus, our focus is on the ability of the COPS to explain incremental variance in intent after accounting for the effect of prior beliefs as a control variable. Table W3 of Web Appendix B shows summary statistics and correlations among Study 1 variables.

Data Checks

The survey also included a number of data checks geared at ensuring that the child and young teen respondents completed the survey free of parental involvement. At the beginning of the survey, we implemented screening questions on child presence, ability to read, online use, child assent, and parent and child age, and we gave instructions for parents to leave the room once parental consent was granted. In the middle of the survey, we also asked the child respondents for their age and matched each child’s response to the parent's response of the child's age. All of these checks were affirmed, giving us confidence that the child (rather than a parent/adult) completed the survey. Web Appendix A shows these data check questions. Further, the mean and median amounts of time to complete the survey were 14.78 and 11.47 minutes. These mean times did not differ by age group (age 6–7 years, M = 16.30 minutes; age 8–12 years, M = 14.25 minutes; age 13–15 years, M = 13.72 minutes; F(2, 148)= .60, p = .55).

Analyses and Results

Intent to share information online

We used hierarchical regression analyses to examine ability of the COPS to predict children's intent to share personal information online, controlling for demographics, parents’ restriction of online behavior, and initial online privacy beliefs. Given the preceding results, we also controlled for the cognitive defense versus control manipulation by coding it as 0 or 1. The second and third columns of Table 2 show the results.

OPEN IN VIEWER

Table 2.

Study 1 and Study 2 Intent to Share Regression Results.

	Study 1 Intent to Share Model 1	Study 1 Intent to Share Model 2	Study 2 Intent to Share Model 1	Study 2 Intent to Share Model 2	Study 2 Intent to Share Model 3
Control Variables
Gender (1 = male, 0 = female)	1.066 (.248)***	.874 (.203)**	−.136 (−.076)	−.092 (−.051)	−.105 (−.058)
Race (1 = White, 0 = other)	.011 (.002)	.022 (.004)	.027 (.013)	−.053 (−.026)	−.038 (−.019)
Age 6 to 7	.431 (.093)	−.021 (−.005)	.220 (.106)	.017 (.008)	.044 (.022)
Age 8 to 12	−.262 (−.058)	−.375 (−.084)	.027 (.015)	−.009 (−.005)	.017 (.008)
Parents restrict online behavior (1 = yes, 0 = no)	−.327 (−.064)	−.442 (−.086)	−.166 (−.080)	−.202(−.097)*	−.195 (−.094)*
Cognitive defense (1 = present; 0 = absent)	−.339 (−.073)	.039 (.008)	—	—	—
Prior beliefs	−.300 (−.150)*	−.139 (−.070)	−.152 (−.126)**	−.006 (−.005)	−.032 (−.019)
Subjective knowledge	—	—	−.149 (−.126)**	−.092 (−.077)	−.104 (−.088)*
Need to be popular	—	—	.226 (.251)***	.168 (.186)***	.180 (.200)***
Hypothesized Predictors
COPS	—	−.264 (−.488)***	—	−.093 (−.399)***	−.089 (−.381)***
COPS × need to be popular	—	—	—	—	−.031 (−.128)**
R-square	.108	.317	.132	.248	.264

*p < .10, **p < .05, ***p < .01.

Notes: The 13-to-15-year-old age category was used as the reference category for age. Although we assessed several categories of race, the samples were predominantly White. Thus, we coded race as 1 = White and 0 = other. In Study 1, 74% classified themselves as White, 9.3% as African American (non-Hispanic), 8% as Hispanic, 4% as Asian American, 1.3% as Native American, and 3.3% as other. In Study 2, 73.2% classified themselves as White, 5.7% as African American (non-Hispanic), 8.4% as Hispanic, 7.7% as Asian American, .3% as Native American, .3% as Pacific Islander, and 2.7% as other; 1.7% refused to answer. Values are unstandardized regression coefficients, with standardized regression coefficients in parentheses.

Model 1 (F(7, 142) = 2.448, p = .021, R² = .108) in Table 3 with just the control variables (covariates) shows that male participants are more likely than female participants to share their personal information online (β = 1.066, t = 3.048, p = .003). Initial beliefs showed a marginally significant effect (β = −.300, t = 1.814, p = .072). Model 2 (F_change(1, 141) = 43.231, p < .001, R² = .317) shows that COPS score was significantly related to intent to share (β = −.264, t = 6.575, p < .001): an increase of one correct answer on the COPS scale was associated with a .264 decrease on the seven-point scale of intent to share personal information online. This finding supports H_3a.

OPEN IN VIEWER

Table 3.

Study 3 and Study 4 Intent to Share Information Online: Regression Results.

	Study 3 Intent to Share Model 1	Study 3 Intent to Share Model 2	Study 3 Intent to Share Model 3	Study 4 Intent to Share Model 1	Study 4 Intent to Share Model 2	Study 4 Intent to Share Model 3
Control Variables
Gender (1 = male, 0 = female)	.049 (.026)	−.005 (−.003)	.010 (.005)	−.150 (−.083)**	−.147 (−.081)**	−.146 (−.081)**
Race (1 = White, 0 = other)	−.242 (−.118)**	−.197 (−.096)**	−.196 (−.095)**	.071 (.036)	.078 (.040)	.078 (.040)
Age 6 to 7	.225 (.110)*	.028 (.013)	.038 (.018)	.072 (.036)	.018 (.009)	.025 (.012)
Age 8 to 12	.070 (.073)	.019 (.019)	.007 (.007)	.059 (.032)	.018 (.010)	.003 (.002)
Parents restrict online behavior (1 = yes, 0 = no)	−.071 (−.032)	−.057 (−.026)	−.087 (−.039)	−.180 (−.086)**	−.188 (−.090)**	−.183 (−.088)**
Prior beliefs	−.295 (−.201)***	−.079 (−054)	−.126 (−.086)	−.130 (−.102)**	−.072 (−.057)	−.079 (−.063)
Subjective knowledge	.014 (.010)	.081 (.058)	.070 (.051)	−.040 (−.031)	−.023 (−.018)	−.025 (−.01)9
Social desirability bias	—	—	—	−.052 (−.042)	−.043 (−.035)	−.048 (−.039)
Thrill seeking	.318 (.311)***	.220 (.215)***	.194 (.190)***	.198 (.202)***	.189 (.193)***	.172 (.176)***
Social media use	.034 (.209)***	.031 (190)***	.026 (.160)***	.146 (.205)***	.138 (.194)***	.124 (.174)***
Need to be popular	—	—	—	.201 (.239)***	.188 (.232)***	.200 (.238)***
Hypothesized Predictors
COPS	—	−.103 (−.425)***	−.095 (−.393)***	—	−.034 (−.140)***	−.033 (−.134)***
COPS × thrill seeking	—	—	−.031 (−.129)**	—	—	−.006 (−.025)
COPS × social media use	—	—	−.004 (−.118)**	—	—	−.019 (−.110)**
COPS × need to be popular	—		—	—	—	−.007 (−.029)
R-square	.240	.368	.404	.264	278	.294

*p < .10, **p < .05, ***p < .01.

Notes: The 13-to-15-year age category was used as the reference category for age. Race was again coded as 1 = White and 0 = other. In Study 3, 71% classified themselves as White, 6.4% as African American (non-Hispanic), 9.7% as Hispanic, 9% as Asian American, .3% as Native American, and 2.7% as other; .7% refused to answer. In Study 4, 68.8% classified themselves as White, 8.4% as African American (non-Hispanic), 11.2% as Hispanic, 6.5% as Asian American, .5% as Native American, .3% as Pacific Islander, and 3.4% as other; .7% refused to answer. Values are unstandardized regression coefficients, with standardized regression coefficients in parentheses.

Procedures, Measures, and Predictions

Participants in Study 2 were 300 children or young teens (different from the participants in the pretests and Study 1) recruited by a professional online survey company (Prodege). We had four nonresponses across several measures, giving us an effective sample of 296 participants (157 female, 139 male); 76 were age 6–7 years, 107 were age 8–12 years, and 113 were age 13–15 years.

Participants completed an online survey containing the 20 multiple-choice COPS items, which were further reduced to 15 items for all subsequent analyses of Study 2 on the basis of our prior results. The same data checks of Study 1 were used in Study 2, and all were affirmed, lending confidence that children completed the survey free of parental involvement. The mean and median amounts of time to complete the Study 2 survey were 12.77 and 10.18 minutes, respectively. These mean times did not differ by age group (age 6–7 years, M = 12.76 minutes; age 8–12 years, M = 13.48 minutes; and age 13–15 years, M = 11.94 minutes; F(2, 293) = .886, p = .414). The survey included three five-point items summed and averaged to measure intent to share personal information online as the primary dependent variable. The survey also included the same measure of prior online privacy beliefs; assessment of child age, race, and gender; and a question asking children if their parents/guardians restrict what they watch online. With these variables, we sought to replicate the results of Study 1.

To offer further evidence of the validity of the COPS, Study 2 included two more measures: a three-item measure of child/young teen subjective knowledge of staying safe online and a four-item measure of need to be popular among friends/peers adapted for children and adolescents (Netemeyer et al. 2015). Recent evidence suggests that child/teen subjective knowledge of online safety predicts online safety attitudes (Macauley et al. 2019), and the “feeling of knowing” endemic to subjective knowledge provides confidence for individuals to act in a positive manner, particularly in matters of youth use of online platforms (Helsper and Smahel 2020). Thus, to more accurately assess the COPS → intent relationship, we first control for a subjective knowledge → intent relationship.

As for need to be popular, it is well accepted that childhood and adolescence are times of an increased need to “fit in” and be popular (Fitzsimons and Moore 2008; Martin et al. 2013). Children or young teens who exhibit a high need to be popular show higher levels of engaging in maladaptive behaviors, ranging from smoking and alcohol use to abuse of prescription opioids (Ford 2008; Netemeyer et al. 2015). A psychological mechanism that has been advanced as affecting these behaviors is a lower risk perception of harm because the behaviors have become normative or viewed as “cool and popular” with peers (Heirman, Walrave, and Ponnet 2013). In fact, the Youth Justice Board (2016) reports that a higher use of social media among children and adolescents is related to being viewed as popular. Thus, we expect that a need to be popular will be positively related to intent to share information online.

Important to the COPS, though, is the degree to which COPS scores may offset the effects of a need to be popular, which would lend confidence to the external validity of the COPS (Lynch 1999). As previously noted, an important element of construct validity is external validity, which includes the degree to which the construct may interact with key background factors (e.g., values, personality traits, behaviors, etc.) in affecting an outcome (Cook and Campbell 1979).

In general, high levels of knowledge have shown the ability to counteract personality traits leading to negative outcomes. In this respect, high COPS scores may have the ability to moderate the effect of need to be popular on intent to share by acting as a defense mechanism increasing one's sense of volition to resist the urges of a need to be popular. This premise is consistent with persuasion theory and Self-Determination Theory (Friestad and Wright 1994; Ryan and Deci 2000), in that high COPS scores may empower children/teens by acting as a self-regulation mechanism to avoid sharing personal information online (Youn and Shin 2020). Thus, we predict that a high score on the COPS will moderate the effect of need to be popular on intent to share (H_4a). Web Appendix A shows all measures, and Table W4 of Web Appendix B shows summary statistics and correlations among measures.

Analyses and Results

Intent to share information online

We first mean-centered the COPS and the need to be popular to create the COPS × need to be popular interaction term (Cohen et al. 2003). The last three columns of Table 2 show the hierarchical regression results for intent to share personal information online. Model 1 with control variables only (F(8, 288) = 5.475, p < .001, R² = .132) shows the expected effects of prior beliefs (β = −.152, t = 2.191, p = .029), subjective knowledge (β = −.149, t = 2.243, p = .026), and need to be popular (β = .226, t = 4.507, p < .001). Model 2 (F_change(1, 287) = 44.438, p < .001, R² = .248) shows the predicted effect of COPS (β = −.093, t = 6.666, p < .001). Note that the effect of prior beliefs becomes nonsignificant when COPS is included in the model. Model 3 (F_change(1, 286) = 6.121, p < .001, R² = .264) adds the COPS × need to be popular interaction. It was significant (β = −.031, t = 2.474, p < .014). These results support our predictions with respect to the linear and moderating effects of COPS (H₃ and H_4a). Figure 2 plots the COPS × need to be popular interaction.

OPEN IN VIEWER

Figure 2.

Study 2: Interaction Effect of COPS × Need to Be Popular on Intent to Share.

Figure 2 shows that when need to be popular is low (one standard deviation below its mean) and the COPS score is high (one standard deviation above its mean), we see the lowest intent to share personal information online (M = 1.588); when the COPS score is high and need to be popular is high, we see a mean of 1.710 for intent to share. These two means did not differ. Low COPS scores produced higher intent to share when the need to be popular is high (M = 2.631) or low (M = 2.026). Both combinations of high COPS scores and need to be popular had lower means for intent to share than the combinations of low COPS scores with need to be popular (p = .014). In sum, a high COPS score offsets the effect of need to be popular on a child's intent to share personal information online.

Overview

Study 3 further validates the COPS by examining its ability to moderate one more perceptual variable known to affect child/teen behavior, namely thrill-seeking tendencies, and a behavioral measure shown to affect online sharing of information, namely social media usage.

Sample and Measures

The initial sample of Study 3 consisted of 300 children/young teens (different from those of all other samples). We again had a few nonresponses across several measures, giving us an effective sample of 296 participants (154 female, 142 male); 86 were age 6–7 years, 108 were age 8–12 years, and 102 were age 13–15 years. The same data vendor (Prodege) of Study 2 was used for Study 3. The same data checks of the previous studies were used in the Study 3 survey; all were affirmed. The mean and median amounts of time to complete the survey were 12.23 and 10.63 minutes. The mean times did not differ by age group (age 6–7 years, M = 11.82 minutes; age 8–12 years, M = 13.16 minutes; and age 13–15 years, M = 11.48 minutes; F(2, 293) = 1.87 p = .156).

Participants competed an online survey containing the final 15-item COPS (coeff. α = .864), the same dependent variable measure of intent to share personal information online, and the same measures of prior online beliefs, subjective knowledge, parents restricting online behavior, and age, gender, and race as used in Study 2. We included a five-item measure of thrill-seeking tendencies (Netemeyer et al. 2015), adapted for children and adolescents from the Mehrabian and Russell (1974) “excitement from risk” factor of the Arousal Seeking Tendency Scale. To assess social media usage, we included a self-reported measure of the average number of hours spent on TikTok, Facebook, Snapchat, Instagram, YouTube, and Twitter. Web Appendix A shows all measures. Table W5 of Web Appendix B shows summary statistics and correlations.

Intent to share information online

Table 3 shows the results for intent to share information online. Model 1 (F(9, 285) = 10.003, p < .01, R² = .240) with just the control variables shows significant effects for race (β = −.242, t = 2.222, p = .027), prior beliefs (β = −.295, t = 3.750, p < .001), thrill seeking (β = .318, t = 5.705, p < .001), and social media usage (β = .034, t = 3.694, p < .001) and a marginally significant effect for the 6-to-7-year age group (β = .225, t = 1.683, p = .093). Model 2 (F_change(1, 284) = 57.313, p < .01, R² = .368) shows that COPS was significant (β = −.103, t = 7.571, p < .001), supporting H_3a. Prior beliefs become nonsignificant in Model 2 when COPS is included as a predictor. Model 3 (F_change(2, 282) = 8.664, p < .01, R² = .404) adds the interaction terms and shows that the COPS × thrill seeking (β = −.031, t = 2.535, p = .012) and the COPS × social media usage interactions (β = −.004, t = 2.310, p = .022) were significant, supporting H_4b and H_4c.

Figures 3 and 4 plot these interactions. Figure 3 shows that when the COPS score is high, it lowers intent to share when thrill seeking is high (M = 1.368) or low (M = 1.231). Both of these points were significantly lower (p = .012) than the low COPS score–high thrill seeking mean (M = 2.319), but not significantly lower than the low COPS score–low thrill seeking mean (M = 1.754).

OPEN IN VIEWER

Figure 3.

Study 3: Interaction Effect of COPS × Thrill Seeking on Intent to Share.

OPEN IN VIEWER

Figure 4.

Study 3: Interaction Effect of COPS × Social Media Usage on Intent to Share.

Figure 4 indicates that when the COPS score is high, it lowers intent to share when social media use is high (M = 1.368) or low (M = 1.239). Both of these points were significantly lower (p = .022) than the low COPS score–high social media usage mean (M = 2.272), but not significantly lower than the low COPS score–low social media usage mean (M = 1.801).

Overview and Predictions

Studies 1 through 3 examined the COPS → intent relationship with cross-sectional, same-survey data. With Study 4, we demonstrate that the COPS has predictive validity for an index of actual behaviors over time. Further, intent has not always been predictive of behavior (Chandon, Morwitz, and Reinartz 2005), and thus we wanted to demonstrate that controlling for intent, the COPS has incremental predictive validity with regard to behaviors. Thus, Study 4 further validates COPS scores measured at Time 1 with a behavioral index of online privacy behaviors over a one-month period measured at Time 2 (H_3b). We also attempt to replicate the findings of Studies 1, 2, and 3.

Sample and Measures

Data were again sourced from the professional online panel company Prodege. The Time 1 sample comprised 777 children (different from those of Studies 1, 2, and 3), split evenly across gender and age categories, passing the attention screener, and receiving parent/guardian permission to participate. At Time 1, all participants responded to the same measures of Study 3, including the COPS (coeff. α = .863) with the changes noted subsequently. Study 4 assessed thrill-seeking tendencies and need to be popular, and we used a single-item measure of social media usage (see Web Appendix A). We also gathered a five-item short-form version of the Crowne–Marlowe social desirability responding scale to add as a control when predicting online child/teen privacy-related behaviors (Hays, Hayashi, and Stewart 1989). The same data checks of the previous studies were affirmed in Study 4. The mean and median amounts of time to complete the Time 1 survey were 12.32 and 10.43 minutes. The mean times did not differ by age group (age 6–7 years, M = 11.89 minutes; age 8–12 years, M = 12.80 minutes; and age 13–15 years, M = 12.11 minutes; F(2, 578) = .842, p = .428).

At Time 2 (one month later), the same participants responded to a behavioral index of what are considered safe and unsafe online privacy-related behaviors. These behaviors were adapted from COPS issues and were vetted through an expert judgment process similar to that of the expert judging of the original 50 COPS items. We originally drafted 32 behavioral items, and on the basis of responses of 12 expert judges with considerable expertise on child/teen online privacy, we initially retained 26 items. Following a further refinement on COPS-targeted issues, we retained a final set of 16 items for Time 2. Web Appendix A shows the 16 items scored as “yes” or “no.” Items were coded or recoded such that a positive online privacy-related behavior was scored as 1, and then the scores were summed to form an index that could range from 0 to 16. The final data set consisted of 581 of the 777 respondents of Time 1 (or 75%), matched by unique identifiers provided by the professional panel data provider (278 female, 303 male); 157 were age 6–7 years, 218 were age 8–12 years, and 206 were age 13–15 years. The mean and median amounts of time to complete the Time 2 survey were 2.70 and 1.89 minutes. The mean times did not differ by age group (age 6–7 years, M = 2.35 minutes; age 8–12 years, M = 2.91 minutes; and age 13–15 years, 2.68 minutes; F(2, 578) = 1.36, p = .256). Table W6 of Web Appendix B shows summary statistics and correlations among study variables.

Time 1 Analyses and Results

Intent to share information online

Accounting for full values on the social desirability questions, the sample size for the regression models was n = 571. Columns 4, 5, and 6 of Table 3 show the results for intent to share information online. Model 1 (F(11, 559) = 18.201, p < .001, R² = .264) shows significant effects for the control variables of gender (β = −.150, t = 2.260, p = .024), parents restricting online behavior (β = −.180, t = 2.099, p = .032), prior beliefs (β = −.130, t = 2.743, p = .006), thrill seeking (β = .198, t = 4.980, p < .001), need to be popular (β = .201, t = 5.878, p < .001), and social media use (β = .146, t = 4.983, p < .001).

In Model 2 (F_change(1, 558) = 11.237, p < .001, R² = .278) the COPS was again significant (β = −.033, t = 3.175, p < .001), supporting H_3a. Again, prior beliefs become nonsignificant in Model 2 when COPS scores are included as a predictor. Model 3 adds the interaction terms (F_change(3, 555) = 4.038, p = .007, R² = .294) and shows that the COPS × social media usage interaction was significant (β = −.019, t = 2.888, p = .004); thus, H_4c is supported. Figure 5 plots this effect.

OPEN IN VIEWER

Figure 5.

Study 4: Interaction Effect of COPS × Social Media Usage on Intent to Share.

When the COPS score is high and social media usage is low, intent to share information online is at its lowest (M = 1.958). This point was significantly lower (p = .004) than just the low COPS score–high social media usage mean (M = 2.521). No other combinations differed (i.e., high COPS score–high social media use: M = 2.089; low COPS score–low social media use: M = 2.032).

Time 2 Analyses and Results

We used a similar hierarchical regression approach for predicting the behavioral index with a focus on the degree to which COPS scores were significant in Model 2 after controlling for these covariates in Model 1: age, gender, race, social desirability bias, parents restricting online behavior, initial beliefs, intent to share, thrill seeking, need to be popular, and social media use. Model 1 (F(12, 558) = 3.623, p < .01, R² = .072) showed significant effects for race (β = .342, t = 2.060, p = .040), prior beliefs (β = .242, t = 2.218, p = .027), and social media usage (β = −.166, t = 2.417, p = .016) and a marginally significant effect of intent to share (β = −.179, t = 1.845, p = .066). No other predictor was significant. Model 2 (F_change(1, 557) = 11.829, p < .01, R² = .092) shows that COPS was significant in predicting the index of online behaviors (β = .080, t = 3.493, p < .001), supporting H_3b.

Summary of Findings

Across studies, we find support for the reliability and validity of the COPS. In Study 1, we find that the COPS is influenced by a cognitive defense experimental manipulation, as predicted. This suggests that education efforts to increase objective knowledge pertinent to children’s or young teens’ online privacy knowledge have efficacy; thus the COPS may be a useful tool to assess these educational efforts. In general, we also find expected mean score age differences for the COPS. Consistent with the child cognitive development literature, the 13-to-15-year age group scored higher than the 6-to-7-year age group across the four studies, and in three of the four studies, the 8-to-12-year age group scored higher than the 6-to-7-year age group.

In terms of predictive validity, the COPS (1) explained incremental variance in intent to share personal information online across all four studies controlling for a host of covariates known to predict intent; (2) explained incremental variance in an index of online privacy behaviors in Study 4; and (3) moderated the effects of need to be popular, thrill seeking, and social media usage in four of six predictions across the four studies. The introduction of the COPS also reduced the effect of initial online privacy beliefs to nonsignificance on intent and behaviors. The totality of these results suggests that the objective knowledge/literacy embedded in the COPS can predict both intent and behaviors related to online privacy, and, more importantly, offset the tendencies of some pervasive child/teen personality traits and behaviors known to be associated with sharing personal information online. This speaks favorably for the validity of the COPS and the use of the COPS as a screening tool to assess child/teen online privacy knowledge.

Public Policy Implications

The development and testing of the COPS offers several important implications for privacy policy. First, restricting everything that children and teens view online is undesirable, is improbable, and has been shown to be unsuccessful in many cases (60 Minutes 2022; Wendling 2017). At the same time, little is known about exactly what children/teens know and do not know about online privacy protection. Our reliable and valid COPS measure can help provide insight on the privacy knowledge/literacy that children acquire and can illustrate potential threats and risks to children. Such knowledge can benefit government organizations, tech companies, service providers, and educational organizations working to revise federal regulations (e.g., FTC and Congress with the revision of the COPPA; Kids Online Safety Act) and improve industry self-regulation (e.g., the BBB National Programs' Children's Advertising Review Unit).

Understanding how much children and teens know about privacy via the COPS can help determine when children or teens are sharing information (i.e., actively protecting their information and exhibiting trust in the interaction) or surrendering information (i.e., passively protecting their information and exhibiting faith) (Walker 2016). Since trust plays a crucial role in marketing exchanges, the COPS can be a useful tool for social responsibility issues involving children as a vulnerable population. The ability to identify whether children are surrendering or sharing information online is essential for assessing risk and preventing potential harm (Walker 2016). Emerging student privacy issues involve children's use of educational technology for learning at school and at home (Walker et al. 2022). Since digital literacy is often taught in schools, the COPS can be useful for assessing risks associated with educational technology.

In addition, the COPS measure can impact the practices of institutions and online service providers, inform the technical aspects of privacy protection, and improve strategies for individual privacy control. For instance, the COPS may be especially useful with legislation that places legal obligations on companies, such as the California Age-Appropriate Design Code Act (2022) requiring online platforms to promote privacy for children by default. This act compels companies that design any digital product or service to proactively assess the privacy and protection of children. Interestingly, the act specifically speaks of privacy as a child's right by noting that the United Nations Conventions on the Rights of the Child “recognizes that children need special safeguards and care in all aspects of their lives” (California Age-Appropriate Design Code Act [2022], Section 1.a.1).

The COPS can also serve as a beneficial baseline for measuring change and longitudinal work, much like that of other measures assessing adolescent risky behaviors, such as with tobacco, e-cigarette, and drug use (e.g., University of Michigan's Monitoring the Future, the Center for Disease Control and Prevention's Youth Risk Behavior Surveillance System, and the National Institutes of Health and Food and Drug Administration's Population Assessment of Tobacco and Health). In addition, the COPS can serve as an important measure for the evaluation of possible national, evidence-based privacy education campaigns. With proper message testing and controls, such campaigns can be effective in reducing youth risk behaviors (Farrelly et al. 2017). Recently, for example, the United Kingdom launched a “Twisted Toys” campaign (with Share Bear and Stalkie Talkie characters) showing how tech companies have at times preyed on children’s or teens’ private online data (Zakrewski 2021).

Limitations and Future Research

Limitations in the development and testing of the COPS are as follows. Although we implemented several data checks across studies, we still cannot completely rule out the possibility that parents may have helped their children complete the studies. While the general pattern of results we offer in Tables 2 and 3 and Web Appendix B Tables W3 to W6 are consistent with children or young teens completing the survey, parental involvement still may have occurred with online data collection. Although this limitation is inherent in any online studies with children or teens as participants, more rigorous tests will be needed to further validate the COPS, especially with younger age groups to evaluate their reading comprehension level with respect to the COPS items. This will require a controlled face-to-face setting in which only a trained researcher administers the COPS to children, with parental consent but without the possibility of parental involvement.

Second, other variables beyond the ones used in our studies could be used for further validity testing of the COPS. Online technology and privacy regulations can evolve over time, and the COPS may need revision in the future to account for such fundamental changes. In addition, obtaining and/or tracking actual behaviors of children, rather than self-reported behaviors, in academic research may prove challenging with Institutional Review Board guidelines and restrictions.

Future research might also involve testing the COPS more rigorously with underrepresented groups based on gender, race, and ethnicity. Our sample sizes ranged from 150 to 571 participants, with few respondents classifying themselves in categories other than “White.” For example, it is uncertain whether the results we reported would hold in studies with much larger and more representative samples of respondents classifying themselves as Black or African American, Hispanic/Latinx, Asian American, Native American, and/or multiracial. With much larger samples, it would be interesting to examine interactions of COPS × race and COPS × race × gender on important outcome variables for children's online privacy issues. We believe that efforts in this regard will be helpful in measuring exactly what children and teens know and do not know about privacy protection and in helping them protect their personal information and interactions online.