Measuring emotion socialization in families affected by pediatric cancer: Refinement and reduction of the Parents’ Beliefs about Children’s Emotions questionnaire

Abstract

The aim of this study is to conduct a multimethod psychometric reduction in the Parents’ Beliefs about Children’s Emotions (PBCE) questionnaire using an item response theory framework with a pediatric oncology sample. Participants were 216 pediatric oncology caregivers who completed the PBCE. The PBCE contains 105 items (11 subscales) rated on a 6-point Likert-type scale. We evaluated the PBCE subscale performance by applying a partial credit model in WINSTEPS. Sixty-six statistically weak items were removed, creating a 44-item PBCE questionnaire with 10 subscales and 3 response options per item. The refined scale displayed good psychometric properties and correlated .910 with the original PBCE. Additional analyses examined dimensionality, item-level (e.g. difficulty), and person-level (e.g. ethnicity) characteristics. The refined PBCE questionnaire provides better test information, improves instrument reliability, and reduces burden on families, providers, and researchers. With this improved measure, providers can more easily identify families who may benefit from psychosocial interventions targeting emotion socialization. The results of the multistep approach presented should be considered preliminary, given the limited sample size.

Keywords

Emotion socialization item response theory measurement theory Parents’ Beliefs about Children’s Emotions pediatric oncology Rasch model rating scales

Statistics from the National Cancer Institute (2010) estimate over 15,000 new pediatric cancer diagnoses each year worldwide, with rates by race and ethnicity being fairly similar across all diagnostic groups (Jemal et al., 2008). Fortunately, survival rates for pediatric cancer have greatly improved over the last half-century, such that over 80% of patients attain five-year survival following diagnosis. Given the average pediatric cancer treatment duration of three years, we can expect several thousand families have cancer-specific psychosocial stressors at any given time (Graziano et al., 2016; Kazak et al., 2001). These data suggest an increased need for continued research on quality-of-life outcomes among pediatric oncology survivors (Eiser et al., 2000).

Extant research suggests approximately 25% of pediatric cancer survivors experience significant problems with social competence, self-esteem, and academic and employment achievement (Aldridge and Roesch, 2007). A significant subset also experience life-disrupting symptoms of depression, anxiety, and treatment nonadherence (Kazak et al., 2005). Researchers have begun evaluating the unique role of parent emotion socialization (ES) in pediatric chronic illness groups, including pediatric oncology, in predicting patients’ psychosocial outcomes and long-term adjustment; however, many of these studies remain ongoing and have not disseminated findings. ES refers to ‘the socialization of children’s understanding, experience, expression, and regulation of emotion’ (Eisenberg et al., 1998). Eisenberg et al. (1998) and Morris et al. (2013) posited parents’ ES beliefs influence children’s socioemotional outcomes by teaching children about the acceptability of emotional experiences and about emotion display rules. Most research has supported the idea of dichotomous supportive versus nonsupportive parent ES practices (e.g. Hildenbrand et al., 2013; Sanger et al., 1991); however, nearly all published empirical studies evaluating the links between parents’ ES practices and child outcomes have been limited to healthy families and have not considered the potentially important role of childhood chronic illness in moderating outcomes. Children with cancer are removed from their social context (Corsano et al., 2015; Diller et al., 2009; Gurney et al., 2009) and behavior management is often interrupted during treatment (Kazak et al., 2005), which may make ES a particularly salient predictor of this group. Additionally, parents may view their ill child as vulnerable and their parenting role as that of making children happy (Mullins et al., 2007; Whiting, 2013), which may interfere with their ability to teach children how to appropriately manage both positive and negative emotions. Unfortunately, no published studies have examined ES in families with children with cancer.

Empirical studies with healthy children have utilized the Parents’ Beliefs about Children’s Emotions (PBCE; Halberstadt et al., 2013) scale to measure parents’ beliefs about shaping children’s emotional experiences; however, the 105-item PBCE may place undue burden on pediatric cancer families already managing cancer-related stressors. One way researchers have begun minimizing patient and family demands associated with study participation has involved reducing the length of measures (James et al., 2002). Questionnaire length has been proposed to affect response burden, or the effort required by an individual to answer a questionnaire (Rolstad et al., 2011). Response burden subsequently affects response rates – how likely participants are to complete study instruments (Rolstad et al., 2011). The longer the questionnaire, the higher the response burden placed on participants, diminishing the number of completed instruments in any given study design (Galesic and Bosnjak, 2009). This finding is particularly relevant to researchers interested in measuring constructs in pediatric oncology, given the overwhelming demands already placed on these families (James et al., 2002).

Fortunately, there are modern measurement methodology frameworks – such as item response theory (IRT) that can assist in refining and reducing existing measures to alleviate participant burden (Embretson and Reise, 2000; Hambleton and Jones, 1993). Classical test theory, in contrast to more modern measurement methodology, is a measurement framework that is based on the traditional ideas of reliability and validity, while forms of modern test theory (e.g. IRT) are based on the assumption that performance on tests are based on individual ability level (e.g. trait-level of person on each of the PBCE constructs) and item qualities (e.g. difficulty of item; Embretson and Reise, 2000). Given that a goal in IRT is to maintain only those items that contribute to unique variance in the measured construct, IRT has been used to reduce the length of measurement scales and to provide more precise psychometric understanding of each item’s contribution to an overall mean score (for review, see Embretson and Reise, 2000).

Understanding psychometric properties of the PBCE questionnaire in the context of pediatric oncology may help inform measurement literature and ultimately provide a richer quantitative perspective on how ES constructs may impact psychosocial outcomes and long-term adjustment in pediatric cancer patients. In addition, test information provided within an IRT framework may provide markers to identify families who are ‘low’ or ‘high’ on each of these constructs (e.g. negative emotions are valuable, children use emotions to manipulate others, children can control their emotions) and potentially inform psychosocial interventions. Overall, the refined PBCE questionnaire will be able to gauge PBCE in families affected by pediatric cancer to support coping and adjustment to cancer. To provide better test information, improve instrument reliability, and reduce measure length to limit participant burden in families affected by pediatric oncology, the current study examined the PBCE questionnaire within an IRT framework.

Method

Caregivers were approached and recruited by research personnel during (1) pediatric oncology patients’ outpatient clinic appointments or (2) inpatient at a pediatric oncology treatment center located in the southwest region of the United States. To participate, adult caregivers needed to have a child with an existing oncological diagnosis (e.g. leukemia, Wilms tumor, brain tumor) for a minimum of six weeks, identify as a primary caregiver (i.e. live with child at least 50% of the time and provide at least 50% of the child’s care), and be able to read and/or speak English or Spanish. Caregivers were approached by research personnel, provided consent, and completed several paper and pencil measures as part of a larger study. The local institutional review board granted ethical approval prior to study initiation.

Measures

Demographics questionnaire

Caregivers completed a brief demographic questionnaire regarding age, sex, ethnicity, and race pertaining to self and child. Additionally, caregivers provided information on their child’s oncological illness (e.g. diagnosis, treatment received, date of diagnosis), family constellation, family income, and caregiver educational level.

PBCE questionnaire

The Parents’ Beliefs about Negative Emotions questionnaire (Halberstadt et al., 2008) and its successor, the PBCE scale (Halberstadt et al., 2008), are existing measures of parents' beliefs about children's emotions. The PBCE contains 105 items across 11 subscales that caregivers rate on a 6-point Likert-type scale (1 = Strongly disagree, 6 = Strongly agree). Exploratory and confirmatory factor analyses have demonstrated good factor structure for the PBCE when used with Caucasian, African-American, and Lumbee Native American parents (Halberstadt et al., 2013). Subscales of the PBCE (see Table 1 for a list of subscales and definitions) have also shown good predictive validity for parents’ ES practices (Wong et al., 2009), children’s emotional understanding, and children’s perceptions of competence with peers (Wong et al., 2008). Subscales of the PBCE scale have shown good internal consistency in other studies as well (α = .78 to .86; Halberstadt et al., 2008). Spanish translation of study questionnaires was conducted by native Spanish speakers and included forward and backward translation (forward and backward translation conducted by different translators). Caregiver completion time for the 105-item PBCE ranged between 30 and 120 minutes when completed in one administration; however, some caregivers chose to complete the PBCE measure over several days or appointments secondary to fatigue, response burden, or interruptions by members of their child’s medical team. The PBCE completion time could not be assessed for caregivers who chose to complete the measures over several days and/or appointments.

Table 1.

PBCE subscales, constructs, and items.

Subscale	Sample item	Original number of items	Reduced number of items
Control	‘Children can control their feelings if they really want to’	10	4
Developmental processes	‘Children can change their emotional styles as they grow up’	8	4
Parents need to guide	‘It is a parent’s job to teach children how to handle their emotions’	9	4
Children can learn on their own	‘Children can learn to manage emotions without help from parents’	8	3
Contempt	‘Parents should never put down a child’s feelings’	8	0
Manipulation	‘Children sometimes show affection to get something they want’	10	5
Privacy	‘Parents don’t need to know everything their child is feeling’	7	3
Positive emotions are valuable	‘It is important for children to share their positive emotions with others’	10	4
Negative emotions are valuable	‘It is useful for children to feel angry sometimes’	12	7
All emotions are dangerous	‘Kid’s feelings can get hurt if they show too much emotion’	13	6
Emotions just are	‘Feeling all emotions is a part of life, like breathing’	10	4
		105	44

PBCE: parents’ beliefs about children’s emotions.

Note: For the privacy subscale, the higher the score, the more parents believe that they need to know what children are feeling, and the lower the score, the more parents are accepting children’s privacy.

Statistical analyses

The IRT framework was approached in three distinct phases and utilized a graded response model (Samejima, 1997), which is appropriate for ordered categorical responses like the PBCE Likert scales. The analyses presented below follow recommendations for measurement refinement from an IRT perspective (Linacre, 2013) but deviate somewhat from initially proposed procedures, given that instrument modification is a partially data-driven and iterative process. Reise and Yu (1990) suggest IRT models can be estimated with as few as 250 participants but recommend including 500 participants to reduce the standard errors of parameter estimates. The current study had a limited recruitment phase, which interfered in obtaining a large sample of participants. As such, the results of the multistep approach presented here should be considered preliminary, given the sample size (N = 216).

Analyses progressed in several stages. First, for each subscale, response option performance was assessed by examining Rasch–Andrich thresholds using WINSTEPS (Linacre, 2012), which provides graphical representations of probability curves for each item. In addition, the frequency of endorsed response options was examined to identify whether items were particularly discriminating among participants. Based on these findings, refinement of the response options (e.g. collapsing categories) was considered and employed for each subscale to improve the psychometric properties and the discriminating nature of each subscale. Second, item performance was evaluated to assist in scale reduction. Scale reduction involved eliminating items whose response options did not conform to an ordered Rasch–Andrich threshold or demonstrated poor item infit/outfit statistics.

Third, model fit of the refined and reduced PBCE instrument was analyzed using a Rasch Rating Scale Model (RSM) and a Partial Credit Model (PCM) in WINSTEPS version 3.74.0 (Linacre, 2012). χ ² difference analyses were then employed to identify the best-fit model for the data. The most optimal and parsimonious model was then used in all subsequent analyses.

Lastly, several item-level and instrument-level statistics were examined in WINSTEPS, including item fit, person fit, item discrimination, and instrument dimensionality. IRT assumes local independence of the items and unidimensionality of each factor or subscale. Currently, there is no standard procedure for establishing adequate unidimensionality; generally, evidence of a dominant factor explaining a large proportion of variance and goodness-of-fit indices are assessed (Embretson and Reise, 2000).

Results

Participants

Caregivers (N = 216, 74.59% female) ranged in age from 21 to 65 years (M = 38.76, SD = 9.99) and self-identified as mothers (75%), fathers (18%), or other guardians (7%). Caregivers identified as Caucasian (64.86%), with the remaining identifying as African-American (19.84%) or other (17.29%). The majority of caregivers identified as non-Hispanic (65.57%). Most caregivers obtained a high school diploma or commiserate certificate (40.4%), with the remaining not having finished high school (10.3%), having some college or associates/professional degree (13.8%) or having obtained an undergraduate or graduate degree (35.5%). Caregivers reported household incomes of US$0–US$20,000 (25.2%), US$20,001–US$40,000 (18.1%), US$40,001–US$60,000 (11.1%), and US$60,001–US$80,000 (34.0%). Patients (N = 216) receiving pediatric oncology treatment protocols ranged in age from 1 year to 19 years (M = 9.96, SD = 4.91). The percentage of males (63%) was higher than that of females. Patients were predominantly Caucasian (64.29%), with the remaining patients identifying as African-American (20.00%) and other (15.71%). The majority of patients identified as non-Hispanic (62.04%). Most patients had a diagnosis of acute lymphoblastic leukemia (41.5%), followed by central nervous system (CNS)/brain tumor (16.3%), osteosarcoma (8.9%), neurofibromatosis (8.1%), lymphoma (8.2%), neuroblastoma (6.5%), and other types of cancer (10.6%). Per chart review, the majority of patients (50.3%) were receiving assistance in the form of government-funded insurance (i.e. Medicaid).

Refinement of the rating scale

Examination of Andrich thresholds across each subscale of the PBCE revealed disordered averages between categories 1 (strongly disagree) and 2 (somewhat disagree), 3 (slightly disagree) and 4 (slightly agree), and 5 (somewhat agree) and 6 (strongly agree). That is, participants appeared to have difficulty differentiating between response options, indicating a possible misunderstanding between adjacent response options. For example, endorsing slightly agree did not require a substantially higher level of the subscale construct (e.g. negative emotions are valuable) than slightly disagree. Similar results were found between adjacent categories 5 and 6, as well as 1 and 2. Consequently, categories 1 and 2 were collapsed to create a new response category of disagree, categories 3 and 4 were collapsed to create a new response category of neither agree nor disagree, and categories 5 and 6 were collapsed to create a new response category of agree. A PCM with the new response categories for each subscale of the PBCE (i.e. disagree, neither agree nor disagree, agree) revealed an improvement in rating scale fit.

Item performance

Examination of item fit for each subscale of the PBCE suggested appropriate infit and outfit (less than 1.33), with the exception of 14 items (see boldface entries in column (a) of Table 2). Due to disordered thresholds and poor item fit, these 14 items were eliminated. Many of the remaining 91 items on the PBCE subscales appeared similar at face value, requiring time and effort on the participant’s part to complete. For example, within the control subscale, the item, children can keep themselves from getting too happy when they want to, appeared to capture a very similar concept as the item, children can keep themselves from getting overly excited when they want to. As such, item discrimination was examined for the remaining items on each subscale of the PBCE utilizing bivariate correlations and paired samples t tests. Results of these analyses suggested 44 items added little to each subscale’s ability to measure the construct, therefore allowing those items to be eliminated from their respective subscales. Our goal in eliminating statistically weaker items was to reduce the questionnaire length without compromising reliability. Reexamination of item fit statistics for the remaining 47 items (see column (b) of Table 2) revealed appropriate infit and outfit (less than 1.33).

Table 2.

Item and scale information from the IRT analysis of the PBCE questionnaire.

Subscales and items		Original fit stats MNSQ^a		Refined fit stats MNSQ^b		Dimensionality eigenvalue (%)^c		Model Fit^d
Subscales and items		Infit	Outfit	Infit	Outfit	Original	Refined	Δχ²
Control						1.8 (10.5)	1.5 (19.3)	4.99
7	Children can control their emotions.	.97	.96	.94	.97
14	*Children can keep themselves from getting too happy when they want to.	.95	.92	–	–
20	When children are very angry, they can control what they show to others.	.96	.97	.97	.91
27	When children are feeling sad, they can stop those feelings.	.99	.93	1.13	1.19
33	*It is easy for children to control their happiness.	1.19	1.21	–	–
40	*When children are angry, it’s hard for them to hide their feelings. (R)	1.73	1.88	–	–
47	*Children can keep themselves from getting overly excited when they want to.	.91	.97	–	–
53	Children can control how they express their feelings.	.93	1.05	1.01	.94
58	*When children are feeling angry, they can stop those feelings.	.86	.83
63	*Children can control how they feel if they really want to.	.80	.78
Developmental processes						1.8 (12.6)	1.6 (24.8)	2.04
5	Children can change their emotional patterns over time.	1.11	1.27	1.10	1.11
12	*Children can change their emotional styles as they grow up.	.85	.83	–	–
19	How children express emotions can change over time.	.72	.67	.76	.75
25	*Children can grow and change is how they approach emotional situations.	1.19	1.07	–	–
32	How children feel can change throughout their childhood.	.82	.75	1.03	1.21
38	*Children’s emotional styles tend to stay the same over time. (R)	1.91	1.97	–	–
45	*Children’s emotional styles can change a great deal as they mature.	.74	.82	–	–
52	As children grow up, they can change how they express emotions.	.80	.79	1.04	1.07
Parental guidance						1.6 (10.8)	1.5 (22.7)	4.66
10	It’s the parent’s job to teach children how to handle negative feelings.	.72	1.07	.98	.88
17	It’s the parent’s job to help children know when and how to express their positive emotions.	1.01	1.17	.90	.88
30	*It’s important for parents to help a child who is feeling sad.	1.37	1.59	–	–
37	It is important for parents to teach children when and how to show pride in themselves.	.93	.93	.90	.91
43	*It’s the parent’s job to teach children about happiness.	1.09	1.19	–	–
50	*When children are feeling angry, parents can help them work through those feelings.	1.06	1.02	–	–
61	It is the parent’s job to teach their children how to handle their emotions.	1.00	1.19	1.26	1.22
76	*It’s important parents teach children the best ways to express their feelings.	.99	1.05	–	–
79	*It’s the parent’s job to teach children how to deal with distress and other upsetting feelings.	.82	.74	–	–
Child guidance						1.7 (12.0)	1.6 (33.7)	2.78
4	*When children become sad or upset, parents can let them manage their feelings on their own.	1.36	1.47	–	–

23	*How and when to show positive emotions is something that children have to figure out for themselves.	1.18	1.17	–	–
56	*Children can figure out how to express sad feelings on their own.	1.04	1.01	–	–
66	*Children generally learn how to deal with their angry feelings, without parents telling them how.	.93	1.01	–	–
69	*It’s usually best to let a child work through their negative feelings on their own.	.85	.81	–	–
73	Children can learn to manage their emotions without help from parents.	1.00	1.05	1.03	1.02
83	When children are angry, it is best to just let them work it through on their own.	.90	.94	1.12	1.11
86	Children can figure out how to express their feelings on their own.	.81	.79	.85	.83
Contempt						2.0 (19.4)	1.6 (41.4)	–
2	*Sarcasm is an effective way to get children to change what they are doing.	1.18	1.40	1.04	1.02
22	*Making fun of children’s behavior is never a good idea. (R)	1.21	1.42	–	–
42	*Making fun of children’s feelings is sometimes a good way to get them to change their behavior.	.94	.94	.92	.88
59	*Mocking children can be a good way to teach children to change what they are doing.	.95	.90	–	–
71	*Parents should not mock children’s feelings. (R)	.88	.74	–	–
84	*Making fun of children’s behavior sometimes helps children to change what they are doing.	.92	.89	1.05	1.03
93	*It is not helpful for parents to make fun of their children’s feelings. (R)	1.07	1.23	–	–
99	*Parents should not mock their children’s behavior. (R)	.91	.80	–	–
Manipulation						1.8 (9.1)	1.6 (18.8)	8.55
9	Children sometimes act angry, just to get attention.	1.11	1.04	1.11	1.06
28	Children often act sad just to get their own way.	.94	.92	1.01	.96
48	Children often act angry just to get their own way.	.83	.84	.90	.89
64	Children use emotions to manipulate others.	.90	.98	1.00	1.00
74	*Children sometimes show emotion to try and control the situation.	1.20	1.24	–	–
81	*Children sometimes say ‘I love you’ to get what they want.	1.09	1.26	–	–
88	*Children’s emotions are sincere rather than manipulative. (R)	1.54	1.54	–	–
91	*Children sometimes act sad just to get attention.	.65	.65	–	–
95	*Children know they can get their own way if they act really excited about what they want.	1.11	1.16	–	–
101	Children sometimes show affection to get something they want.	.90	.80	1.00	1.06
Privacy						1.9 (11.6)	1.6 (22.2)	1.14
15	*Parents don’t have to know about all their child’s feelings. (R)	1.83	1.84
35	It is important for children to share all of their feelings, good or bad, with their parents.	1.00	1.08	.91	1.10
54	Parents need to know everything that their children are feeling.	.93	.92	1.00	.98
68	*It is important for children to tell their parents everything that they are feeling.	.82	.83	–	–
78	Parents should encourage their child to tell them everything they are feeling.	.76	.72	1.03	1.00
90	*When children are experiencing an emotion, they should talk to their parents about it.	.85	.82	–	–
97	*Parents need to know what their children are feeling.	.76	.74	–	–
Positive emotions are valuable						2.0 (14.5)	1.6 (32.1)	1.71
8	*It’s good for the family when children show their positive emotions.	1.37	2.19	–	–
11	*It is important for children to be able to show when they are happy.	.75	.68	–	–
13	*It is important for children to express their happiness when they feel it.	.92	.95	–	–
31	It is important for children to develop lots of ways to be happy.	1.15	1.16	1.00	1.02
51	*Joy is an important emotion to feel.	.97	.73	–	–
60	Having lots of joy is very important for a child.	1.08	1.23	1.01	.99
67	When children show pride in what they have done, it is a good thing.	1.13	1.31	1.04	.77
77	It is important for children to share their positive emotions with others.	.88	.89	1.04	1.04
82	*It is important for children to feel pride in their accomplishments.	.97	.93	–	–
87	*It is important for children to be proud of a job well done.	.82	.64	–	–
Negative emotions are valuable						2.0 (10.5)	1.5 (15.1)	13.67*
1	*Getting mad can help children do things they need to, like sticking with a task that’s hard, or standing up for themselves.	1.26	1.36	–	–
18	It is good for children to feel sad at times.	.85	.91	1.15	1.15
39	It is important for children to show others when they feel upset.	1.12	1.09	1.11	1.12
46	Children’s anger can be a relief to them, like a storm that clears the air.	1.01	1.05	.88	.92
49	It is useful for children to feel angry sometimes.	.82	.77	.79	.85
55	Sometimes it is good for a child to sit down and have a good cry.	.92	.85	.97	.82
70	It is good for children to let their anger out.	1.02	.97	.95	.97
72	When children show anger, they are letting you know that something is important to them.	1.03	.99	1.17	1.08
89	*Feeling sad helps children to know what is important to them.	.83	.87	–	–
96	*The experience of anger can be a useful motivation for action.	.92	.93	–	–
103	*Being angry can motivate children to change or fix something in their lives.	1.15	1.40	–	–
104	*Expressing anger is a good way for children to change or fix something in their lives.	1.16	1.20
All emotions are dangerous						2.0 (9.6)	1.5 (17.9)	11.72*
21	*Feeling negative emotions is sort of a dead end street, and children should do whatever they can to avoid going down it.	1.12	1.36	–	–
24	Showing anger is not a good idea for children.	.92	.90	.96	.97
29	When children get angry, they create more problems for themselves.	.85	.86	.78	.77
36	Feeling sad is just not good for children.	.93	.96	.90	.89
44	*When children are too loving, others take advantage of them.	1.08	1.09	–	–
57	*When children get angry, it can only lead to problems.	.77	.75	–	–
65	When children are too happy, they can get out of control.	1.18	1.30	1.20	1.21
75	*It is important for children to avoid feeling sad whenever possible.	1.01	1.08	–	–
92	*When children express anger, someone in the family ends up having to deal with the consequences.	1.07	1.13
94	Anger in children can be emotionally dangerous.	.95	.97	.89	.90
100	*Children can think more clearly when emotions don’t get in the way.	1.15	1.13	–	–
102	Children’s feelings can get hurt if they love too much.	1.14	1.23	1.29	1.27
105	*When children start to show strong emotions, one never knows where it will end up.	.89	.93
Emotions just are						1.9 (13.0)	1.6 (29.9)	8.99*
3	*It is okay when children feel angry, and it is okay when they don’t.	1.20	1.70	–	–
6	Showing emotions isn’t a good thing or a bad thing, it’s just part of being human.	1.36	1.41	1.12	1.07
16	Feeling sad sometimes is just a part of life.	1.05	1.34	1.01	.96
26	*Feeling all emotions is a part of life, like breathing.	.89	.84	–	–
34	Feeling angry sometimes is just a part of life.	.86	.82	.81	.89
41	*It is okay when children feel sad, and it is okay when they don’t.	1.07	1.23	–	–
62	*Showing sadness is neither bad nor good, it is just part of being human.	.92	.81	–	–
80	*Being sad isn’t ‘good’ or ‘bad’ – it is just part of life.	.73	.70	–	–
85	*Being angry isn’t ‘good’ or ‘bad’ – it is just part of life.	.88	.81	–	–
98	It is okay when children feel happy, and it is okay when they don’t.	1.00	1.19	.96	1.02

IRT: item response theory; PBCE: Parents’ Beliefs About Children’s Emotions; RSM: Rasch rating scale model; PCM: partial credit model.

Note: Infit and outfit values >1.33 indicate poor fit. Boldface entries indicate items with poor fit.

*Items dropped to improve model fit. Items were dropped secondary to (1) disordered Andrich thresholds, (2) poor infit/outfit statistics, or (3) item’s poor discrimination (paired samples t-test results) paired with clinical judgment.

(R) Reverse-coded item.

^aValues reported based on original 6-item Likert response scale.

^bValues reported based on refined 3-item Likert response scale.

^cSubscale dimensionality reported for RSM.

^d χ ² difference values between PCM and RSM.

Subscale dimensionality

Following examination of rating scale performance and item fit, dimensionality was examined for each refined subscale (see column (c) of Table 2). With the exception of the contempt subscale (eigenvalue = 1.6 and 41.4%), unexplained variance in the first contrast for each subscale suggested unidimensionality. As a result, the contempt subscale was eliminated due to multidimensionality and poor discriminating properties, yielding a total of 44 items across 10 subscales (see Table 1 for a comparison of original versus final number of PBCE items).

Comparison of model fit

Following instrument refinement, model fit indices from a PCM and RSM were examined for each of the remaining 10 subscales of the PBCE in WINSTEPS. The PCM is a more complex IRT model because it allows thresholds to vary across items, while the RSM is considered more parsimonious because it restricts thresholds to be equivalent across items. For example, the RSM conceptualizes the measurement scale on a standard metric while the PCM does not. This means that in the RSM, differences between response options are the same – the difference between a score of 4 and 6 is the same as the difference between 2 and 4. Therefore, Rasch response scales are often easier to interpret and provide better clinical utility (Hays et al., 2000). A χ ² difference test was employed to examine the fit between the two models to identify whether the PCM significantly improved model fit over the more parsimonious RSM. Results from the χ ² difference test indicated the PCM did not significantly improve model fit over the RSM for the majority of the refined subscales (see column (d) of Table 2). The χ ² difference test indicated the PCM significantly improved model fit over the RSM for the negative emotions are valuable (Δχ ²(6) = 13.67, p = .03), all emotions are dangerous (Δχ ²(5) = 11.72, p = .04), and emotions just are (Δχ ²(3) = 8.99, p = .03) subscales. However, the more parsimonious RSM was retained and utilized for all subsequent analyses for the purposes of increasing the clinical utility of the refined PBCE measure. By using the RSM all responses can be scored on a standard interval level, allowing results to be compared among those who answer the PBCE differently (Hays et al., 2000).

Examination of item and person fit

Next, item fit and person fit were examined in WINSTEPS 3.74.0 using a RSM. Acceptable item fit suggests that the item parameters are valid (i.e. they accurately represent how participants respond to items), while adequate person fit indicates that individual trait levels (e.g. ES) are valid indicators of that person’s position on the latent continuum (e.g. negative emotions are valuable; Embretson and Reise, 2000). According to Baker (2001), infit/outfit cutoff statistics for item-fit parameters are ≥1.33, while person-fit cutoff statistics are ≥2.00 (Li and Olejnik, 1997; Seo and Weiss, 2013. Examination of item fit revealed that all remaining items on the 10 remaining subscales of the PBCE revealed acceptable infit and outfit statistics (i.e. they did not evidence infit and/or outfit statistics above the 1.33 cutoff) and were retained for further analyses (see column (b) of Table 2). Examination of person fit on all of the remaining PBCE subscales revealed, on average, approximately 12 (5.55%) participants with an inadequate infit or output value above 2.00. Given the preliminary nature of the current investigation with a limited sample size (N = 216), participants were not eliminated from any of the instruments based on person-fit statistics.

Discussion

Data from health and psychosocial measures are likely to be compromised if patients find completing instruments a burden (Jenkinson et al., 2001). Instruments with a relatively moderate number of items can present a considerable challenge for families who are already overwhelmed by treatments, numerous medical appointments, and the psychosocial toll associated with a life-threatening illness such as pediatric cancer. Consequently, brevity is to be sought whenever possible in the design and implementation of research measures (Jenkinson et al., 2001). We conducted a psychometric evaluation to reduce and refine the 11-subscale, 105-item PBCE using IRT methodology. We identified 61 statistically weak PBCE items based on data collected from 216 families affected by pediatric cancer. These items were removed resulting in a 10-subscale, 44-item PBCE. This newly refined and shortened instrument has improved psychometric properties with considerably reduced response burden compared to the original PBCE.

As part of the measurement refinement process, an important goal of the present study was to explore the performance of the six-option response scale. While Likert scales with numerous options are usually developed to capture a variety of responses, measurement researchers suggest numerous response options are not necessarily better given that they are sensitive to nuanced measurement problems (Jacoby and Matell, 1971). In fact, several empirical studies suggest Likert scales with too many options tend to be unreliable (Jacoby and Matell, 1971; Matell and Jacoby, 1972). It appears that the large number of response options on the PBCE in the current study became too cumbersome for respondents to use, as evidenced by the samples’ lack of endorsement of some response options entirely on several items (e.g. when children show pride in what they have done, it is a good thing; it is important for children to show their positive emotions with others; it’s the parents’ job to teach children how to deal with distress and other upsetting feelings). One plausible explanation to these findings is that the purported benefits of increasing variability in response options are outweighed by participant fatigue (Lavakas, 2008). Additionally, the analytical sensitivity of the response scale may be compromised when respondents interpret the response options in different ways. In other words, what one participant may describe as slightly agree may mean the same, in absolute terms, as what another participant might describe as slightly disagree. This phenomenon is amplified when the number of potential responses is large, which decreases the interpretability and therefore clinical utility of findings (Jamieson, 2004; Lavakas, 2008). As such, rating scale performance analysis resulted in new response categories for each subscale of the PBCE (i.e. disagree, neither agree nor disagree, and agree), which revealed an improvement in rating scale fit. Results of the present study suggest researchers in measurement development or refinement minimize the number of Likert response options in order to reduce participant fatigue, increase participant interpretation of response scales, improve the overall psychometric properties of the instrument, and improve the clinical utility of measures.

An additional goal of the present study was to empirically explore the structure of each of the 11 original subscales (e.g. negative emotions are valuable) of the PBCE using IRT. Following instrument refinement, results from the current study revealed a one-factor structure for each subscale of the PBCE with the exception of the contempt subscale. After examining item-fit statistics (i.e. infit and outfit values), only three items remained as part of the contempt subscale (see Table 2, column (b)). It is possible that these items, although statistically sound, were inappropriate for measuring the construct of contempt for children’s emotions. For example, item 2, sarcasm is an effective way to get children to change what they are doing, appears to have poor face validity. It is also possible that contempt for children’s emotions, as currently measured by the PBCE, is not a relevant construct for caregivers of pediatric oncology patients. This might be particularly true for a subgroup of families (approximately 25–30%) that have significant difficulties in personal and social domains (Colletti et al., 2008; Vannatta and Gerhardt, 2003). Often, these families show increased parental overprotection and perceived child vulnerability (Colletti et al., 2008). Parental overprotection involves protective parenting behavior that is considered excessive given the child’s developmental age (Colletti et al., 2008; Thomasgard and Metz, 1997). Caregivers who exhibit overprotective parenting and perceive their child to be vulnerable might refrain from showing contempt for their child’s emotion as this might be seen as detrimental to a child who is undergoing cancer treatment. Consequently, the contempt subscale was eliminated due to multidimensionality and poor discriminating properties, resulting in 10 subscales.

Following the initial instrument refinement phase, several subscales of the PBCE contained both positively and negatively worded items. All reverse-coded items (e.g. It is not helpful for parents to make fun of their children’s feelings; see Table 2 for all reverse-coded items) were removed from the PBCE because they appeared to distort the measurement model. Findings contradict earlier work in the area of scale development that indicates a preference for reverse-coded items in most summated measures (DeVellis, 2003). Originally devised as a way to minimize inattention and acquiescent responding in individuals secondary to item fatigue, measurement problems outweighed the potential benefits of the inclusion of items that are worded in the opposite direction (Lavakas, 2008). One of the problems is that reverse-coded items frequently produce unexpected factor structures (Netemeyer et al., 2003), an undesirable characteristic of subscales that are supposed to be unidimensional. Another issue when making or utilizing a scale composed of items with opposing meanings is miscomprehension (Swain et al., 2008), as it is easy for respondents to misinterpret phrases that include negation. These problems are often compounded when scales are translated for use in other languages (Wong et al., 2003). Measurement issues concerning linguistic translations are increasingly important as psychosocial research aims to include representative samples of ethnic and cultural groups reflecting the composition of the US population (Varricchio, 2004). Findings suggest researchers in measurement development or refinement refrain from using reverse-coded items, particularly in instruments with few items where fatigue is unlikely to play a role in participant response selection.

The current study had notable strengths, including being the first to examine the PBCE using IRT methodology to reduce participant burden as well as utilizing a diverse (e.g. ethnicity, language) and unique sample (i.e. caregivers of pediatric oncology patients). Despite numerous strengths, a major limitation regarding sample size is worth noting. The current study included 216 participants, which rendered analyses informative, yet exploratory and preliminary in nature. Having more participants would have increased the power and robustness of statistical tests. With more participants, differential item function analyses among caregiver subsets could have been explored. These analyses would have determined whether specific items (e.g. Sometimes it is good for a child to sit down and have a good cry) function differently among caregivers based on sex, race/ethnicity, patient diagnosis (e.g. leukemia vs. CNS tumors), or caregivers’ linguistic preferences (e.g. Spanish vs. English).

The role of IRT in new instrument development and refinement has grown substantially over the last decade, with a greater number of studies examining the usefulness of IRT for developing disease-specific questionnaires (Jenkinson et al., 2001). The current study adds to this growing literature with findings that suggest the newly refined, psychometrically sound PBCE can be used to acquire data relevant to ES in families affected by pediatric cancer. These data will be important in determining which families may benefit from psychosocial interventions targeted at ES.

Footnotes

Declaration of Conflicting Interests

The author(s) declare that they have no conflict regarding this article and have no involvements that might raise the question of bias in the work reported or in the conclusions, implications, and opinions stated.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Aldridge

Roesch

(2007) Coping and adjustment in children with cancer: a meta-analytic study. Journal of Behavioral Medicine 30: 115–129.

Baker

(2001) The Basics of Item Response Theory. College Park, MD: ERIC Clearinghouse on Assessment and Evaluation.

Colletti

Wolfe-Christensen

Carpentier

. (2008) The relationship of parental overprotection, perceived vulnerability, and parenting stress to behavioral, emotional, and social adjustment in children with cancer. Pediatric Blood & Cancer 51(2): 269–274.

Corsano

Cigala

Majorano

. (2015) Speaking about emotional events in the hospital: The role of health-care professionals in children emotional experiences. Journal of Child Health Care 19(1): 84–92.

DeVellis

(2003) Scale Development: Theory and Applications. Thousand Oaks, CA: Sage Publications, Inc.

Diller

Chow

Gurney

. (2009) Chronic disease in the childhood cancer survivor study cohort: a review of published findings. Journal of Clinical Oncology 27(14): 2339–2355.

Eisenberg

Cumberland

Spinrad

(1998) Parental socialization of emotion. Psychological Inquiry 9(4): 241–273.

Eiser

Hill

Vance

(2000) Examining the psychological consequences of surviving childhood cancer: systematic review as a research method in pediatric psychology. Journal of Pediatric Psychology 25(6): 449–460.

Embretson

Reise

(2000) Item Response Theory for Psychologists. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

10.

Galesic

Bosnjak

(2009) Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opinion Quarterly 73(2): 349–360.

11.

Graziano

Rossi

Spano

. (2016) Comparison of psychological functioning in children and their mothers living through a life-threatening and non life-threatening chronic disease: a pilot study. Journal of Child Health Care 20(2): 174–184.

12.

Gurney

Krull

Kadan-Lottick

. (2009) Social outcomes in the childhood cancer survivor study cohort. Journal of Clinical Oncology 27(14): 2390–2395.

13.

Halberstadt

Dunsmore

Bryant

Jr . (2013) Development and validation of the parents’ beliefs about children’s emotions questionnaire. Psychological Assessment 25(4): 1195.

14.

Halberstadt

Thompson

Parker

. (2008) Parents’ emotion-related beliefs and behaviors predict children’s reactions to the September 11, 2001 terrorist attacks. Infant and Child Development 17: 557–580.

15.

Hays

Morales

Reise

(2000) Item response theory and health outcomes measurement in the 21st century. Medical Care 38(9): II28.

16.

Hildenbrand

Alderfer

Deatrick

. (2013) A mixed methods assessment of coping with pediatric cancer. Journal of Psychosocial Oncology 32(1): 37.

17.

Hambleton

Jones

(1993) Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice 12(3): 3847.

18.

Jacoby

Matell

(1971) Three-point Likert scales are good enough. Journal of Marketing Research 8(4): 495–500.

19.

James

Keegan-Wells

Hinds

. (2002) The care of my child with cancer: parents’ perceptions of caregiving demands. Journal of Pediatric Oncology Nursing 19(6): 218–228.

20.

Jamieson

(2004) Likert scales: how to (ab) use them. Medical Education 38(12): 1217–1218.

21.

Jemal

Siegel

Ward

. (2008) Cancer statistics, 2008. CA: A Cancer Journal for Clinicians 58(2): 71–96.

22.

Jenkinson

Fitzpatrick

Garratt

. (2001) Can item response theory reduce patient burden when measuring health status in neurological disorders? Results from Rasch analysis of the SF-36 physical functioning scale (PF-10). Journal of Neurology Neurosurgery & Psychiatry 71(2): 220–224.

23.

Kazak

Prusak

McSherry

. (2001) The psychosocial assessment tool (PAT): pilot data on a brief screening instrument for identifying high risk families in pediatric oncology. Families, Systems, & Health 19(3): 303.

24.

Kazak

Boeving

Alderfer

. (2005) Posttraumatic stress symptoms during treatment in parents of children with cancer. Journal of Clinical Oncology 23(30): 7405–7410.

25.

Lavakas

(2008) Encyclopedia of Survey Research Methods. Thousand Oaks, CA: Sage Publications, Inc. DOI: 10.4135/9781412963947.

26.

MNF

Olejnik

(1997) The power of Rasch person-fit statistics in detecting unusual response patterns. Applied Psychological Measurement 21(3): 215–231.

27.

Linacre

(2012) WINSTEPS® (Version 3.74.0) [Computer Software]. Beaverton, Oregon: Winsteps.com. Available at: http://www.winsteps.com/ (accessed 1 January 2016).

28.

Linacre

(2013) Rasch-Model Computer Program Manual. Chicago, IL: MESA Press.

29.

Matell

Jacoby

(1972) Is there an optimal number of alternatives for Likert scale items? Effects of testing time and scale properties. Journal of Applied Psychology 56(6): 506–509.

30.

Morris

Denham

Bassett

. (2013) Relations among teachers’ emotion socialization beliefs and practices and preschoolers’ emotional competence. Early Education and Development 24(7): 979.

31.

Mullins

Wolfe-Christensen

Pai

ALH

. (2007) The relationship of parental overprotection, perceived child vulnerability, and parenting stress to uncertainty in youth with chronic illness. Journal of Pediatric Psychology 32(8): 973.

32.

National Cancer Institute (2010) Cancer statistics. Available at: https://www.cancer.gov/about-cancer/understanding/statistics/. (accessed 10 December 2016).

33.

Netemeyer

Bearden

Sharma

(2003) Scaling Procedures: Issues and Applications. Newbury, CA: Sage Publications, Inc.

34.

Reise

(1990) Parameter recovery in the graded response model using MULTILOG. Journal of Educational Measurement 27(2): 133–144.

35.

Rolstad

Adler

Rydén

(2011) Response burden and questionnaire length: is shorter better? A review and meta-analysis. Value in Health 14(8): 1101–1108.

36.

Samejima

(1997) Graded response model. In: Wim

van der

Linden

Hambleton

Ronald K

(eds) Handbook of Modern Item Response Theory. Springer: New York, pp. 85–100.

37.

Sanger

Copeland

Davidson

(1991) Psychosocial adjustment among pediatric cancer patients: a multidimensional assessment. Journal of Pediatric Psychology 16(4): 463.

38.

Seo

Weiss

(2013) I_z Person-fit index to identify misfit students with achievement test data. Educational and Psychological Measurement 73(6): 994–1016.

39.

Swain

Weathers

Niedrich

(2008) Assessing three sources of misresponse to reversed Likert items. Journal of Marketing Research 45(1): 116–131.

40.

Thomasgard

Metz

(1997) Parental overprotection and its relation to perceived child vulnerability. American Journal of Orthopsychiatry 67(2): 330.

41.

Vannatta

Gerhardt

(2003) Pediatric oncology: psychosocial outcomes for children and families. In: Roberts

(ed) Handbook of Pediatric Psychology. New York, NY: Guilford Press, pp. 342–357.

42.

Varricchio

(2004) Measurement issues concerning linguistic translations. In: Marilyn

Frank-Stromborg

Sharon

. Olsen (eds) Instruments for Clinical Health-Care Research. Sudbury, MA: Jones and Bartlett Publishers, pp. 56–64.

43.

Whiting

(2013) Impact, meaning and need for help and support: the experiences of parents caring for children with disabilities, life-limiting/life-threatening illness or technology dependence. Journal of Child Health Care 17(1): 92–108.

44.

Wong

Diener

Isabella

(2008) Parents’ emotion related beliefs and behaviors and child grade: associations with children’s perceptions of peer competence. Journal of Applied Developmental Psychology 29(3): 175–186.

45.

Wong

McElwain

Halberstadt

(2009) Parent, family, and child characteristics: associations with mother- and father-reported emotion socialization practices. Journal of Family Psychology 23: 452–463.

46.

Wong

Rindfleisch

Burroughs

(2003) Do reverse-worded items confound measures in cross-cultural consumer research? The case of the material values scale. Journal of Consumer Research 30: 72–91.