Educational and Psychological Measurement

The Mathematics Anxiety Rating Scale (MARS) was submitted to a reliability generalization analysis (RG) to characterize the variability of measurement error in MARS scores across administrations and identify possible study characteristics that are predictive of score reliability variations. In general, the MARS and its variants yielded scores with strong internal consistency and test-retest reliability estimates, although variation was observed. Adult samples were related to lower score reliability compared to other age groupings. Inclusion of total score standard deviation in the regression models resulted in roughly 25% increases in R ² effects.

Logistic regression was used for modeling the observation-to-variable (n/v) ratio required for the standard error scree (SEscree) procedure to correctly identify the number of factors in simulated data. The correlation matrices were generated to possess known characteristics: number of factors (f), number of variables (v), sample size (n), magnitude of pattern coefficients (p), and degree of interfactor correlations (r). The results indicated that under all conditions, the n/v ratio required for the SEscree procedure to correctly identify the true number of factors with high probability exceeded the minimum of 5:1 recommended in some of the related literature. This study demonstrated the ability of the logistic regression to simplify summarizing and reporting findings from simulation studies that involve a large number of conditions.

Teacher efficacy has proven to be an important variable in teacher effectiveness. It is consistently related to positive teaching behaviors and student outcomes. However, the measurement of this construct is the subject of current debate, which includes critical examination of predominant instruments used to assess teacher efficacy. The present study extends this critical evaluation and examines sources of measurement error variance in the Teacher Efficacy Scale (TES), historically the most frequently used instrument in the area. Reliability generalization was used to characterize the typical score reliability for the TES and potential sources of measurement error variance across studies. Other related instruments were also examined as regards measurement integrity.

Autonomy and sociotropy, reflecting excessive concern about autonomous achievement and interpersonal relationships, respectively, have been proposed as the two personality dimensions that prompt vulnerability to depression. Accordingly, several self-report questionnaires have been developed to measure these two dimensions. The revised Personal Style Inventory (PSI) represents a significant improvement over existing measures in that it assesses entire domains of autonomy and sociotropy. However, the PSI’s usefulness has been undermined by its unclear factor structure. Using 574 college students in Korea, the present study examined the factor structure of the revised PSI using hierarchical confirmatory factor analysis. Several first-order and two two-factor hierarchical models were tested, with a hierarchical model specifying perfectionism as associated with both hierarchical factors (Autonomy and Sociotropy) deemed the best fitting.

The present study examined the factorial validity of scores on the Teacher Interpersonal Self-Efficacy Scale across two samples of 416 teachers each. Following self-efficacy theory, which posits that self-efficacy beliefs are linked to specific activities, it was hypothesized that the three Teacher Interpersonal Self-Efficacy subscales comprised three different activities linked to teacher self-efficacy beliefs. Confirmatory factor analysis results from the two samples supported the hypothesis by showing an adequate fit of a three-factor oblique model. This model fits the data better than either a two-factor or one-factor alternate model. Scores from the three subscales yielded coefficient alphas in excess of .90.

The Adolescent Anger Rating Scale (AARS) was designed to (a) measure two distinct types of anger: instrumental and reactive, and (b) assist researchers and practitioners in identifying specific types of anger in adolescents. The present study investigated the construct validity of AARS scores. Seven hundred ninety-two 12-to 19-year-old adolescents in Grades 7 through 12 participated in the study. Factor analysis yielded three factors: Instrumental Anger, Reactive Anger, and Anger Control. Moderate to moderately high Cronbach alphas and test-retest reliability coefficients indicated that scores from the AARS are internally consistent and stable when measuring anger subtypes. Discriminant validity evidence supported the AARS scores’ ability to measure specific types of anger different from constructs of anger measured by the Multidimensional Anger Inventory (MAI).

Although the use of computerized assessment tools in educational and psychological settings has increased dramatically in recent years, limited information is available about the properties of computerized self-concept measures. The authors evaluated the characteristics of computerized and paper-and-pencil versions of the Rosenberg Self-Esteem Scale (SES)—one of the most widely used self-concept measures in educational and psychological research. Results showed that administration mode (computerized versus paper and pencil) had little effect on the psychometric properties of the SES (i.e., score magnitude, variability, and factor structure) but that the computerized version took longer and was preferred by examinees. With the exception of administration time, these results support the use of the computerized SES and its comparability to the paper-and-pencil version.

The purpose of the present study was to examine the internal structure of scores from Bandura’s Multidimensional Scales of Perceived Self-Efficacy (MSPSE). Data were obtained from college freshmen, a population previously untested with MSPSE. Results show that scores from nearly all items aligned almost exclusively with theoretically and empirically derived dimensions. Some differences in empirically derived dimensions occurred as a result of extraction procedures, although the differences observed still provided theoretically interpretable constructs. These results suggest that Bandura’s scales, with slight revisions, show promise for research on self-efficacy with undergraduate college students.

The reliability and validity of a time-domain, amplitude-based automatic Rapid Automatized Naming (RAN) scoring software program were investigated in two studies. The software was designed to mark the articulations and pauses inherent in rapid-naming tasks. In Study 1, the RAN letters subtest digital wave files of 44 primary grade students were scored using the automatic scoring software program. The pause and articulation component durations were statistically compared to pause and articulation durations that had previously been manually scored. Software-scored pause and articulation component times were not found to be significantly different from the manually scored times. Ratings from both software-scored components demonstrated internal consistency and statistically significant correlations with the manually scored components. In Study 2, statistically significant correlations among the computer-scored RAN letters pause component, the RAN letters pause consistency, and concurrent reading ability of 144 second-, third-, and fourth-grade students were demonstrated.

The Child Evaluation Measure (CEM) for preschool children was developed to provide an inclusive assessment instrument that covers primary academic content areas (reading, literacy, mathematics, and science), domains of child development (social, emotional, and physical/motor skills), and other areas foundational for optimal development of the young child in today’s society (music and creative arts, health and wellness, character education, and technology applications). The present study investigated CEM score validity on the 12 subscales and the total measure using data for 301 children assessed on each of two occasions. Both score reliability and score validity analyses were conducted.