Educational and Psychological Measurement

A re-examination of Spearman's (1910) paper led to the conclusion that he did not subscribe to a platonic distinction between true and error scores, or to the restrictive and empirically intractable postulate of constant true scores in repeated measurements with the same test. He argued instead for the reality of possible systematic variations in observed scores under repeated measurements, variations which required determination and not elimination, with an overlay of nonsystematic variations due to factors whose modes of operation were such as to baffle investigation and control. Spearman's distinction was a pragmatic one between the predictable and the unpredictable with theoretical overtones hitherto overlooked in psychometrics, and contained an implied plea for an experimental psychometrics.

A test was developed so that right and wrong answer subtests could be compared. A clear pattern emerged which supported the a priori construct validity assumptions. The results were sufficiently conclusive to apparently refute the assumption often made or implied that "wrong answers contain no achievement information not included in the right answers or subtests thereof."

Empirical .05 and .01 rates of Type I error were compared for the Tukey and Scheffé multiple comparison techniques. The experimentwise error rate was defined over five sets of the all possible 25 differences of averages contrasts. The robustness of the Tukey and Scheffé statistics was not only related to the type of assumption violation, but also to the sets containing different numbers of contrasts. The Tukey method could be judged as robust a statistic as the Scheffé method.

A measure of the magnitude of the effect in a one-factor multivariate analysis of variance design is considered. Cooley and Lohnes have proposed the use of the quantity (1 — | W |/| T |) as a multivariate extension of the correlation ratio, where | W | is the determinant of the within-groups cross-products matrix and | T | is the determinant of the total cross-products matrix. The measure is based on the use of | W | as the estimate of a generalized measure of within-groups variation and | T | as the estimate of a generalized measure of total variation. If a multivariate correlation ratio is defined as the proportion of variance in the multivariate domain predictable from the factor, it is argued that cr_M = 1 - Tr( WW ^-1)/ Tr( TW ^-1) is a more suitable multivariate generalization of the univariate correlation ratio.

A method for the multidimensional scaling of dichotomous item data is presented which is derived from ordering theory. This method is related to the methodological multivariate extension of Guttman's scalogram analysis developed by Coombs and his students. An example is provided and comparison is provided between the data analytic results of the ordering theoretic method and those of the method of Coombs and his students for their conjunctive model. Some relationships of this method to conventional psychometric data analytic procedures are discussed.

The effect of the number of scale intervals of a continuous variable on the results of principal components factor analysis was investigated. Analyses were performed for seven different numbers of scale intervals. The general effect was a decrease in the size of the eigenvalues, communalities, and factor loadings as the number of scale divisions was reduced. The magnitude of the effect was, however, not large and the pattern of the rotated factor loadings was not appreciably affected.

Several goodness of fit measures for factor analysis were applied to the results of maximum-likelihood factor analyses of data with known factors. Results showed that p was a fairly good measure of the degree to which the factor analytic model held in the population, although some values were large for smaller sample sizes. Two measures, c and r, were used to compare the obtained sample factors with the population factors. These measures could be used in confirmatory studies, when an hypothesized factor matrix could be written down. They showed that fairly good agreement with an hypothesis can be obtained, even when the factor analytic model does not exactly hold in the population. This would require that the investigator have a large sample size, variables with high communalities, and a low ratio of number of factors to number of variables.

Theoretical problems with the factor analysis model have resulted in increased interest in component analysis as an alternative. It is therefore of interest to assess empirically some of the asserted differences between the methods. One of the positive features sometimes attributed to factor analysis, as opposed to component analysis, is superior stability of results under sampling from a population of variables. The comparative stability of three methods, Maximum Liklihood Factor Analysis, principal component analysis, and rescaled image analysis, is investigated. Random samples are drawn from a "population" of variables. The sample pattern is then compared with the corresponding population pattern. The results suggest that none of the three methods can be considered superior with respect to stability.

The connection between highest purpose and lowest objective may be visualized as a multi-level hierarchy or tree. Most measurement work emphasizes the actual test item or behavioral objective, but such a "bottom-up" approach is here criticized as uncontrollable. A more feasible "top-down" approach begins with the definition of highest value, here called bentee (for benefit T-score). This measure is a normalized and weighted sum of a profile of test scores. The weighting may be done at each node of the tree for the branches below, by judges selected for authority or expertise. It may be done for a total society, or for specific groups, occupations, or individuals. Such a tree has mathematical properties similar to those for a tree of independent probabilities.

A long and widely held belief has been that a distribution can be assumed symmetric if the mean and median are coincidental. Unfortunately this is not the case. Coincidence of mean and median is a necessary but not sufficient condition for symmetry. Two counter examples proving this point are provided. Plotting of the data and third moment computation are recommended alternatives.

Two groups of subjects were asked to fake good on selected MMPI scales, and to estimate the social desirability of 100 trait adjectives. These two tasks were counterbalanced for order. A control group was administered the same MMPI scales under self-report instructions and instructed to judge the social desirability of the 100 trait adjectives. The first two groups of subjects were also given 17 trait adjectives in pair-comparison format and asked to judge the difference in social desirability for each trait-pair. A multidimensional scaling analysis of the subjects' group average difference judgments among the trait words indicated three dimensions, labeled General Social Desirability, Dominance-Submission, and Excitability-Control. Individual differences were uncovered in the weights which individual subjects gave to these three dimensions. These weights were found to be related to MMPI scales showing specifiable change under faking instructions, and to the accuracy of social desirability estimation.

A 30-item multiple choice word analogy test and a corresponding 30-item picture analogy test (in which the pictures corresponded to the words in the word analogy test) were administered to 289 civil service clerical employees. The equivalence of semantic (word) and figural (picture) presentation of the same item was determined by comparing the responses of each subject on each item pair. The proportion of correspondent responses (both items answered correctly or both items answered incorrectly) ranged from .69 to .91 with a median of .84. The proportion of correspondent responses significantly exceeded the expected chance proportion on 27 of the 30 item pairs. The correlation between scores on the two test forms was .86. Over 84% of the respondents gave correspondent responses with greater than chance frequency. Score distributions were practically identical. It was concluded that semantic and figural parallel test forms can be constructed.

This study investigated the effects of giving subjects additional information about personality tests. One group received only standard instructions; a second group also had an opportunity to examine typical items; a third had, in addition, an oral exposition of one variable—either the Marlowe-Crowne Social Desirability Scale or the Similes Preference Inventory. Additional information was associated with slightly more positive attitudes but did not increase response homogeneity or the uniformity of subjects' perceptions of the variable being measured. It was concluded that, when a subject is given the task of describing himself, his responses are determined primarily by his interaction with each item, rather than by general attitudes elicited by the instructions.

The effect of social desirability response set on a measure of individual differences in stimulation—seeking—the Sensation-Seeking Scale—was studied in a 2 × 3 × 5 design consisting of factors of sex, instructions (fake good, fake bad, and control) and five subscales, with 15 undergraduate Ss per cell, making a total of 90 Ss. No significant effect of instructions was found, nor were any interactions of any variables significant. However, a significant (p < .01) main effect associated with sex was obtained, with males scoring significantly higher than females. In addition, a significant (p < .01) main effect of subscales was found, indicating significant variation among subscale means. It was concluded that response set, as aroused, was not a significant factor in this measure.

The article presents the development of an instrument to assess need satisfaction among lower socio-economic populations. The four most basic levels of the Maslow need hierarchy were considered—physiological, physical safety and security, affection, and self-esteem. Each level was represented by a subscale. Pilot testing and revision of initially-constructed items was performed. Finally, 200 delinquent adolescents were administered the Need Satisfaction Schedule (NSS). Stability was demonstrated for each subscale and the total schedule (p < .041). Internal consistency was relatively higher for the upper levels of the hierarchy and for the total NSS. Convergent validity of the subscales was established (p < .005). Nomological validity was established by the use of a correlation matrix approach. Utilization of the NSS among lower socio-economic was discussed.

A multivariate analysis of variance computer program is presented which employs general linear regression methods to obtain solutions consistent with any of the available methods for dealing with unequal and disproportionate cell frequencies. Documentation is provided.

Three different statistical procedures for identifying aptitude-treatment interactions (ATI's) are discussed with regard to statistical power. The three procedures are (1) treatment × blocks analysis of variance, (2) homogeneity of group regressions and (3) extreme groups analysis. The treatment × blocks analysis of variance offers the least power while the extreme groups analysis offers the greatest power. A general discussion of statistical power is presented along with a FORTRAN function for the calculation of power. A FORTRAN program is also provided for the calculation of extreme groups analyses. As a special feature, this program allows the researcher to determine what form of extreme groups analysis will provide maximum power.

The technique of order analysis was proposed as an inferential alternative to factor analysis. The method reconceptualizes the problem of analysis of data matrices from the perspective of formal logic. Dominance matrices preserving the direction of variance were used in lieu of the correlation matrix of factor analysis. The roots of linear equations, as dimensionality indicators, were supplanted by branching routines developed within the context of graph theory. The program provides for graphic construction of the manifest and latent structures of the data matrix and does not analyze patterns of random variation.

A computer program to be used in interpreting factor analysis results has been written. Given a factor matrix and item descriptions, the program will produce, for each factor, a printed display of the factor loadings and descriptions of items which load significantly on the factor. The display for each factor may also include, for each item listed, a summary of the other factors on which the item loads significantly.

An easy to use program for the interactive solution of linear models is described. The program allows one to enter and to edit simply the complex design matrix, and then separately to solve the linear model. The interactive nature of the program results in a great saving of time for the student. In addition, teleprinter output is timed carefully so that one has the feeling of actually doing the computations.

Initial uniqueness estimates given by the diagonal matrix U ² = I are suggested as a useful starting point in carrying out by computers canonical factor analyses.

This program for three-factor analysis of variance with repeated measures on two factors is based on an unweighted means analysis, permitting the use of unequal sample sizes. Each factor may be fixed or random since the program will select an appropriate error term or will calculate a quasi-F ratio. The number of possible levels of each factor is extended over those in other programs. Rounding errors are minimized.

The program performs Hotelling's T ² for single or related samples. It has utility as a research tool in that it follows the test with any of three post-hoc analyses: simultaneous confidence intervals for individual variables, for user-supplied linear compounds, and for the optimum linear compound of the variables. Also, the program has utility as an instructional tool in that it tests more than one hypothesis for a given data set and provides corresponding univariate tests for comparison. Finally, the program generates the equation of the confidence region for the population centroid and determines the relative location of each hypothesized centroid.

In an introductory psychology class of size 200, a computer-based testing and scoring system was tried out. Individual test forms were generated by a random process from an item pool and were printed by computer; test-response data were scored by computer; and test-score records for each student were maintained by computer. Advantages included: (1) increased frequency of tests, (2) immediacy of feedback, (3) accessibility and currency of cumulative records, (4) decreased opportunities for cheating, (5) decrease in difliculties and extra work associated with retests and make-up examinations, (6) decreased need for typing and secretarial services, (7) shortened lead time between initial stages of test preparation and administration, and (8) autonomy of the process, as it required little high-level decision making once in progress. Cost analysis indicated that such a system would be feasible for adoption by an individual instructor, given the required facilities at the university computer installation.

This paper presents a FORTRAN program that computes all the indices (such as indirect-direct ratio and extended direct) and all the cells, by individual subject and by totals, of the Flanders' Interactional Matrix. The output consists of printed as well as punched output.