Test Review: Test of Mathematical Abilities

General Description

The Test of Mathematical Abilities–Third Edition (TOMA-3; Brown, Cronin, & Bryant, 2013) is a norm-referenced mathematics achievement test for ages 8-0 to 18-11 designed to help identify students with math difficulties and use them for research purposes. The TOMA-3 can be administered individually or to a group. However, the authors suggest individual administration if an examiner suspects that a student has math difficulties or other concerns that may affect test performance.

The TOMA-3 consists of four core subtests and one supplemental subtest. Administration requires approximately 90 min. Test materials consist of the examiner manual, record form, and a student response booklet. The student will need several sharpened pencils, the student response booklet, and scratch paper. Use of a calculator is prohibited, and examinees are expected to read each item independently. For each core subtest, examiners read a short instructional script to the student that describes the task and includes a sample item. Instructions can be read from the examiner manual or examiner record form. For the supplemental subtest, Attitude Toward Math, scripted instructions are also provided. However, there is no scripted set of instructions to introduce the sample items presented in the student response booklet. Once the examiner provides the instructions, individual students work independently until a ceiling of three incorrect items is met. If this test is administered to a group of students, each student may work independently for no more than 20 minutes. If the time limit expires and an individual student does not reach a ceiling, that student can return to the subtest items at a later time and complete additional items until a ceiling is met.

Subtest Descriptions and Scoring

The four core subtests are Mathematical Symbols and Concepts, Computation, Mathematics in Everyday Life, and Word Problems. Core subtest items are scored as 1 or 0. Attitude Toward Math subtest items are scored 1 through 4 based on self-report. Subtest results can be described as scaled scores (M = 10, SD = 3), age equivalents, grade equivalents, or percentiles. An overall Mathematical Ability Index (M = 100, SD = 15) is also provided.

The 40-item, multiple-choice Mathematical Symbol and Concepts subtest assesses knowledge of math signs, symbols, words, and phrases. The Computation subtest has 40 items and requires students to complete math problems that increase in difficulty. The Mathematics in Everyday Life subtest consists of 35 multiple-choice items assessing general mathematics knowledge as applied to everyday situations. Word Problems has 30 problems. For the supplementary Attitude Toward Math subtest, students are expected to answer all 15 items. Four response options are provided for each item: Yes, definitely!, Closer to Yes, Closer to No, and No, definitely! The use of double-negatives for some items and response options may be confusing for some students and result in inaccurate reporting of attitudes toward mathematics. For example, one item is, “There’s no reason to take math every year,” and one response option is No, definitely! Results from this subtest are not used to replace a spoiled core subtest and do not contribute to the overall Mathematical Ability Index score. For this subtest, directions on the student response booklet contradict directions on the examiner record form and in the examiner’s manual. On the record form and in the manual, the student is told that the examiner cannot read any part of the problems for him or her, but at the bottom of page 20, on the student response booklet, a note to examiners states that the examiner may read the statements to the students. Thus, it is unclear whether examiners are allowed to read any items on this subtest to a student or not.

Standardization Sample

The TOMA-3 normative sample is comprised of 1,456 individuals, ages 8-0 to 18-11, from 21 states. There were at least 100 individuals at each of the 11 age levels. Data were collected in the winter of 2010 and the summer of 2011.

Overall, the sample appears representative of the U.S. population based on The Statistical Abstract of the United States (U.S. Bureau of the Census, 2011). Sample characteristics such as gender, geographic region, ethnicity, exceptionality status, household income, and parent educational attainment are similar to the Census data. Urban and rural residence data are not reported.

The normative sample also includes individuals from the following disability groups: language impairment, behavior disorder, emotional disturbance, Asperger’s disorder, mental retardation, blind/partially sighted, deaf/hearing impaired, articulation disorder, autistic disorder, and developmental delay. No more than 1% of the sample consists of each of these disability groups.

Reliability

To evaluate reliability, coefficients of .80 were considered minimally acceptable for subtests, and coefficients of .90 were minimally acceptable for composite scores. Based on the normative data, internal consistency results exceeded .80 for each subtest for each age level. Coefficients for the Mathematics Ability Index ranged from .95 to .97 across the 11 age levels. Internal consistency was evaluated for gender, race, and ethnicity, as well as across special groups: gifted and talented, students with attention deficit hyperactivity disorder (ADHD), and those with a mathematics learning disability. Correlations for the various groups ranged from .84 to .95 across subtests. For the Mathematical Ability Index, correlations ranged from .96 to .98. Thus, overall, the results appear to have good internal consistency for these groups.

Test–retest reliability was evaluated for 51 students between the ages of 8 and 17 with an average retest period of 14 days. The corrected reliability coefficients for each subtest were above .80, except for the Mathematics in Everyday Life subtest, which was .73. The multiple-choice format of this subtest may contribute to its lower reliability. The corrected coefficient for the Mathematical Ability Index was .89. Although the reliability for the Mathematics in Everyday Life subtest was low, overall the TOMA-3 results appear stable over time.

Validity

In terms of content validity, item discrimination and item difficulty were used in selecting TOMA-3 items, and potential item bias was evaluated using logistic regression. A total of 435 comparisons were made across three dichotomous groups: male/female, African American/non- African American, and Hispanic American/non-Hispanic American. Results suggested that two items were biased against males and one against non-Hispanic Americans. However, the authors suggested the magnitude of the bias was “negligible,” and the items were retained.

Criterion-predictive validity was evaluated by comparing TOMA-3 results to the Comprehensive Mathematical Abilities Test (Hresko, Schlieve, Herron, Swain, & Sherbenou, 2003) and the Iowa Algebra Aptitude Test–Fifth Edition (Schoen & Ansley, 2005). Correlations were .92 and .83 respectively. In addition, the TOMA-3 has good sensitivity (.95) and specificity (.79) in terms of classifying students with and without a math disability. The receiver operating characteristic/area under the curve (ROC/AUC) value was .87, suggesting the test does well in differentiating between these two groups.

Construct-identification validity was also examined by age, relationships among the subtests, relationship to intelligence, and differences among groups. Results indicate that for the core subtests, scores increase with age for ages 8 to 14 and begin to level off for ages 15 to 18. For the supplemental subtest, Attitude Toward Math, no relationship was found between attitudes toward math and age. Examination of the relationships between subtests suggests that the core subtests are measuring a similar construct, but still independently measure different skills within the construct.

TOMA-3 results were compared with those of the Test of Nonverbal Intelligence–Fourth Edition (TONI-4; Brown, Sherbenou, & Johnsen, 2010). Corrected correlations for the five subtests ranged from .54 to .75. The relationship between the nonverbal intelligence score and the Mathematical Ability Index was large (r = .79). These results suggest TOMA-3 results are related to intelligence as measured by the TONI-4.

Mean standard scores for subtests and the Mathematical Ability Index were compared across gender, ethnicity, gifted and talented, ADHD, and mathematics learning disability groups. Mathematical Ability Index means for the mainstream and ethnic subgroups were within the average range. As anticipated, students with mathematical learning disabilities had the lowest mean scores, and those who were gifted had the highest. In addition, exploratory and confirmatory factor analyses support a single-factor model. Thus, considerable data are presented in support of the validity of this edition of the test.

Scoring

When converting subtest raw scores to scaled scores, tables are listed in 6-month age intervals for individuals 8-0 through 10-11. For individuals 11-0 through 14-11, tables have 1-year intervals, and for students 15-0 through 18-11, one table is presented. As mentioned earlier, performance on the test appears to level off for each subtest between ages 15 and 18, so grouping the four age levels in this last table may be appropriate. Thus, the TOMA-3 results seem to be more sensitive to change for younger than older students.

Floors

The overall TOMA-3 Mathematical Ability Index has an adequate floor beginning at 8-0, but some subtests do not. Mathematical Symbols and Concepts has an adequate floor at 8-0, but Computation does not until 8-6, and the subtests Mathematics in Everyday Life and Word Problems do not until 9-0.