A Test Review: Mathematics Fluency and Calculation Tests (MFaCTs) review,by C. R. Reynolds,J. V. Voress,& R. W. Kamphaus

General Description

The Mathematics Fluency and Calculation Tests (MFaCTs) are a series of measures designed to assess for arithmetic calculation skills and calculation fluency in children ages 6 through 18. There are five main purposes of the MFaCTs: (a) identifying students who are behind in basic math fact automaticity, (b) evaluating possible delays in arithmetic calculation skills, (c) progress monitoring, (d) identifying students with advanced levels of mathematical achievement, and (e) use in research. The MFaCTs was published in 2015 by Pro-Ed Inc. in Austin, TX, and was authored by Cecil R. Reynolds, PhD; Judith K. Voress, PhD; and Randy W. Kamphaus, PhD. The theoretical underpinnings of the MFaCTs measures stem from the belief that mathematical fluency and calculation skills are essential components in the development of math skills in students. With three alternate forms for assessment, the MFaCTs is able to measure mathematical fluency in first grade through fifth and calculation skills in first grade through 12th grade. Due to the construction of the test, the MFaCTs is able to be administered in both group and individual settings. The authors suggest that trained professionals should administer, score, and interpret the results of the test. They mention that teacher aides may administer the measures, but it is suggested that trained professionals should interpret the results. Training is provided through the test manual; no extra training modules need to be completed. A wide range of professionals are able to administer, score, and interpret the MFaCTs, including, but not limited to, school counselors, special education teachers, general education teachers, school psychologists and other related psychologists, educational diagnosticians, neuropsychologists, and any other professional with training in assessment (Reynolds, Voress, & Kamphaus, 2015).

Specific Description

The global format of the MFaCTs includes measuring fluency and calculation skills in school-aged individuals. This is accomplished through a series of math facts provided to the test-taker where he or she is asked to complete the problems to the best of his or her ability. If assessing for fluency, then the test-taker is asked to complete the problems as quickly and accurately as possible. If assessing for calculation skills, then there is no time limit. There are no subtests on the MFaCTs.

Scoring System

The standard score is generated separately for the Calculation and Fluency tests by converting the raw score obtained by the test-taker into a standardized score with a mean of 100 and a standard deviation of 15. The qualitative descriptors of the standard scores can be found in the test manual. The MFaCTs authors provide age/grade equivalents as well in the manual. The grade equivalents were calculated by finding the mean score of students at a specific grade level and point. It should be noted that the age/grade equivalents are not normed, nor do they have equal units along their scale, similar to the percentile rankings, which will be discussed next. The final type of score the MFaCTs provides is percentile rankings. Like age/grade equivalents, percentile rankings are ordinal and do not have equal units along their scale. It should be noted that the test authors recommend using standard scores when comparing scores across tests due to the equal units along its scale.

Test Materials and Stimuli

There are not many test materials for the administration of the MFaCTs. Typically, it involves a booklet and a pencil to take the tests. With that being said, the booklets are well-made and the test manual is sturdy and able to withstand many uses before showing signs of wear and tear. The plastic coverings on the manual are strong and resistant to tearing as well as liquid damage. The pages inside are fairly water-resistant and tear-resistant. The booklets/protocols are well-designed, creating a nice flow throughout the testing process. The booklets are not too cluttered nor are they too spacious. The text is easily read and big enough for all populations to see. One area of improvement for the MFaCTs could be the structural soundness of the box the materials arrive in. It is made from cardboard, is easily bent, and is subject to wear and tear. Throughout my assessment of the test, I did not notice any indications of racial or gender bias. Overall, the test is well-made and should last an appropriate amount of time without needing to be replaced.

Test Manual

The test manual is very clear and organized; it makes navigating and understanding the testing process very simple for even unexperienced administrators. The instructions on administration are convenient and well-explained. I appreciate the bolding of the administrator’s script. In addition, the test manual does a good job at conveying the constructs behind the tests, as well as explaining the validity and reliability behind the measures. The breakdown of the chapters is logical. The wording used by the authors is professional, yet simple enough for understanding.

Test Construction

The tests were constructed with three primary functions in mind: screening, diagnosis, and progress monitoring. Due to this, the tests were created to be brief and simple to score. There are no criteria for ceilings or baselines that need to be established for an accurate score to be recorded. The Fluency tests are timed: 5 min for Grades 1 and 2, and 3 min for Grades 3 through 5; the Calculation tests are untimed. The students are asked to simply complete all of the problems to the best of their ability in the allotted time.

Item Analysis

Item bias was assessed for using differential item functioning (DIF). The analysis indicated that there was less than 1% of the item pool that demonstrated bias, meaning that the usage of the MFaCTs items will not adversely affect any specific subgroup or population, meaning that it is acceptable to administer these tests to various cultural groups.

Standardization Sample

The standardization sample used for the MFaCTs consisted of 1,620 students from a variety of states including Alabama, California, Idaho, Nebraska, New Mexico, New York, North Dakota, Ohio, and Texas. The tests were administered to these individuals from summer 2008 through fall of 2010. In each of the geographic areas, a major city was identified and used as the primary testing site. At each of these sites, students were tested that matched the demographics of the region as a whole. In turn, this method amounted to 766 students being tests. The remaining 854 students were located by additional examiners recruited by the test authors. The test authors made an effort to correlate the percentages of their testing population with the school-age populations provided by the 2011 U.S. Census Bureau. They did a commendable job, exhibiting very little difference (typically 1 to 2 point differentials) in the percentages provided in the test manual. Specific tables can be found in the test manual, but overall, the MFaCTs demonstrates a testing sample that represents the U.S. population well.

Adequacy of Reliability Estimates

When analyzing the internal consistency of the test scores provided by the MFaCTs, all of the reliability values exceeded .90, suggesting that the MFaCTs Fluency and Calculation tests contain consistent items and alludes to idea that the standard scores have little measurement error.

The test authors reported alternate-form reliability as well. With three different forms available for both the Fluency and Calculation measures, this is an important construct to assess. The authors used both age-based norms and grade-based norms to assess the alternate forms for both the Fluency and Calculation measures. The results showed that for the Fluency measures, the medians for younger students was .81 and for older students (ages 8 through 11) it was .88, both surpassing the field acceptable .80. The Calculation measures showed a median correlation of .96 across all age groups, indicating that the various forms can be used interchangeable with little deviation in the scores. There was a similar pattern observed in the grade-based norms as well, with Calculation tests presenting a .94 and a .96 reliability coefficient for Grades 1 through 2 and Grades 3 through 5, respectively. The Fluency measures provided a .81 and a .88 for Grades 1 through 2 and Grades 3 through 5, respectively. Provided in the test manual are the results from the alternate-form delayed reliability measures. The results show above .80 coefficients for Fluency and Calculation, with the exception of Fluency for younger students. The younger students on Fluency demonstrated an alternate-form delayed reliability of .64 and .72 for age-based and grade-based norms, respectively. The attributions of these lower scores will be discussed briefly in the next paragraph when similar results arise. Overall, the alternate-form reliability data suggest that the A,B, and C forms can be used interchangeably to a reasonable degree, although Fluency less so that Calculation.

In addition, the test authors provided information regarding test–retest reliability. This construct is an important concept for progress monitoring; one of the primary uses of the MFaCTs. The authors utilized a test–retest interval range of 1 to 2 weeks. The tables can be found in the manual, but overall, all of the test–retest coefficients landed above the .80 mark, with the exception of Fluency scores for the younger students, particularly ages 6 through 8 and Grades 1 and 2, which landed in the .70 and above region. The fluctuation in scores for the younger students can be attributed to the rapid growth seen in students during this developmental period and the various characteristics of younger students, such as inattention.

Overall, these data suggest that the scores generated by the MFaCTs are a reliable measure for the population it reports to assess; however, there should be some caution when interpreting results of the Fluency tests for younger students due to the malleability of their cognitions and dynamic knowledge base.

Adequacy of Validity Estimates

The test authors provide three aspects of validity estimates for the MFaCTs: evidence based on test content, evidence based on relations to external variables, and evidence based on consequences of testing. The content validity of the MFaCTs demonstrates the idea that both the Fluency and Calculation tests provide valid measurements of the skills they report to assess. The authors discuss how the values of the correlations cannot be too high (in the .80s), because that would allude to the idea that they measure the same concept, when in actuality, fluency and calculation are related, but not so much that they should be highly correlated. The data show that there are lower correlations in the lower grades (Grades 1 and 2) for Fluency and Calculation than for Grades 3 through 5 where there are higher correlations (in the .50s and .60s). These data dictate that the two tests are measuring relatively independent skills in the earlier grades. The median item discrimination indices provided by the researchers were all positive and above the .40 threshold, with many far exceeding it, supporting the belief that these tests measure mathematical skills using moderately difficult questions. This is accomplished by including test items that differentiate students with mastery from those who do not have the required knowledge to answer correctly. Finally, the hypothesis that math skills should continue to increase over the years with a rapid growth spurt in early ages is supported through the correlations on both the Fluency and Calculation tests. In early grades for Calculation, the raw scores correlate highly with age in elementary (.70s) and begin to trickle down into the .20s as the student go into high school. Similarly, the Fluency tests exhibit significance at the early grades and ages (in the .30s) and trickle down into the .20s as students approach the higher grades.

The authors also measured the validity of the MFaCTs to two other similar tests, the Comprehensive Mathematical Abilities Test (CMAT) and the Wide Range Achievement Test–Fourth Edition (WRAT-4). The calculation subtests were used to compare the calculation for the MFaCTs. The results from the correlations between the various measures were good, indicating that the MFaCTs Calculation test measured the appropriate construct. The correlation between the MFaCTs Calculation test and the WRAT-4 Arithmetic test ranged from the .60s to the .80s on both the age-based and grade-based norms. Similarly, the correlations between the MFaCTs Calculation test and the CMAT were high, with most of them above the .70 mark.

Receiver operating characteristic (ROC) curve analysis was utilized to estimate the predictive accuracy of the MFaCTs in classifying children at risk for a learning disability. All of the areas under the curve, as produced by the ROC analysis for the Calculation test, landed in the .80 and above region regardless of the form. This reveals that the Calculation test is a relatively good at determining the presence of a specific learning disability in mathematics. Despite the success of the Calculation test, the ROC analysis of the Fluency test provided no support for it being a good determiner of specific learning disabilities in mathematics.

Overall, these data suggest that the Fluency and Calculation tests lead to valid decisions of the respective math skills and can be used for progress monitoring as demonstrated by the correlations with other measures. The ROC curves indicate that the MFaCTs is an appropriate measure for screening and the DIF studies show that it will not produce an unwarranted positive or negative impact on the population.