Abstract
In Queensland, school assessment scores at the completion of secondary school are adjusted to achieve a measure of statewide comparability across subjects and across schools. All students whose scores are to be scaled complete a scholastic aptitude test which serves as the common reference test for all subjects. Inevitably such tests are to some extent fallible. Unreliability in the reference test introduces unwanted error into the scaled scores. Several Monte Carlo experiments were conducted to estimate the seriousness of this error for various student group sizes and for a range of student abilities. The results show that the error is not negligible for small and moderately sized groups. Care is therefore needed in interpreting and making decisions based on small differences in single and aggregated adjusted scores, especially for students of high ability.