Language Testing

Based on the observation that scale-level methods are sometimes exclusively used to investigate measurement invariance for test translation, this article describes the results of a simulation study investigating whether item-level differential item functioning (DIF) manifests itself in scale-level analyses such as single and multi-group factor analyses and per group coefficient alpha. The simulation factors were two levels of DIF (moderate and large) and four levels of percentage of items with DIF (ranging from approximately 3-41% of the items). The results indicate that item-level DIF did not manifest itself in the scale-level results. Clearly, then, translation efforts in language testing should ensure measurement equivalence by investigatingitem-level translation DIF, and it may be misleading to give consideration only to the scale-level methods results as evidence of translation equivalence.

Activity in the area of language testing is expanding beyond second language acquisition. In many contexts, tests that measure language skills are being translated into several different languages so that parallel versions exist for use in multilingual contexts. To ensure that translated items are equivalent to their original versions, both statistical and qualitative analyses are necessary. In this article, we describe a statistical method for evaluating the translation equivalence of test items that are scored dichotomously. We provide an illustration of the method to a portion of the verbal subtest of the Psychometric Entrance Test, which is a large-scale postsecondary admissions test used in Israel. By evaluating translated items statistically, language test-developers can ensure the comparability of tests across languages and they can identify the types of problems that should be avoided in future translation efforts.

An extensive adaptation exercise was undertaken by the Australian Council for Educational Research (ACER) when it was decided to modify the 1988 version of the American ‘SIGI Plus’ (System for Interactive Guidance, plus more) (Educational Testing Service, 1981 and later dates) for Australian conditions. The information is of several kinds, requiring varying degrees of validation and checking for local appropriateness.Although the same language, English, was involved, there are substantial differences in education systems and educational pathways, and noticeable differences in occupational conditions and people’s value systems.

Adaptations were needed at the level of the language (terminology), factual details relating to the vocational world (starting salaries, workplace conditions, legislation), and the educational world (educational programs, extracurricular facilities, further education). The self-assessment modules had to be adapted and then validated against Australian occupational criteria. Teams comprising occupational psychologistsfrom government departments, career teachers, and counsellors both from educational and corporate settings adapted and verified all information and assessment components. Finally, all adaptations needed to be implemented in the computer program, requiring frequent and thorough system testing. The steps in the adaptation and validation procedures are outlined following a brief discussion of elements of career guidance and a description of SIGI Plus itself.

The Test for Creative Thinking - Drawing Production (TCT-DP) is a creativity test, suitable for most age and ability groups. It was developed by Jellen and Urban (1986) and has been used mainly in Europe. This study explores the possibility of implementing TCT-DP amongst kindergarten children in Korea. Results from a TCT-DP field study on 1366 Korean pre-school children are comparable to those found in previously reported German studies of TCT-DP. The article presents steps taken to adapt TCT-DP to a different culture. These steps include: test translation, test implementation, scoring, selection and training of test agents and graders, norm construction, and score interpretation. The findings provide useful information for practitioners wishing to adapt foreign psychological tests such as TCTDP to different cultures, especially in relation to the assessment of pre-school children.

The use of translated versions of formal educationalassessments is becoming more common in the United States of America (USA). This is due to two converging factors:

1) legislation passed during the Clinton and Bush administrations;

2) an influx of non-English-speaking immigrants, especially from Latin America. Against this background, this article discusses theoretical and practical issues pertaining to the translation or adaptation of educational assessments in the USA. These issues include the role of language proficiency and academic background in performance on standards-based achievement tests in different languages, factors affecting the decision whether to translate tests, translation methods and procedures, the degree to which translated tests are used in the USA, current laws influencing the use of translation, and the role that language testing specialists may play in this arena.

The first cycle of the OECD Programme for International Student Assessment (PISA) was conducted in 2000. Students in 32 countries were surveyed to assess their reading literacy, mathematical literacy and scientific literacy. It is essential in an international study of this type that the assessment materials be culturally appropriate for the many participating groups. They must also be linguistically equivalent, and make sense psychometrically for individual countries as well as for the group of participants as a whole. This article discusses the steps taken to attain cultural relevance and appropriateness in the reading literacy construct, and in the stimulus materials and items which operationalizeit. It explains the influence of multilingual considerations on the development of the reading literacy assessment instrument and notes some psychometric procedures used to maximize the validity of the instrument in an international context.

As a part of the OECD Programme for International Student Assessment (PISA), an international survey was conducted in 2000 to assess the Reading, Mathematics and Science literacy of 15-year-old students in 31 countries (see McQueen and Mendelovits, this issue; see Grisay, 2002). The article describes the procedures implemented by the PISA International Co-ordination Centre for the development of national versions of the assessment instruments in all instruction languages used in the participating countries. It also presents data (collected during the field trial of the instruments) that provide some empirical information on the effectiveness of these procedures. The International Centre developed two source versions (in English and French) of the instruments. It was recommended that the national adaptation teams produce two independent translations (one from the English and the other from the French source version) of the assessment material into the language of instruction in their country and that they reconcile them into a single national version. A group of international verifiers appointed and trained by the International Centre then checked the equivalence of all national versions against the source versions.