The applicability paradigm has been used in five studies to evaluate the applicability of items from two North American instruments designed to measure students' evaluations of teaching effectiveness. In the present investigation, the paradigm is used at the new University of Western Sydney, Macarthur (UWSM). Items from both instruments were seen as appropriate and important, and differentiated among lecturers chosen as good, average and poor teachers. A multitrait-multimethod analysis of responses from the two instruments supported their convergent and divergent validity. The pattern of items judged to be most important at UWSM was more similar to patterns found at two research universities than patterns in the technical and further education sector or at two other institutions. These results support the applicability of the instruments at UWSM and across a diversity of educational settings.
BraskampL.A.BrandenburgD.C.OryJ.C. (1985). Evaluating teaching effectiveness: A practical guide. Beverly Hills, CA: Sage
2.
CampbellD.T.FiskeD.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105
3.
CentraJ.A. (1979). Determining faculty effectiveness. San Francisco: Jossey-Bass
4.
ClarksonP.C. (1984). Papua New Guinea students' perceptions of mathematics lecturers. Journal of Educational Psychology, 76, 1386–1395
5.
CohenP.A. (1980). Effectiveness of student-rating feedback for improving college instruction: A meta-analysis. Research in Higher Education, 13, 321–341
6.
CohenP.A. (1981). Student ratings of instruction and student achievement: A meta-analysis of multisection validity studies. Review of Educational Research, 51, 281–309
DunkinM.J. (1990). Willingness to obtain student evaluations as a criterion of academic staff performance. Higher Education Research and Development, 9, 51–60
9.
FeldmanK.A. (1976). The superior college teacher from the student's view. Research in Higher Education, 5, 243–288
10.
FeldmanK.A. (1983). The seniority and instructional experience of college teachers as related to the evaluations they receive from their students. Research in Higher Education, 18, 3–124
11.
FeldmanK.A. (1984). Class size and students' evaluation of college teacher and courses: A closer look. Research in Higher Education, 21, 45–116
12.
FreyP.W. (1973). Student ratings of teaching: Validity of several factors. Science, 182, 83–85
13.
FreyP.W. (1978). A two-dimensional analysis of student ratings of instruction. Research in Higher Education, 9, 69–91
14.
FreyP.W.LeonardD.W.BeattyW.W. (1975). Student ratings of instruction: Validation research. American Educational Research Journal, 12, 327–336
15.
HarmanG. (1989). The Dawkins reconstruction of Australian higher education. Paper presented at the Annual Meeting of the American Educational Research Association, 27–31 March, San Francisco
16.
HaytonG.E. (1983). An investigation of the applicability in technical and further education of a student evaluation of teaching instrument. Unpublished thesis, Faculty of Education, University of Sydney
17.
HildebrandM.WilsonR.C.DienstE.R. (1971). Evaluating university teaching. Berkeley, CA: University of California, Center for Research and Development in Higher Education
18.
IsaacsG. (1989). Changes in ratings for staff who evaluated their teaching more than once. Assessment and Evaluation in Higher Education, 14, 1–10
19.
MarshH.W. (1977). The validity of students' evaluations: Classroom evaluations of instructors independently nominated as best and worst teachers by graduating seniors. American Educational Research Journal, 14, 441–447
20.
MarshH.W. (1980). The influence of student, course and instructor characteristics on evaluations of university teaching. American Educational Research Journal, 17, 219–237
21.
MarshH.W. (1981a). Students' evaluations of tertiary instruction: Testing the applicability of American surveys in an Australian setting. Australian Journal of Education, 25, 177–192
22.
MarshH.W. (1981b). The use of path analysis to estimate teacher and course effects in student ratings of instructional effectiveness. Applied Psychological Measurement, 6, 47–60
23.
MarshH.W. (1982a). SEEQ: A reliable, valid, and useful instrument for collecting students' evaluations of university teaching. British Journal of Educational Psychology, 52, 77–95
24.
MarshH.W. (1982b). Validity of students' evaluations of college teaching: A multitrait-multimethod analysis. Journal of Educational Psychology, 74, 264–279
25.
MarshH.W. (1983). Multidimensional ratings of teaching effectiveness by students from different academic settings and their relation to student/course/instructor characteristics. Journal of Educational Psychology, 75, 150–166
26.
MarshH.W. (1984). Students' evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility. Journal of Educational Psychology, 76, 707–754
27.
MarshH.W. (1986). Applicability paradigm: Students' evaluations of teaching effectiveness in different countries. Journal of Educational Psychology, 78, 465–473
28.
MarshH.W. (1987). Students' evaluations of university teaching: Research findings, methodological issues, and directions for future research [Special issue]. International Journal of Educational Research, 11, 253–388
29.
MarshH.W. (1988). Multitrait-multimethod analyses. In KeevesJ.P. (Ed.), Educational research methodology, measurement and evaluation: An international handbook (pp. 570–580. Oxford: Pergamon Press
30.
MarshH.W. (1991). Multidimensional students' evaluations of teaching effectiveness: A test of alternative higher-order structures. Journal of Educational Psychology, 83, 285–296
31.
MarshH.W.BaileyM. (1991). Multidimensional students' evaluations of teaching effectiveness: A profile analysis. Manuscript submitted for publication
32.
MarshH.W.FleinerH.ThomasC.S. (1975). Validity and usefulness of student evaluations of instructional quality. Journal of Educational Psychology, 67, 833–839
33.
MarshH.W.HocevarD. (1991a). The multidimensionality of students' evaluations of teaching effectiveness: The generality of factor structure across academic discipline, instructor level, and course level. Teaching and Teacher Education, 7, 9–18
34.
MarshH.W.HocevarD. (1991b). Students' evaluations of teaching effectiveness: The stability of mean ratings of the same teacher over a 13-year period. Teaching and Teacher Education, 7, 303–314
35.
MarshH.W.OverallJ.U. (1980). Validity of students' evaluations of teaching effectiveness: Cognitive and affective criteria. Journal of Educational Psychology, 72, 468–475
36.
MarshH.W.OverallJ.U. (1981). The relative influence of course level, course type, and instructor on students' evaluations of college teaching. American Educational Research Journal, 18, 103–112
37.
MarshH.W.OverallJ.U.KeslerS.P. (1979). Validity of students' evaluations of teaching effectiveness: A comparison of faculty self-evaluations and evaluations by their students. Journal of Educational Psychology, 71, 149–160
38.
MarshH.W.TouronJ.WheelerB. (1985). Students' evaluations of university instructors: The applicability of American instruments in a Spanish setting. Teaching and Teacher Education: An International Journal of Research and Studies, 1, 123–138
39.
McKeachieW.J. (1979, O). Student ratings of faculty: A reprise. Academic, pp. 384–397
40.
MeekV.L.O'NeillA.Organizational change in Australian higher education: Process and outcome. Australian Educational Researcher, 17, 1–23
41.
MosesI. (1986a). Self and student evaluations of academic staff. Assessment and Evaluation in Higher Education, 11, 76–86
42.
MosesI. (1986b). Student evaluation of teaching in an Australian university: Staff perceptions and reactions. Assessment and Evaluation in Higher Education, 11, 117–129
43.
MurrayH.G. (1980). Evaluating university teaching: A review of research. Toronto: Ontario Confederation of University Faculty Associations
44.
NunnallyJ.C. (1978). Psychometric theory (2nd ed). New York: McGraw-Hill
45.
OverallJ.U.MarshH.W. (1979). Midterm feedback from students: Its relationship to instructional improvement and students' cognitive and affective outcomes. Journal of Educational Psychology, 71, 856–865
46.
OverallJ.U.MarshH.W. (1980). Students' evaluations of instruction: A longitudinal study of their stability. Journal of Educational Psychology, 72, 321–325
47.
ProsserM.TrigwellK. (1990). Student evaluations of teaching and courses: Student study strategies as a criterion of validity. Higher Education, 20, 135–142
48.
SPSS. (1986). SPSS-x user's guide (2nd ed.). New York: McGraw-Hill
49.
WarringtonW.G. (1973). Student evaluation of instruction at Michigan State University. In SockloffA.L. (Ed.), Proceedings: The first invitational conference on faculty effectiveness as evaluated by students (pp. 164–182). Philadelphia: Temple University, Measurement and Research Center
50.
WatkinsD.MarshH.W.YoungD. (1987). Evaluating tertiary teaching: A New Zealand perspective. Teaching and Teacher Education: An International Journal of Research and Studies, 2, 41–53
