Abstract
To ensure the most successful instruction of children with learning disabilities, the teacher must have considerable understanding of each pupil’s psychoeducational strengths and weaknesses. No superficial overview of a child’s performance will suffice. Knowledge of IQ, reading grade level, or neurological status, while interesting, does not provide enough information with which either to establish appropriate goals or to construct a reality-based training program for a specific child. Therefore, the implementation in the schools of an effective, educationally oriented evaluation program is of the utmost importance for a successful instructional experience for children with learning disorders.
The Total Evaluation
All activities that contribute information or data to the teacher’s knowledge of the child and his problem constitute the total evaluation process. It is this information that is synthesized and used to formulate an appropriate instructional intervention for a particular child. The elements of the total evaluation fall naturally into two divisions: the administration and interpretation of standardized tests (the formal evaluation) and the use of informal diagnostic techniques (the informal evaluation). While the teacher may administer an occasional test, he is more likely to depend upon the informal approaches to assessment, leaving the bulk of the formal testing to other school personnel.
The aims of the total evaluation are (1) to identify children who are likely to have trouble in school; (2) to refer children for medical or psychiatric attention when needed; (3) to isolate specific areas of difficulty, such as perceptive and motoric disabilities, language disabilities, academic deficiencies, and mild concomitant emotional behaviors; and (4) to probe in depth the parameters of these particular problems. Therefore, because of the breadth of the information needed, it is highly unlikely that a single individual will possess the necessary skills or time to manage the total evaluation. Instead, ideally, the total evaluation should be a joint venture to which the school psychologist, teacher, speech therapist, remedial-reading specialist, and auxiliary personnel, such as the physician, optometrist, social worker, etc., contribute their unique abilities. Unfortunately, in actual school practice, even where the team approach is used, the total evaluation rarely reflects an educationally relevant focus and often is accomplished for the sole purposes of labeling, placing, or referring youngsters who fail in school. While these are essential functions, in today’s schools, they are nonetheless of extremely limited value to the teacher who must prepare a daily program for the child.
If the information that is acquired during the total evaluation process is ever to be translated into instructional action, the teacher must be recognized by the school as a primary contributor to, and interpreter of, the assessment results. It must be kept firmly in mind that where a particular child’s educational problem is concerned, the teacher observes more of his learning behavior than anyone else. Therefore, it is reasonable for the teacher of learning-disabled children to participate in a meaningful way in the evaluation process. Where the major assessment effort does not contribute directly to better instruction for children, the total evaluation concept does not exist.
Let there be no mistake concerning this position: I do not maintain that teachers should administer IQ tests or other batteries, such as the Wechsler Intelligence Scale for Children (WISC), 1 the Illinois Test of Psycholinguistic Abilities (ITPA), or projective instruments— although this is not an unheard-of idea. I do suggest that teachers learn to incorporate the findings of such measures with the much more important data obtained through their own critical, diagnostic teaching. I have reached the conclusion that the teacher of learning disabled children must assume responsibility for a considerable portion of the total diagnostic effort; and, that it is unreasonable to expect the school psychologist to write an “educational prescription,” which the teacher dutifully implements in the classroom. Few psychologists possess teaching experience in learning disabilities, nor do they have familiarity with the wide variety of potential intervention strategies, nor do they see the child long enough to identify with surety the subtle aberration of educationally significant behavior—all of which are fundamental to the preparation of a viable “prescription.”
If assessment results that are meaningful for school use are to evolve from the total evaluation, the teachers and others charged with the diagnostic function must recognize that instruction and evaluation are not separate worlds, but that they are inseparably meshed. Successful teaching, namely, teaching where the child learns, is, in itself, a reflection of a series of effective teacher assessments. This point of view is essential if the child is to be helped in a maximal fashion in the classroom.
The Formal Evaluation
The formal evaluation is that part of the total diagnostic process (1) that is characterized by the use of standardized tests, (2) that is administered by specially trained persons, (3) and that is usually made in settings other than the classroom. The information acquired is of a decidedly quantitative nature and tends to compare a given child’s performance with national or regional normative data. The results, therefore, are often reported in terms of quotients, scaled scores, grade equivalents, or percentiles. In general, an attempt is made in the formal evaluation to assess many areas of mental function, including intelligence, language, academic achievement, speech, perceptual-motor skill, and social-emotional development. For this purpose, the following are examples of the most commonly used tests in the schools: intelligence—WISC, Stanford-Binet Intelligence Scale (revised), Slosson Intelligence Test for Children and Adults; language—ITPA, Mecham Verbal Language Development Scale; achievement—group achievement batteries, such as the California Achievement Tests, Metropolitan Achievement Tests, and Stanford Achievement Tests, as well as individual achievements tests, such as the Wide-Range Achievement Test, Durrell-Sullivan Reading Capacity and Achievement Tests, Gates Reading Readiness Scales; speech—tests of articulation, such as The Templin-Darley Tests of Articulation; perceptual-motor—the popular and ubiquitous Bender Visual-Motor Gestalt Test, Marianne Frostig Developmental Test of Visual Perception, Wepman Auditory Discrimination Test, Graham-Kendall Memory for Designs Test, or Benton Visual Retention Test.
In school practice, the psychologist usually collects the results of the tests, which he or others have administered, and integrates them into a report for teacher and parents. As you know, this report will vary widely in educational relevance. At best, the findings will (1) eliminate or confirm the presence of mental deficiency, (2) point out general areas and levels of failure in such subjects as reading, spelling, and arithmetic, (3) indicate possible areas of language deficit, (4) demonstrate modality strengths and weaknesses, (5) identify patterns of disruptive and undesirable behavior, (6) recommend areas for diagnostic teaching, and (7) request feedback from the teacher.
At worst, the formal evaluation is instructionally useless and will (1) demonstrate the obvious, namely, dwell at length on what is already vividly apparent to the teacher, (2) stress excessively etiological factors, such as brain dysfunction, which are of no value to the teacher, or (3) dwell at length on the interpretation of minimal and dubious evidence.
During my second year of teaching, I recall being extremely frustrated by a nine-year-old youngster who could not or would not read. In spite of the fact that he was apparently of normal intelligence, I had completely failed in my efforts to teach him even the fundamentals of the reading process. Naturally, he was referred for a formal evaluation. As the district, during those years, did not provide comprehensive psychoeducational services, the pupil was sent to a private diagnostic clinic in the community. Two months (and fifty dollars) later the report arrived. The contents can be summarized into two salient points: the pupil was diagnosed as (1) “dyslexic” and (2) “brain injured.” As the original complaint specified reading as the problem and as the pupil was rather obviously a spastic hemiplegic, the report was useless.
Fortunately, most reports are of considerably more educational value. This is especially true when the reports are interpreted by the examiner or reinterpreted by the teacher into an education framework. However, no matter how exhaustive the formal testing of a child has been, the teacher will need additional information. This need arises from numerous factors, most of which are inherent in the formal evaluation approach. Therefore, before the teacher can integrate the findings of diagnostic teaching with the results of formal assessment, he must be aware of the shortcomings of the latter.
There are several specific limitations of formal testing. The first limitation deals with the sparsity of in-depth information that can be derived from tests. The other limitations are concerned with problems of false positive and false negative diagnoses, which commonly result from formal evaluations. A false positive diagnosis occurs when the results of the evaluation report that the child evidences a particular problem when, in fact, he does not. Conversely, in the false negative cases, the child does, indeed, have a serious problem, which was not detected during the evaluation process.
Lack of Test Information
Standardized tests may be useful for indicating specific areas of deficit; unfortunately, they do not often provide the critical, detailed information upon which an educational strategy can be based for an individual child. For example, low performance on the Wepman test may suggest a sound-discrimination difficulty; however, the results do not specify the particular sounds in need of training. With regard to reading, there is no standardized test that yields the relevant kinds of information provided by the informal reading inventories. The use of these devices provides essential data regarding the child’s independent reading level, his instructional level, his frustration level, and his level of listening comprehension. In addition, most inventories distinguish between word recognition skill and reading comprehension ability, and they measure the child’s oral and silent reading speed as well.
Intrasubject Variability
Children, especially children with learning disorders, vary markedly in day-to-day performance. This variability is reflected in their test scores and often results in an apparent inconsistency between testing-room and classroom behavior. A child with a measured IQ of 85 on Monday, can easily have a quotient of 100 or 70 when retested on Tuesday, where the discrepancy is caused by pupil attitude or temperament. What is thought to be an extremely disabling visual-motor deficit, as evidenced in an ITPA profile of one week, has vanished or is hardly detectable two weeks later. To be removed from class, led by the hand down the hall into an examination room, introduced to a stranger who is going to do something that is not quite understood, and then to be left alone, is indeed a strange and threatening experience for the child. If he reacts to the situation rather than to the tests, it is certainly understandable. Diagnostic teaching, over days or weeks, in the familiar classroom, permits the teacher to more fully fathom the learning-disabled child’s actual performance level.
Not all of the variability in testing is associated with the subject. Examiners vary daily in their patience, temperament, and skill; and by no means can one assume that an examiner, even a highly trained examiner, is in all cases free from error. This point has recently been demonstrated by C. K. Miller, N. M. Chansky, and G. R. Gredler, 2 who gave a completed WISC protocol to thirty-two psychologists in training, all of whom had completed the course work in the administration and scoring of the WISC. These individuals scored the protocol in a most dissimilar fashion; the resultant Full Scale IQs ranged evenly from 76 to 93.
Overgeneralization of Findings
From time to time, one encounters children who fail, for example, the visual-motor items on a test and become “visual misperception” suspects. Yet, in their school work, little or no disability is noted in that modality; and, the teacher is surprised by the application of the label. This situation can occur when one generalizes the failure on one visual task to probable failure on all tasks in that channel. An example, in the assessment of auding, is the ITPA subtest of Auditory Reception. In this subtest, the child is asked questions that are graded in increasing difficulty, such as “Do dogs fly?” to which he responds “yes” or “no.” In terms of the ITPA model, the task is Representational Level, Auditory Channel, and Receptive Process. There exist a score or more of similar but different tasks that are identical to Auditory Reception in level, process, and channel; for example, most tests of listening comprehension, which are frequently included in group measures such as the Metropolitan Readiness Tests (MRT). One cannot predict with confidence a particular child’s performance on the MRT Listening subtest from his score in Auditory Reception; and, it is hazardous to diagnose a generalized auditory deficit on the basis of performance on any one test.
With this last point, all psychoeducational examiners would probably agree. Still, lists of remedial suggestions that are based upon low performance on a single test find their way into reports. Only last week, a case came to my attention where a “pronounced visual-motor problem” was diagnosed and a “remedial program geared to these perceptual-motor difficulties” was prescribed; all this on the basis of poor performance on the Draw-A-Person and the Bender Gestalt Test. Yet the same evaluation provided ample contradictory findings, which were not mentioned at all. For example, the WISC Verbal IQ was 99; the Performance IQ, which is comprised almost entirely of visuomotor items, was 103. The child actually had scale scores of 14 on Mazes, 10 on Coding, and 12 on Block Design. Does the child, in fact, have a visual-motor problem?
Low Test and Subtest Reliabilities
A third reason for false positives and negatives in the formal assessment of children emanates from the comparatively low reliabilities of many standardized tests. For example, Auditory Reception is reported in the ITPA manual, by J. N. Paraskevopoulos and S. A. Kirk (see references), to have a test–retest reliability coefficient of .63 when used with six-year-olds. Six of the twelve ITPA subtests have stability coefficients equal to or below that figure. In the manual of the Marianne Frostig Developmental Test of Visual Perception, it is recommended that individual training exercises be initiated on the basis of a child’s performance on each of the five subtests. Yet the test–retest reliability coefficients of these subtests range from .33 to .83 at kindergarten and from .40 to .67 at the first-grade level. The statement could be made that most presently available, standardized “diagnostic” batteries lack the necessary reliabilities upon which to base effective educational interventions for individual children.
Other Limitations
Occasionally, even when the child’s test performance does adequately reflect his ability, difficulties can occur in interpretation. For example, in converting raw scores into intelligence quotients on the Peabody Picture Vocabulary Test, a child with a chronological age (CA) of 5-5 who scored 49 has an IQ of 99, while a CA of 5-6 and an identical score of 49 will yield an IQ of 87—an IQ difference of 12 points. The teacher should recall that test scores have standard errors of measurement; subsequently, each score for a child cannot be interpreted as an absolute value. A score is nothing more than an estimate of the child’s ability on a particular test. His “true” ability will most likely range above or below that figure. If scores are interpreted absolutely, the discrepancy between a child’s performance on two tests may appear greater than it is in fact. This point is demonstrated in the 1961 ITPA manual (p. 102). The Visual and Auditory Decoding scores, if interpreted rigidly, suggest a one-year discrepancy between tests. If the standard errors of measurement are applied to the scores, the similarity between the two is graphically evident.
A final limitation concerning formal testing involves the questioning of the use of tests with groups of children who differ from the standardization population. In manuals, reports of item analyses, validity and reliability coefficients, and administration procedures are meticulously reported. But they are generally based upon the performance of “normal” or “representative” youngsters—the very children with whom we never use the tests. The ITPA normative sample was devoid of both bright and dull children. The Frostig test sample included no lower class or black children. Even where the sample reflects the general population, there is little evidence that the test will hold up when used with children who are called mentally retarded, hyperactive, perceptually handicapped, distractible, learning disabled, etc. Are the test reliabilities and validities affected when the measure is used with these youngsters? Generally, the answer is yes. For example, as the IQ of the sample declines, the validity and reliability coefficients are reduced correspondingly; but no mention of this is made in most test manuals. Subsequently, examiners continue to interpret subtests as if the standardization data reported in the manual were applicable—it often is not. When the WISC was factor analyzed using normal and mentally retarded children, different factor structures emerged. 3 This suggests that procedures for interpretation of subtest results may be different for different samples of children.
If these limitations are kept in mind, coupled with the understanding that diagnostic, informal procedures are the next step in the total evaluation process, the results of formal assessment can contribute to the educational effort. However, where the total evaluation is viewed as testing-room based, as most are in practice, there is little hope that any instructional good will come from the diagnostic effort. The examiner, in such a situation, has no defense against the making of erroneous or irrelevant evaluations and recommendations. Neither will a five- or ten-minute visit with the teacher and a cursory observation of the child in the classroom measurably improve the situation. The formal assessment is only part of the total educational evaluation, which must be considered as incomplete until diagnostic teaching procedures have probed in depth the findings of the formal assessment. It is only after the educational information that resulted from the formal evaluation has been checked out by the diagnostically oriented teacher in the classroom environment, and after feedback is provided to the other members of the diagnostic team, that the total evaluation of a child can be considered complete. Now begins the period of ongoing reevaluation in which the teacher will engage as long as the youngster receives services. Diagnostic teaching is one way to minimize the likelihood that the child will be misplaced or miseducated.
The Informal Evaluation
The informal evaluation is that part of the total diagnostic process characterized (1) by the use of informal procedures, (2) administered by the educational diagnostician (usually a teacher), (3) in a continuing educational setting, and (4) is frequently called “diagnostic teaching.” The goals of this part of the total evaluation are to expand, probe, verify, and, if need be, discard the conclusions and recommendations of formal assessment. Optimally, the informal and formal aspects of assessment would occur simultaneously, making possible the continued modification of educational hypotheses about children. Mostly, the informal approach is employed after the formal evaluation is completed. This lessens the likelihood of interaction between the team members. In actual practice, diagnostic teaching, as a systematic ingredient of the total evaluation, is not undertaken at all in most of the schools. And the teachers who engage in such procedures do so for their own classroom benefit, not to improve the evaluation process.
The teacher of learning-disabled children who works in the schools must recognize that in most cases the educational evaluation will be left to him alone. He can learn from the school authorities which types of diagnostic tests or devices he can use without violation of administration policies; but for the most part, he will depend upon informal techniques to obtain the information that is vital for successful teaching. While these techniques can be used to verify the findings of formal assessment; they can also be substituted, if need be, for standardized test results.
It would be impossible to describe all of the possible informal techniques that can be used to assess a child’s various abilities. Most of the recent books concerned with the learning disabilities of children provide the reader with a multitude of pertinent activities. Even these, however, will prove insufficient, for the diagnostic teacher will soon be confronted with a unique problem and a need to devise his own procedures.
The following informal procedures do not include every area that can be probed, but the description is detailed enough to provide some insight into the process of informal testing. These particular techniques are borrowed from the book, Methods for Learning Disorders. 4
Auditory Functions
The following procedures are primarily auditory in nature; although other functions may be tested simultaneously, they are of lesser importance in the child’s performance.
Auditory decoding Recognizes environmental noises. Understands parts of speech—nouns, verbs, adjectives, prepositions, etc. Follows one, two, three, or four instructions. Recognizes names of colors. Understands a story read to him.
Auditory association Matches noisemakers by sound, e.g., two horns or two whistles. Speech-sound discrimination—words, nonsense syllables.
Auditory closure Recognizes incomplete words. Auditory blending of sounds to form words. Simple analogies.
Visual Functions
As with the auditory functions, the procedures used to test visual competencies may overlap into other areas of language function, but the tasks are primarily visual in nature.
Visual decoding Recognizes objects and pictures. Recognizes a picture cut into two, three, or four pieces. Recognizes colors.
Visual association Matches colors, objects, pictures. Matches object with picture. Matches geometric forms—two- or three-dimensional forms or three-dimensional forms with pictures of forms.
Visual closure Recognizes incomplete pictures. Recognizes incomplete letters, numerals, or words.
Tactile-Kinesthetic Functions
Recognizes by touch alone objects placed in either hand.
Matches sandpaper forms by touch.
Recognizes simple geometric forms, letters, words, etc., when they are drawn on the back of his hand or on his back.
Vocal Functions
Uses words, phrases, or sentences.
Uses adequate grammar—proper inflectional endings, tenses, etc.
Uses adequate sentence structure—words are not omitted, transposed, substituted.
Tells a story in logical sequence.
Mean sentence length appropriate for age.
Motor Functions
Imitates examiner’s actions.
Pantomimes everyday actions—combing hair, brushing teeth, batting a ball.
Copies geometric designs.
Draws human figures.
Writes his name, letters, numerals, words, sentences.
Memory Functions
Repeats series of digits in, or not in, sequence.
Recalls a set of objects seen.
Recalls a set of pictures, letters, numerals, words.
Sequencing Functions
Recalls a pattern of taps, pitches.
Recalls in sequence a series of objects, pictures, forms.
Recalls a series of unrelated words, a sentence, a series of related words.
The use of the techniques described above will enable the teacher to assess a child’s ability to perform fundamental skills. Informal procedures are just as effective when applied to problems in basic school subjects such as reading, spelling, writing, and arithmetic. The range of information that can be provided by the use of informal reading inventories has already been mentioned; Marjorie Johnson and Roy Kress provide a comprehensive review of these useful instruments in a 1964 publication of The International Reading Association. 5 This work is strongly recommended for those teachers who are unfamiliar with the informal reading approaches.
S. H. Linn’s chapter, “Spelling Problems: Diagnosis and Remediation,” in Building Spelling Skills in Dyslexic Children, includes suggestions for delimiting spelling problems through careful scrutiny of the child’s spelling lesson and work habits. 6 Diagnostically significant questions are posed, such as the following: Can the pupil recall the letter and sound symbols quickly, accurately, and produce them on paper correctly? Can he fuse the sound parts of words together into whole words? Can he remember what you have written on the board a few minutes after it is erased? Can he identify sounds?
To assess problems in writing, the book by Doris Johnson and Helmer Myklebust, Learning Disabilities: Educational Principles and Practices, is recommended to teachers. 7 Techniques are presented that enable the teacher to distinguish among dysgraphia, deficits in revisualization, and deficiencies in formulation and syntax. As the educational approach will differ with each of these writing problems, proper teacher assessment is fundamental to successful instruction.
An optimal evaluation procedure for use in the public schools has been described in this paper. This process is characterized by its heavy educational orientation, reliance upon informal assessment techniques, and recognition of the teacher as a partner in the diagnostic decision-making phases of the evaluation.
Of course, basic changes will have to be made in present school policies and personnel thinking before this program can be implemented. The school psychologist, who now bears the primary responsibility for the assessment process, will have to share that responsibility with teachers. In the process, he will also have to acquire a considerably better understanding of teachers’ needs and expectations of the school evaluation. The teacher, on the other hand, will have to learn the fundamentals of diagnostic teaching and perfect his new skills through experience; he must learn to be specific in expressing what he wants from the other team members when a child is referred for assessment. The school administration will have to alter existing policies and permit selected teachers to engage actively and openly in evaluative procedures and to establish new programs involving diagnostic classes, transition rooms, itinerant programs, and resource rooms, which lend themselves better to diagnostic education than do self-contained classes. And finally, teacher-education programs must begin to train teachers who can “field” problems as they arise; who feel comfortable in the management of auditory problems, visual deficits, behavioral disturbances, academic deficiencies, motor difficulties, and language inadequacies; and who, most of all, no longer view themselves as the “crippled” teacher, the “emotionally disturbed” teacher, or, the most semantically damaging of all, the “mentally retarded” teacher.
Footnotes
Editors’ Note
The first decade of Intervention in School and Clinic, then titled Academic Therapy, was an interesting one. The field was beginning to explore new thoughts and ideas. The focus was on learning differences, assessment, educational planning, and atypical behaviors. As we perused the archives, we read articles by Dr. Barbara Bateman, Dr. Janet Lerner, Dr. Barbara Keogh, Dr. Roger Kroth, and Dr. Stephen Larsen. All became leaders in our field. We selected an article by Dr. Don Hammill, then a professor at Temple University. Dr. Hammill’s discussion of the direct role evaluation plays in planning appropriate instruction for a student is as pertinent today as it was 50 years ago. He speaks to the teacher, school psychologist, and administrator, reminding them that the ultimate goal is student success in school. While some of the evaluation instruments have changed, the challenge remains for all of us to become an evaluation team member and learn from one another.
This paper was originally published as: Hammill, D. D. (1971). Evaluating children for instructional purposes. Academic Therapy, 6(4), 341–353.