Standards for assessing teaching effectiveness are key: a response to Schalock, Schalock and Myton.
In their review of the articles on quality teaching in the November 1996 Kappan, Del Schalock, Mark Schalock, and David Myton suggest that the education profession is stuck on an "input" approach to assessing teaching that does not concern itself with effects on student learning. They offer the Oregon Work Sampling Methodology (WSM) as an example of what the profession should strive to adopt in lieu of portfolios and performance assessments, such as those of the National Board for Professional Teaching Standards (NBPTS) and INTASC (Interstate New Teacher Assessment and Support Consortium). These approaches, they believe, fall short of adequately considering student learning. They argue that "student learning must be the touchstone by which teachers and teacher educators are gauged." On this we emphatically agree. The issue is how to accomplish this goal.
Although the authors quote some passages that clearly express the interest of the National Commission on Teaching and America's Future in linking teaching to student learning (for example, my statement that teachers must be equipped with "the knowledge and skills they need to enable all children to learn"), they do not see how assessments such as those recommended by the commission accomplish this linkage. It appears that they have not examined closely the assessments in use by INTASC and the NBPTS, beyond reading either the brief descriptions offered in the Kappan articles or the commission report. In fact, both of these assessments require that teachers include samples of student work over time, linked to evidence of teaching plans and activities, teacher feedback to students, and analyses of student needs and progress. This close linkage between artifacts that demonstrate student learning and analyses that show how that learning grows out of specific teaching actions and decisions is critical to judging teaching effects. The connections between teaching and learning and their relationship to standards are less well developed in the Oregon WSM approach.
To be sure, the Oregon WSM has much to recommend it. In contrast to the assessments of teachers in use in most states, it actually looks at teaching, and it does so in the context of teachers' goals, classroom contexts, and student learning - measured in ways that attempt to link learning to the educational goals being sought. In these respects, it offers real promise for evaluating teaching in ways that might also be useful in helping teachers improve.
There is, however, much more work to be done to make this approach a useful tool for formative or summative evaluation of teachers. In terms of measuring student learning, the WSM is dependent on the quality of assessments teachers can devise on their own; these can be highly variable and may fail to evaluate important kinds of learning well. As evaluation expert Daniel Stufflebeam noted in a review of the WSM, "It matters little if ratings of teacher performance are associated with pupil gains in work sample performance if the teachers employ biased outcome measures or assess low-level knowledge and skills.'"
Furthermore, the WSM still envisions very limited kinds of performances that can be tested with a batch of items to produce a "percent correct" score, rather than more complex performances that support higher-order learning, such as designs for experiments, mathematical models, research papers, extended essays, oral presentations, or debates. In theory and in practice, these kinds of measures can be scored and improvements measured over time, but the methods of evaluation must extend beyond "percent correct" indices if they are to be useful.
In addition, the self-referential quality of the WSM system means that the teaching observed will be only that associated with the objectives the teacher has selected, rather than a sample of performance assessed against a common standard of practice. As Peter Airasian noted in another review of the WSM, "While it is important to align objectives, instruction, and assessment to one another, it is also important that the objectives themselves be meaningful and worthwhile."(2) There are no standards in the WSM against which to evaluate instructional goals, activities, or teaching practices for their usefulness and appropriateness in pursuing meaningful ends. If a teacher devotes three weeks of instructional time to memorizing the Dewey decimal system or the parts of a grasshopper and shows an increase in student scores on tests measuring their recall of such information, this would appear to count as much toward a judgment of teaching "effectiveness" as if the same time had been spent learning more generalizable skills, such as writing a persuasive essay or critiquing the methodology of a science experiment. The WSM has yet to meet the challenge of ensuring that observers apply common criteria for assessing standards of professionally appropriate practice.
Finally, there is currently no means within the WSM system to link teaching practices to student outcomes. If student outcomes appear to improve by whatever measure is used, the effect is attributed to the teacher's practice, without an analysis that shows how the two are related. Thus, if an English teacher's students improve their writing because of the efforts of their history teacher to help them write a term paper, the English teacher would be deemed "effective," even if she hadn't taught these skills. Because longitudinal evidence on specific children is not collected, if one or more students are transferred into a teacher's class from a special education setting and the average scores of her class decline as a consequence, she could appear to be "ineffective," even if other students showed gains. This disjunction between the evidence on teaching and the evidence on learning limits what can be learned about good practice by teacher educators and teachers alike.
Despite these shortcomings, there is value in an approach to teacher assessment that points practitioners toward the careful evaluation of practices, contexts, and outcomes, including the systematic consideration of student and teacher work. I have no doubt that such an approach encourages teachers to develop diagnostic habits of thinking as well as specific practices. In this respect the WSM resembles the assessments developed by the NBPTS (for accomplished teaching by veteran teachers) and by INTASC (for initial licensing of beginning teachers). These assessments also make use of work sampling methods to examine teachers' plans, instructional practices, assessment of student work, and feedback in the form of student work samples for several students whose progress is followed over time. However, these assessments differ from the WSM in some key respects that make them more appropriate for high-stakes decisions.
First, the assessment tasks are substantially standardized: English teachers must conduct a literary discussion and submit a videotape of the lesson and their analysis of it, and they must show how they teach writing by presenting several assignments with student work samples and feedback over time. Middle-grades teachers must outline and conduct a several-week interdisciplinary unit that ties several fields of study together. Mathematics teachers must show how they teach a small-group lesson using technology or manipulatives and a large-group lesson focusing on mathematical discourse.
Second, as these descriptions suggest, the tasks and their evaluations are pegged to clear standards of practice that were developed by expert teachers, linked to emerging student standards, and validated in the field.
Third, the examinations include both on-demand performance tasks (for example, a critique of a textbook or piece of educational software against criteria indicated in a task) and samples derived more directly from the teachers' own work. It is important to have some tasks determined in advance so that evaluators can determine whether candidates have certain skills commonly accepted as essential for success in the field.
Fourth, the scoring systems are highly developed and have been validated and tested for reliability. They evaluate important dimensions of teaching and are administered by trained assessors who must achieve reliable performance in scoring.
Fifth, a careful process of standard setting has been undertaken and validated.
Finally, the examinations and scoring systems of the National Board have been thoroughly vetted for evidence of bias, a critical concern for high-stakes testing, especially when the identity of candidates is known to evaluators.
Although the WSM has not yet met all these standards for high-stakes assessments, it certainly holds promise as a valuable tool for preparing and assessing both beginning and veteran teachers. Perhaps the most important feature of truly professional assessment is that it enables teachers to inquire productively into the effects of their teaching on student learning, thus developing the commitment and the disposition to evaluate themselves in light of how well they can help students succeed. Teachers who have undergone the INTASC assessments or the portfolio and performance assessment processes of the NBPTS, as well as teachers who have served as assessors, consistently cite the experience as the most powerful professional development of their careers. They believe that they are better teachers as a result of this work. I suspect that Oregon teachers also find the WSM helpful in inquiring into their teaching. In the long run, this is a touchstone of a real profession and will support student learning. On this, I'm sure we agree.
1. Daniel L. Stufflebeam, "Oregon Teacher Work Sample Methodology: Educational Policy Review," in Jason Millman, ed., Grading Teachers, Grading Schools: Is Student Achievement a Valid Evaluation Measure? (Thousand Oaks, Calif.: Corwin Press, 1997), p. 57.
2. Peter W. Airasian, "Oregon Teacher Work Sample Methodology: Potentials and Problems," in Millman, p. 47.
LINDA DARLING-HAMMOND is William F. Russell Professor in the Foundations of Education at Teachers College, Columbia University, New York, N.Y., and executive director of the National Commission on Teaching and America's Future.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||in this issue, p. 468|
|Publication:||Phi Delta Kappan|
|Date:||Feb 1, 1998|
|Previous Article:||Effectiveness - along with quality - should be the focus.|
|Next Article:||Calculations about calculators.|