K-12 teachers' assessment of student learning.

Abstract

This article analyses the factors affecting how K-12 teachers assess student learning. To probe how they assess student learning, I present them with a transcript of a classroom interaction and ask them to analyze it to determine and explain whether learning has taken place. I find that student learning is assessed largely on the basis of how teachers teach rather than by whether students learn.

Introduction

Developing K-12 teachers' abilities to assess student learning has always been assumed to be part of teacher preparation programs. In fact, recent legislations such as the No Child Left Behind (NCLB) Act have increased attention to assessment. Professional organizations such as the National Board for Professional Teaching Standards (NBPTS, 2005, p. 16) and the Interstate New Teacher Assessment and Support Consortium (INTASC, 1992, p. 31) have called for a renewed emphasis on assessment. NBPTS's five core propositions focus on assessment by stressing that "Teachers are responsible for managing and monitoring student learning". One of INTASC's ten principles requires teachers to understand and use "formal and informal assessment strategies to evaluate and ensure the continuous intellectual, social, and physical development of the learner". This attention to assessment has generated at least two strands of research. One strand concerns standardized testing and examines its consequences for schools, curriculum, teacher behaviors and student learning (Sacks, 1999; McCracken & McCracken, 2001). The second investigates professional development that engages K-I 2 teachers in analyses of student work (Little, Gearhart, Curry & Karma, 2003; Kazemi & Franke, 2004). This research demonstrates that teachers, even those with experience, may have difficulties in interpreting student work and targeted professional development can expand their understanding of learner strategies and strengthen their assessment capabilities. To the best of our knowledge, not much attention has focused on how K-12 teachers assess student learning, in particular, what factors influence their assessment.

This article is concerned with the factors impacting teachers' assessment of learning. To probe how teachers assess student learning, I present pre-service and in-service teachers with a transcript of a classroom interaction and ask them to analyze it to determine and explain whether learning has taken place. Their group discussions and reports are analyzed. My findings reveal that student learning is assessed largely on the basis of how teachers teach rather than by whether students learn.

Design

To examine teachers' assessment of student learning, I presented both groups with an analytic task. The analytic task revolves round a dialogue regarding tariff in Borich (2003, pp. 152-153), part of which is shown here.

* Carlos: I don't understand why anybody would want to tax themselves.

* Mr. Marks: Give us the answer to my question.

* Carlos: Well, what I meant is, it would seem people would complain if they had to pay more for something they wanted just because someone else wanted to place a tax on it.

* Mr. Marks: And actually they did complain -just before the Civil War. At that time, in 1862, the government placed taxes on domestic manufacturers and duties on imports for two reasons. Can you think of one of these reasons? * Carlos: Well, to raise money to run the government, I suppose, and maybe like we said last week--to get ready for a war.

* Mr. Marks: Okay, so a tariff on manufactured goods, whether created in this country or another, raises lots of money that could be used to run a government. What about another reason? Angel [who is sitting next to Carlos in the front row], what do you think?

* Angel: It seems to me that taxes to run a government and taxes on goods coming into the country are different. If products that are made in this country are taxed, what's the point of taxing imported products, which might make TV sets and cars coming from Japan too expensive?

* Mr. Marks. And then what? [Nodding to Angel to continue.[

* Angel: Everyone will stop buying them because they won't be able to afford them.

* Mr. Marks: And then what? [Nodding to Angel again, but now Angel looks confused.] Okay, Carlos.

* Carlos: Now I get it! Sure, we'll stop buying, or at least buy fewer of these imported products, forcing us to buy ones made in the United States.

This excerpt is a slice of a social studies class in which the teacher, Mr. Marks, interacted with 33 students. Much of the instructional content was constructed by students, elicited through teacher questioning. Typical of teacher-led class discussions, a few students participated directly. ! selected this dialogue for two reasons. First, this dialogue includes not only student responses to teacher questions, but also information on Mr. Marks' teaching, student reaction and interaction. It provides insights into the types of information subjects deem relevant in assessing learning. Second, not all students participated, a point made explicit by the questions subjects were asked to address: a) whether there was evidence of student learning; b) for whom there was evidence of learning, for whom there was not and for whom no evidence existed; and c) how they came to their conclusions. A critical analysis requires distinguishing three groups of students: a) those who demonstrate an understanding; b) those who do not; and c) those for whom no evidence is provided.

Data were collected from 53 participants (11 male and 42 female) enrolled in four sections of my online graduate education research course. They included 37 in-service and 16 pre-service teachers, with their concentrations ranging from English (12), social studies (18), math (4) and science (6) to special education (10), and foreign language (3). Of the 37 in-service teachers, all had obtained initial teaching certification from New York State. All pre-service teachers except for 5 had their initial certifications. Of the 53 students, 16 did not have teaching jobs apart from student teaching experiences. The 37 in-service teachers averaged 28 months of teaching, including substitute teaching. Two sets of data were obtained for analysis. First, the participants conducted online discussions in groups of 3 to 4 members with a mixture of in-service and pre-service teachers. Second, each group submitted a group report, addressing the questions raised. The data reported here are based on the group discussion and the group reports.

Results

I classified the responses into two groups. One group of participants (including in-service and pre-service teachers) understand that learning concerns what students do. For them, understanding whether and what students have learned should be based on assessment of student work. They know student work involves not just test results or written work but can include such things as verbal responses to teacher questions. They understand that students may demonstrate their learning at different levels, distinguishing understanding from application. Regarding this analytic task, these participants understand that they need to focus on student responses to Mr. Marks' questions and on his reaction to student responses. Two examples of such responses are presented below.

   An example of learning that has taken place was when Carlos states
   "Now I get it! Sure, we'll stop buying, or at least buy fewer of
   these imported products, forcing us to buy ones made in the United
   States". Another example is when Mr. Marks asks if anyone can
   summarize and Angel replies: "What he is saying is that as the price
   of imported products goes up we are forced to buy similar products
   made in our own country. And because things like cars and TVs are
   taxed by our own government, money would be raised for running the
   government. Is that right?" Mr. Marks replies that that was a
   correct response. (a pre-service teacher)

   We know that learning has taken place with Carlos because he can be
   quoted as saying, "Now I get it!" His response is validated by Mr.
   Marks wanting his comments summarized by someone. 1 don't think he
   would want misinformation restated. Angel and Amanda both
   successfully summarize information. Amanda even builds on the
   learned information. Her question about the implications of taxing
   imported goods leading to unfair prices is evidence of application.
   (an in-service teacher)

These participants zeroed in on what students said and Mr. Marks's reactions as evidence for student learning. They understand that learning may be demonstrated at different levels, as evidenced by the discussion of Amanda's ability to build on learned information and from this quote.

   There are various levels of learning going on in Mr. Mark's
   classroom. 3 out of 33 students are learning at the upper levels of
   Bloom's Taxonomy. Carlos, Angel, and Amanda are asked to analyze
   and synthesize information pertaining to taxes and tariffs. Tim is
   learning at the knowledge level. He has learned how to locate and
   recite the definition of tariff. (an in-service teacher)

For this group of participants, learning assessment is anchored on what students say and do and must take into account different levels of learning from awareness to understanding to application. While this view of learning accounts for a significant percentage (38%) of responses, most participants view student learning not in relation to students. They view learning in relation to teacher actions. Consider these quotes:

   In this lesson, not every student has had the chance to learn.
   Throughout the lesson Mr. Marks engaged very few students, three to
   be exact. The other students were somewhat spectators. This was also
   a "lecture" type lesson where he was giving out information. The
   students didn't have to develop their own answers, but simply read
   off a worksheet when he did involve them. There was very little
   "critical thinking, and problem solving". (an in-service teacher)

   Learning did not take place in his class because not every student
   had a chance to learn. Real learning includes the use of creative
   problems as well as real world reflections. There may have been
   examples of real world related contexts, but not much creative
   problems. (an in-service teacher)

   Learning involves students in activities that are the same or as
   close as possible to the "real life" task. If this is the case,
   learning did not take place. This lesson was based solely on lecture
   and reading notes, not much of an activity. (a pre-service teacher)

These participants concluded that learning did not take place for most of the students. Their reasoning is based on: i) Mr. Marks lectured; it) Mr. Marks did not engage students in creative problems; iii) Mr. Marks engaged students in "very little critical thinking and problem solving activities'"; and iv) Mr. Marks interacted with a few students. Notwithstanding the fact that Mr. Marks' teaching was not properly classified as a lecture (considering how little instructional content he presented) and that teacher questioning plays a critical role in engaging students in critical thinking, these participants focus their assessment on how a teacher teaches, what kinds of instructional activities he engages his students in, and how many students he interacts with. In addition to these factors, examination of the data reveals a range of teacher-related factors impacting my participants' assessment from whether teachers engage students in collaborative learning to the seating arrangement. For many, lecturing and lack of group work are sure signs of bad teaching and, for some, sure signs that student learning has not taken place.

While nobody would dispute that effective teaching plays a crucial role in student learning, assessment of learning cannot depend merely on teacher actions. This emphasis on teacher action is reflected most strikingly in the negative reactions to Mr. Marks' statement that "This [tariff lesson] isn't going to be as interesting as our discussion of the battles of the Civil War last week, but pay attention anyway". Most participants cited this statement as part of the evidence that learning did not take place. This quote highlights the weight that this statement carried in their assessment:

   Also he states from the beginning that they would not find the
   information interesting. This method of introducing a concept just
   lost the attention of half his class. (an in-service teacher)

This dialogue presents no evidence that Mr. Marks lost "half his class" nor does it present evidence that students were disengaged. While Mr. Marks' statement is inappropriate, the impact of individual teacher statements should not be exaggerated and should be weighed in the context of the entire dialogue. Nevertheless, for many participants, this and some of Mr. Marks' other statements carried significant weight, leading 51% of the study participants to the conclusion that learning did not take place for most students. For some study participants (11%), this emphasis on teacher actions results in the conclusion that learning has not taken place for any student, as this quote shows: "I don't see any evidence of learning going on during this lesson". There is evidence that two students did not demonstrate learning. The dialogue presents no evidence either way that the majority has or has not learned.

As the participants of this study include in-service and pre-service teachers, one important question raised by the reviewers is whether in-service teachers' experience of working in a school as opposed to pre-service teachers' lack of it makes a difference in how they assess student learning. The answer is negative. Consider these statistics. Of the 33 participants who assess student learning in relation to teacher action, 22 (67%) of them are in-service teachers and 11 (33%) are pre-service teachers. This distribution roughly mimics the ratio of in-service to pre-service teachers in this study: 37 (70%) in-service vs. 16 (30%) pre-service. In other words, the added teaching experience has not led to a significant improvement in how in-service teachers assess student learning. Two reasons may be responsible for this result. First, the in-service teachers in this study are still early in their teaching careers and have only limited experience of working directly with students and assessing student learning. More importantly, as Kazemi and Franke (2004) show, the mere experience of teaching in a school does not always translate into improved assessment capabilities. Targeted professional development focusing on developing participants' ability to assess student work is critical.

To summarize, this finding--assessing student learning on the basis of teacher actions--cuts across both pre-service and in-service teachers. It is consistent with the research on expert versus novice teachers, which shows that novice teachers tend to concentrate on what they do rather than on whether students learn (See Hogan and Rabinowitz (2003) for a recent synthesis). For this reason, my participants' focus on teacher action may be part of a broader pattern characteristic of novice teachers.

Implications

This research is limited to a specific population in a specific locale. It is not clear to what extent these results can generalize to K-12 teachers in general. Nevertheless, I believe this study offers some insights into K-12 teachers' assessment of student learning.

First, beginning K-12 teachers tend to view assessment of student learning according to teacher rather than student actions. Moreover, increased teaching experience does not result automatically in increased assessment capabilities. These findings suggest that K12 teacher training and professional development should be directed partially to assessment of student learning. K-12 teachers must learn to distinguish assessment of teaching effectiveness from assessment of student learning. Even though effective instruction can result in student learning, learning assessment must rely on analyses of student work. It cannot be determined just by teacher actions. K-12 teachers must learn to focus on what students do and say and to analyse student work for insights into student learning and learner strategies. Second, many of my participants express difficulty with analysing student learning, with some confessing that they do not know.

   I am a substitute teacher during the school year, I have held some
   longer term and part time positions but I am still looking for a
   job. I am certainly still inexperienced and I do not have a clear
   vision on how to recognize that students have in fact learned.

This difficulty stems from the fact that learning assessment is a complex task, requiring the consideration of factors such as curricular goals, student age and ability and learning targets. Assessing learning through classroom interactions is more difficult as events happen quickly in a classroom and teachers have to attend to a range of issues. The difficulty also stems from the fact that most participants had limited experience analysing classroom interactions. Yet much of teachers' informal assessment of learning involves understanding these interactions. If my participants had difficulty analysing classroom interactions presented in a written form, one can imagine the difficulty they might face in a real classroom setting. These findings underscore the importance of providing targeted professional development for K-12 teachers. Little. et. al (2003) and Kazemi and Franke (2004) have shown that K-12 teachers can develop their assessment capabilities under carefully designed professional development programs.

References

Borich, G. D. (2003). Observation skills for effective teaching (4th edition). Upper Sandle River, NJ: Merrill Prentice-Hall, Inc.

Hogan, T. & Rabinowitz, M. (2003). Representation in teaching: inferences from research of expert and novice teachers. Educational Psychologist 38, 235-247.

Interstate New Teacher Assessment and Support Consortium (INTASC). (1992). Model standards for beginning teacher licensing, assessment and development: A resource for state dialog. [Online], Available: http://www.ccsso.org/projects/intcrstate_new_teacher_assessment and_support_consortium/Projects/Standards_Development/#core.

Kazemi, E., & Franke, E. L. (2004). Teacher learning in mathematics: using student work to promote collective inquiry. Journal of Mathematics Teacher Education, 7, 203-235.

Little, J. W., Gearhart, M., Curry, M., & Kafka, J. (2003). Looking at student work for teacher learning, teacher community, and school reform. Phi Delta Kappan, 85(3), 185-194.

McCracken, N. M., & McCracken, H. T. (2001). Teaching in the time of testing: what have you lost? English Journal, 91, 30-35.

National Board for Professional Teaching Standards (NBPTS). (2005). What teachers should know and be able to do. [Online], Available: http://www.nbpts.org/about/coreprops.cfm.

Sacks, P. (1999). Standardized Minds: The High Price of America's Testing Culture and What We Can Do About it. Cambridge, MA: Perseus.

Long Peng, State University of New York at Oswego, NY

Peng, Ph.D, is Associate Professor of Education Research, TESOL and Linguistics