Planning & assessing standards achievement: schools must provide teachers with strategies they can use to efficiently turn school standards resources into standards--achieving instruction.

In 2002, Principal Candy McCarthy of Alisal High School took a big risk on behalf of her teachers and students. Because her school had been designated "under-performing," she received supplemental funding for school improvement purposes. McCarthy decided to use all of the additional funds to purchase collaborative planning time for teachers.

By the time the 2003-2004 school year was under way, small groups of teachers with the same subject assignment in core areas of English, math and science were given as much as three hours per week to plan their lessons.

Allocating all of her school's funding to collaborative planning time called for a leap of faith. McCarthy believed that her staff, given the time and the opportunity to work together, would develop new strategies and effective plans that would lead Alisal High School's students to meet state standards.

Unfortunately, she did not provide teachers with specific directions for using the additional planning time. McCarthy's predicament is not uncommon, and researchers in school reform have noticed that the lack of specific directions for use of planning time is an impediment to standards achievement.

Connecting standards with instructional planning

According to Black and William (1998), the teachers at Alisal High School were facing a "black box" situation often used in describing scientific problems. The "inputs" and "outputs" of the organizational system were known, but the mechanism that teachers would use to turn resources into standards achievement remained a mystery.

Nave, Miech and Mosteller (2000), described the problem when they wrote that "little empirical evidence supports or refutes the existence of a causal link between standards and enhanced student learning." They went on to observe, "If we had a sufficiently detailed theory of action that explained how standards could influence student achievement, then evaluations could focus closely on this hypothesized causal chain ... to see that happens when the new standards are put into practice."

Rodney Ogawa and his colleagues (Ogawa, 2003) revealed that school districts might suffer serious setbacks if they fail to propose a system teachers can use to efficiently turn standards resources into standards-achieving instruction.

Ogawa's study of a southern California school district (identified as the "Delgado" School District) illustrates how one school administration led teachers through extensive standards-based reforms. The reform efforts included writing and selecting critical standards, selecting standards-aligned curriculum materials, and preparing benchmark tests to measure standards achievement.

When the time arrived in Delgado for teachers to receive professional development, the district leadership failed to produce a set of procedures for planning standards-achieving lessons. The administration could do no better than ask a panel of teachers with high test scores to share their teaching strategies with their colleagues. The seeds of cynicism in the standards movement were sewn in Delgado that day because the administration could not answer the question, "What are teachers to do with standards resources?"

Collaboratively planning for standards achievement

Neither Candy McCarthy nor I were aware of the Delgado study when I accepted her invitation to help the Alisal High School physics teachers with collaborative lesson planning during the spring term of 2004. Starting in February, the teachers and I met each Thursday afternoon for 90 minutes in a semester-long effort to improve the performance of many English language learners who were struggling with physics.

At the time of the invitation, I was working with colleagues in New Jersey on a new protocol for planning and evaluating standards-achieving lessons (O'Shea and Kimmel, 2003). This protocol adapts features of four well regarded educational initiatives including:

1. A wonderful approach to teacher collaboration and professional development known as Japanese Lesson Study (Lewis and Tsuchida, 1998);

2. The performance standards concept and related planning tools described by Resnick and Nolan (1995);

3. Collaborative analysis of student work for the purpose of assessing standards achievement; and

4. Some time-honored ideas related to outcomes-based education.

I had described this new protocol, called the Standards Achievement Planning Cycle, in a science methods class attended by Bakari Holmes, one of the Alisal High School physics teachers. Subsequently, he described this protocol to his colleagues, which led to the invitation for me to join the physics collaborative.

Candy McCarthy was present at the first meeting where I described the elements of the Standards Achievement Planning Cycle for the other members of the physics collaborative. The essential elements of this planning and student work evaluation protocol can be seen in the box above.

Collaborative assessment calls for practice and negotiation

In retrospect, perhaps we took the wrong first step. We started our deliberations by looking at student work obtained from science experiments, but we had not planned the lessons as a team. Problems were quick to emerge because we lacked clearly described performance descriptions or instructional targets.

We learned rather quickly that vaguely written performance expectations would invariably lead to disagreements between educators regarding the quality of student work. Without explicit performance descriptions, we could not reach consensus that student work samples met, or did not meet, a given standard.

Fortunately, these early faltering steps turned our attention to writing performance descriptions that met the learning expectations of a given standard selected for a particular lesson.

Next we show two performance descriptions derived from a California physics standard (see page 28). The first description lacks clear criteria for making judgments about student work. When we wrote descriptions similar to the first one, we could not use them effectively to reach consensus on student work sample quality. By contrast, the second description includes expectations for student performances in sufficient detail for judging student work and reaching agreement as to its quality.

While we took some satisfaction in writing performance descriptions that could be used to evaluate student work with less disagreement between us, we soon realized that student completion of physics problems did not necessarily demonstrate the depth of understanding we were seeking. As the spring semester unfolded, we sought additional expressions of their comprehension of the phenomena they were investigating.

As we struggled with these concerns, we began to realize that we were seeking authentic confirmation that students were learning the physics principles we wanted them to acquire as the principles were being taught. We did this by embedding prompts and tasks in lessons that called for students to express their knowledge and skills of selected standards in various ways.

Student interpretation of content statements in their own words

One member of our team, Pat McNeil, proposed the idea of having students engage the content statements of the standards and frameworks directly. He was motivated by a concern for the large number of English language learners we were teaching.

He challenged us to write student assignments that required them to describe content statements of the standards and frameworks in their own words. The assignments would help us measure student understanding of the physics in the standards while simultaneously helping the students to learn English.

McNeil described a writing prompt used by the district to encourage writing across the curriculum. We would reduce paragraphs in the science framework that were rich in physics content statements to simple sentences and phrases. These sentences and phrases would then be given to the students and they would be asked to re-state them in English, using words of their choosing.

Our success with this assessment strategy soon led to another idea for evaluating student understanding of physics principles.

Eavesdropping on student groups to hear the language of the standards

The physics teachers at Alisal High School were deeply committed to cooperative learning during lab investigations. After students made observations, collected data and reached conclusions, each cooperative team of students would be asked to present its findings to its classroom peers. Pat saw these group deliberations during experimentation, data treatment and report preparation as opportunities to listen for the proper use of new terms and vocabulary found in the physics standards.

For instance, we were interested in knowing if students, in the course of their cooperative efforts, could properly use and interconnect such critical terms as "momentum," "product," "mass," "velocity," "kilogram" "kilogram-meters" and "per second" in ways that would evidence sense-making about critical concepts of momentum.

We became so enthralled by students' oral use of the language of the standards during lab investigations that we initiated an action research project (with student and parent approval) that included the videotaping and audio recording of student oral interactions during class sessions. We analyzed the student oral statements captured in audio and video recordings during our collaboration sessions before we looked at student work samples.

By the time two months of weekly sessions had passed, we were actively exploring every avenue of capturing student understanding of the physics standards as they were being achieved.

Multiple representations of student understanding

As spring approached summer, we remained concerned about the difference between the sophistication of physics concepts in the California standards and the relatively superficial treatment of the same concepts in classroom textbooks. However, the physics teachers at Alisal High School were confident that their lab investigations, experiments and demonstrations matched the expectations of the state standards.

We wanted to be sure students were reaching the level of understanding that would be measured by state standards exams. Bakari proposed new expectations for student team reports of experimental findings with this concern in mind.

Holmes, McNeil and the other teachers reached the conclusion that students should be asked to provide multiple representations of their understanding of physics phenomena and concepts during oral presentations of experimental results.

For investigations in momentum, this meant students would be expected to:

1. Draw a picture of two or three objects interacting in a collision where momentum was transferred from one moving body to another;

2. Draw a diagram, derived from the picture, that replaced the pictures of actual objects with arrows that expressed the direction of the moving objects and their speed (vectors);

3. Describe the events taking place in standard written English;

4. Identify objects, their masses and velocities, and represent them by appropriate letter labels (variables); and finally

5. Replace appropriate variables with measured numerical values obtained in their experiments and calculate the transfer of momentum taking place in the events they observed.

By the end of the semester, physics classes at Alisal High School filled with student production of the language of the standards at every appropriate opportunity.

Retrospect

We probably did not cover as much physics as usual during that spring term at Alisal High School, but I do believe the students learned the physics that was covered at a depth described in the standards. It is important to keep in mind that our deliberations that spring were not an experiment to prove the effectiveness of the planning protocol. In truth, the protocol served no greater purpose than to launch us on an extended pursuit of pedagogical questions, all of which began with, "What if....?"

Spring of 2004 provided us with a rare experience: a delightful, intellectual engagement with the California content standards by teachers working collaboratively, and by teachers as they interacted with their students.

At the end of the semester, Holmes showed me the final grades that his students earned at the end of the fall 2004 term, and then he showed me the final grades earned by students in spring 2004 after we had planned collaboratively. The result was a dramatic reduction in the number of "D" and "F" grades. It was then that I figured out the signal in all the background noise we heard in the audio recordings. It was the sound of achievement gaps closing.

The Standards Achievement Planning Cycle

1 The teachers of the collaborative would agree upon the specific elements of only one or two standards to be achieved in an upcoming lesson. We would limit the number of standards to be achieved during any one lesson so that knowledge could be achieved in depth.

2 We would analyze the California Science Frameworks, the selected standards and released state test items to identify content statements in these documents that describe what students were to know or be able to do to achieve the selected standard.

3 Guided by the work of Resnick and Nolan (1995), the all-important next step was to translate the identified content statements into vivid descriptions of student products or performances that would constitute evidence of standards achievement. These descriptions of student performances would become the instructional targets for our lessons.

4 The fourth step of the protocol called for the selection and/or development of student experiences, typically physics experiments, that would lead students to produce the work described in our instructional targets.

5 Finally, we would use the performance descriptions of our lesson plans --our instructional targets--as others might use a rubric. They would be our tools for evaluating student work that each of us would bring to the planning session after each teacher in the collaborative had a chance to teach the lesson.

A California physics standard and two related performance descriptions

* A California Physics Content Standard:

Standard 2.d: Students know how to calculate momentum as the product mv.

The momentum p of an object is defined as the product of its mass, m and its velocity, v. The units of momentum are kg-m/s (kilogram-meter/second).

1. A vague performance description of little use for evaluating student work:

Students obtain the correct product for momentum given values for mass, m, and for velocity, v.

2. A performance description that can be used to evaluate student work:

Given three mass values for a ball (1 kilogram), a bike (10 kilograms), and a car (1,000 kilograms), students will calculate the momentum of all three objects traveling at 5 meters/second and express the answers with proper units for momentum: 5 kilogram-meters/second, 50 kilogram-meters/second, and 5,000 kilogram-meters/second, respectively.

References

Black, P., and William, D. (October 1998). "Inside the Black Box." Phi Delta Kappan. 80 (2), 139-148.

Lewis, C., & Tsuchida, I. (Winter 1998). "A lesson is like a swiftly flowing river: Research lessons and the improvement of Japanese education." American Educator. 14-17; 50-52.

Nave, M., Miech, E., & Mosteller, F. (October 2000). "A lapse in standards: linking standards-based reform with standards achievement." Phi Delta Kappan 82 (2), 128-132.

Ogawa, R., Sandholtz, J.H., Martinez-Flores, M., & Scribner, S.P. (2003). "The substantive and symbolic consequences of a district's standards-based curriculum." American Educational Research Journal 40(1), 147-176.

O'Shea, M. and Kimmel, H., (2003). "Preparing Teachers for Content Standards: A Field Study of Implementation Problems." Presented at the American Association for Colleges of Teacher Education, New Orleans, LA: Jan. 25.

O'Shea, M., (Jan./Feb. 2002) "Teaching to Standards." Leadership. Association of California School Administrators: Sacramento, CA.

Resnick, L.B. & Nolan, K.L. (1995). Standards for Education in Debating the future of American education: Do we need national standards and assessments? D. Ravitch (Ed.) Brookings Institution: Washington, D.C.

Mark R. O'Shea is a professor of science education at California State University-Monterey. He is also the executive director of the Institute for Achievement of Academic Standards. For more information about the institute, check out www.academicstandards.org.