Making the most of reading assessments: principals play a key role in helping their schools develop the tools, support and structure needed to use unit tests to improve instructional practice.

California's state-adopted reading series provide a variety of assessments to measure student achievement in reading. Student progress can be assessed at the end of a lesson, theme or unit, quarter, trimester and year. Diagnostic tools also allow teachers to identify specific strengths and weaknesses of individual students.

The unit or theme assessments, administered at six-week intervals, can be especially helpful to teachers and administrators. These tests include, depending on the grade level, subtests in decoding, fluency, reading comprehension, listening comprehension, vocabulary, spelling, language conventions and writing strategies. These assessments help teachers and administrators monitor student growth in reading and provide timely information for instructional decision-making.

For the past two years, I've had the opportunity to work with many teachers and administrators as they have attempted to analyze unit test results (for our purposes, I will use the term "unit" to represent both unit and theme assessments) and to use the data to improve instruction. I have learned a lot from my travels.

I have found that the assessments are simple to administer and easy to score; however, using the results to improve instruction has been problematic. For the most part, teachers and administrators have not been adequately prepared to make effective use of the data.

For example, I have seen grade-level teams attempt to analyze test results and discuss their implications for instruction. Lacking the necessary tools for the task, they wound up feeling frustrated. They learned very little from the data, gained few insights into their students' reading, and saw little payoff for their efforts. At the conclusion of the discussion, the feeling was clear: "This test and this meeting were a waste of time!"

Over the past few years obstacles to this practice have become clear, ineffective procedures have been identified and effective strategies and structures have been documented. Based on my experience, I believe steps can be taken to prepare teachers and administrators to skillfully use unit assessment results to improve their instructional practice and, ultimately, to effectively respond to the needs of every student.

Principal plays key role

Based on my observations, it is clear to me that the principal plays the pivotal role in this effort. He or she is the only person who can guarantee that teachers receive the necessary training, resources and support they need; and is the only one who can create an environment that promotes continuous improvement school-wide. To ensure that unit assessment data results in improved teaching, the principal, I believe, needs to take the following actions:

* Institute grade-level meetings as the forum for analyzing unit assessment data and discussing their implications for improving instructional practice.

* Schedule time for grade-level teams to discuss the results of the unit assessments.

* Provide teachers with the tools they need to analyze and use unit assessment data to improve their instruction.

* Make the purpose of and "stakes" for the unit assessments perfectly clear.

* Create an internal accountability system to ensure that unit assessment data results in improved instruction.

1. Institute grade-level meetings as the forum for analyzing unit assessment data and discussing their implication for improving instructional practice. The grade-level team meeting provides the most productive forum for collegial discussion and instructional decision-making. The teachers administer the same test, use a common curriculum, confront similar challenges during the course of day-to-day instruction and share common experiences.

The questions and issues that surface during the review of unit assessment results are complex. Typical questions that arise are: What patterns emerge in the data? How do I explain the results? What could I have done differently during the preceding period of instruction? Which interventions were successful during the preceding period of instruction and which were not? Which skills and knowledge were difficult to teach and/or difficult to learn?

Grappling with these kinds of questions and adjusting instructional practice accordingly is difficult work. Team collaboration provides teachers with an opportunity to capitalize on one another's experience and expertise, and benefit from the "collective intelligence" of the group (Schmoker, 1996).

2. Schedule time for grade-level teams to discuss the results of the unit assessments. Grade-level team meetings require approximately 90 minutes: 60 minutes to analyze and discuss the data, and 30 minutes to evaluate previous instruction and make decisions about interventions and future instruction. Principals have sufficient experience now to accurately estimate when the tests will be administered during the year and how long it will take to score them. Consequently, meetings can be scheduled to coincide with the unit assessments.

It is unlikely that additional time will be provided for these meetings, so time must be created within existing schedules, contract constraints and other parameters.

Faculty meetings, to cite one option, might be used for this purpose. Each grade-level team could meet in a different section of the meeting room, discuss their assessment results and then share their findings and instructional decisions with the other teams.

Other alternatives for creating time for these meetings include adjusting the weekly schedule and "banking" time so students can be released early on designated days; scheduling standard grade-level meetings to coincide with the theme assessments; arranging for substitute teachers, specialists and other support staff to teach classes for a particular grade level; scheduling special events for a particular grade level and assigning substitute teachers and support staff to supervise students; and scheduling professional development days to coincide with the targeted meeting dates.

3. Provide teachers with the tools they need to analyze and use unit assessment data to improve their Instruction. Once teachers have time to discuss unit assessment results, they need tools to analyze the data, evaluate their teaching and improve their instructional practice. They must be able to organize data in a user-friendly format and analyze data in a series of simple steps. Teachers also need a common language to discuss data and its relationship to instruction, and they need to understand the "technology of teaching" (Fox, 2003).

The first data a teacher usually sees is an alphabetized list of his or her students and their scores on particular subtests. For example, one of the third grade unit assessments provides scores in fluency, reading comprehension, vocabulary, spelling and language conventions. Before a teacher can do anything with the data, the data need to be reorganized, manually or electronically, so important findings "jump off the page."

Teachers also need a strategy or series of questions for analyzing data. Questions can be open-ended and very broad or concrete and specific. An open-ended question could be, "What did you learn from examining these subtest scores?" A more specific prompt might be, "What was the range of scores on this subtest?"

There is no one right way to analyze data. The specific strategy that teachers and administrators elect to use will depend on many variables; however, the first and last questions in any method of data analysis should be the same. The first question is, "How did the students do on this task?" The last question is, "In light of the data, what are you going to do?" Strategies differ in the number of steps between the first question and the last question, and by the level of specificity of the questions.

A common language for discussing unit assessment data and making the connection between assessment results and instruction is critical. This language must help teachers and administrators distinguish among and recognize the interdependence of three kinds of data: outcome data, which refers to evidence of student learning; demographic data, which refers to variables that affect the manner in and rate at which students learn; and process data, which refers to the "inputs" in school systems (Fox, 2001).

The technology of teaching

Teachers also need a way to step back from their teaching and analyze their instruction in an unbiased manner. This requires that they understand the "technology of teaching," which refers to the technical knowledge and skills that distinguish teaching as a profession.

For example, when evaluating instruction, teachers may need to consider alignment issues--the degree to which the content and format of their instruction matched the content and format of the assessment. They may also need to review their feedback on student work to determine if it was helpful to students, which means it was corrective in nature, timely and criterion-referenced (Marzano, 2001).

4. Make the purpose and stakes for the unit assessments perfectly clear. The principal must make the purpose and consequences of these assessments perfectly clear, as they will affect the manner in which teachers approach the data.

If assessments are understood to be "low stakes," solely for the purpose of improving instructional practice, teachers will likely be more critical and reflective about their instructional practice than if "high stakes" are attached to the results.

In my experience, high stakes assessments, the purpose of which is to judge teachers and administrators (and let's not forget students!), tend to produce--and understandably so--feelings of defensiveness, a need to rationalize the data and a tendency to ascribe the results to factors other than instructional practice.

Low stakes assessments, in my opinion, minimize feelings of defensiveness and encourage teachers to focus on their own practice and variables over which they have control. Teachers need to know if rewards and sanctions are associated with these assessments.

"No-stakes" tests

Actually, I believe the assessments should not be viewed as low stakes or high stakes, but rather as "no-stakes" tests. These assessments should be for one purpose only: improving instructional practice. Knowing that the intent of the assessments is to help improve teaching, teachers will be more inclined to vigorously analyze test results, critically review their own instructional practice, eagerly adjust their instruction, and confront what Jim Collins describes as the "brutal facts of reality" in their work (Collins, 2001).

The principal can anticipate that teachers will ask several questions, including the following: What is the purpose of these tests? How will the results be used? Who will see the results? Are there consequences--positive or negative--for students or teachers? What are teachers expected to do in response to the data? What will the principal do with this information?

5. Create an Internal accountability system to ensure that theme assessment data results in improved instruction. To ensure unit assessment data results in improved instruction, teachers and administrators must hold themselves accountable for improving instruction in reading.

This internal accountability is founded on the teachers' and administrators' commitment to continuously improving their own practice and supporting one another's efforts to improve their skills. This system requires an action plan, a communication system, provisions for follow-up support and a monitoring process.

Developing an action plan

Based on the results of the unit assessment, each grade-level team must develop an action plan that identifies precise areas of student need and describes specific instructional practices to be implemented in the following unit. Action plans should be limited to two areas of instructional improvement: one for the grade level and the other for the individual teacher.

For our purpose here, let's consider a scenario in which a fifth-grade team develops a grade-level action plan.

Based on the most recent unit assessment, all of the teachers agree on a common weakness: students are unable to draw inferences from explicitly stated text. They also agree that they had not provided their students with effective strategies for this particular reading comprehension skill.

Definitions and strategies

During their grade-level meeting they decide that the best course of action is to emphasize metacognition as a means of improving their students' ability to draw inferences from explicitly stated text. They agree to teach their students the strategy of metacognition, model metacognition during class, provide opportunities for students to practice metacognition, apply metacognition to all subject areas and assign homework that requires metacognition.

The teachers also discuss the definition of metacognition and effective strategies for teaching metacognition, and to practice modeling metacognition with one another. They agree to meet once every two weeks to discuss their progress.

Communication

Internal accountability depends on the timely flow of information from the grade-level teams to the principal. The teachers must provide the principal with a summary of the major findings from the data, a list of the students in need of corrective instruction and intervention strategies, and plans for improving instruction in the next unit. The teachers also should describe the nature of support required from the principal to accomplish their goals.

Based on the information from the teams, the principal might, for example, decide to observe certain teachers more frequently than in the past and make suggestions to improve instruction. He or she may also assign a school literacy specialist to work with a particular teacher, arrange for additional team planning or schedule team members to visit one another's classrooms. This kind of communication enables the principal to continually monitor student progress, assess instruction and provide support for the teachers.

Follow-up support

An effective internal accountability system must provide the follow-up support that teachers need to implement their action plans, and the principal plays the pivotal role in providing this support. He/she must be able to anticipate teacher needs, access the right people, manipulate schedules and responsibilities and find the funds, when necessary, to help teachers implement their plans.

As instruction is divided into six-week segments, the principal must be prepared to act quickly.

If a grade-level team requests that a district specialist be brought in to work with them, the principal must make the necessary arrangements without delay. If a teacher asks to observe another teacher conducting a lesson, the principal again must act swiftly. If teachers do not receive the support they require in a timely manner, the impact of the unit assessments is seriously compromised.

Monitoring new Instructional practices

Teachers and principals must be able to monitor new instructional practices and their effect on student learning. It is especially important to continuously assess new teaching practices and, should problems surface, make adjustments promptly.

As Mike Schmoker notes: "A legion of researchers from education and industry have demonstrated that instructional improvement depends on just such simple, data-driven formats--teams identifying and addressing areas of difficulty and then developing, critiquing, testing and upgrading efforts in light of ongoing results" (2003).

When teachers and principals devise a monitoring system, they will need to consider, for example, the following issues: Who will observe teachers, provide feedback and help teachers evaluate their own instruction? How often will observations be scheduled? What kind of data will be collected? How often will teachers discuss the data? How will success be defined? Who will facilitate discussions of the data?

The unit/theme assessments provide essential information for teacher decision-making and administrative support. These assessments can help teachers and administrators continually track student progress over time and adjust their instruction to meet their students' needs.

Unfortunately, in the past, teachers and administrators have not been provided with the tools, support and structures they need to make good use of this assessment data. Now we know what needs to be done and we know that the principal will play the key role in helping teachers use unit/theme assessment data to continually improve their instructional practice and to make certain that all students learn to read with skill, confidence and enjoyment.

References

Collins, Jim (2001). Good to Great. New York: HarperCollins.

Fox, Dennis (2003). Using Unit and Thematic Assessment in Reading for Instructional Decision-Making: Do We Teach in Light of the Data or in Spite of the Data? (Workshop materials).

Fox, Dennis (2001). "Three Kinds of Data for Decisions About Reading." Using Data for Educational Decision-Making. Wisconsin Center for Education Research, University of Wisconsin-Madison, Vol. 6, No. 1.

Marzano, Robert J., Pickering, Debra J., and Pollock, Jane E. (2001). Classroom Instruction That Works: Research-Based Strategies for Increasing Student Achievement. Alexandria, VA: Association for Supervision and Curriculum Development.

Schmoker, Mike (1996). Results: The Key to Continuous School Improvement. Alexandria, VA: Association for Supervision and Curriculum Development.

Schmoker, Mike (February 2003). First Things First: Demystifying Data Analysis, Educational Leadership. Alexandria, VA: Association for Supervision and Curriculum Development.

Dennis Fox is a consultant with the Southern California Comprehensive Assistance Center, Los Angeles County Office of Education.