Survive or thrive? Turning state mandates into local results: Monrovia's accountability system ensures that tests do not become ends in themselves. Instead, principals & teachers work together on targeted strategies to improve teaching and learning.

Clearly, we live in an age of educational accountability. The debate rages on over how to best measure the success of our schools locally, statewide and nationally. This debate often masks the underlying issues related to whether we are measuring student learning, school success or social and cultural attributes of our clients; whether the instruments we use to measure our progress become ends in themselves to which we bend our curriculum and instruction to raise the likelihood that we "look good" on these measures; whether we are engaged in "accounting" (collecting and reporting numbers) or "accountability" (collecting and reporting information about our students' learning, our processes and programs for the purpose of changing and improving our systems); and lastly, rarely include a discussion of how to implement testing and accountability systems in a way that results in fundamental improvements in our schools without creating such a heavy burden that in order to survive, school staff simply find ways to comply.

In the Monrovia Unified School District, we are engaged in a journey to try and figure out how to balance state mandates about accountability and testing with a local system of accountability and learning in a way that will result in an honest inquiry into practice. The purpose of this article is to share what we are doing and learning.

As the assistant superintendent of curriculum and instruction, I was charged by my superintendent to create and implement such a system. Beginning in the summer of 2000, she and I worked to devise a series of processes and activities that our principals would be responsible for facilitating at their sites with their teachers. These activities and the data and information collected make up the core of our administrators' yearly performance assessments.

The major components of the Monrovia Unified Student Learning and Accountability System include: 1) specifying data to be used from those measures mandated and developed locally; 2) setting a process for teachers to analyze these data; 3) setting student learning performance targets (based on the analysis of data); 4) setting student engagement practices and targets; and 5) reporting and 6) engaging in a reporting dialogue with the board of education.

Specifying data to be used

As indicated above, it is difficult to implement these systems locally in part because there is so much student performance information, and the testing systems themselves are under constant revision. Nonetheless, we must use what is mandated and in California that means the STAR system.

Keeping in mind that we are developing a system that will change as the state moves from a dependence on SAT-9 to more reliance on standards tests, we chose to focus our beginning efforts on the Total Reading and Total Mathematics sections of the SAT-9 as the first data to be analyzed. In addition, we are implementing locally-developed writing (similar to the state's fourth- and seventh-grade assessments) and mathematics criterion-referenced assessments aligned with state curriculum standards.

As a district, we have also implemented the Open Court Reading (OCR) program in grades K-3, and as part of the Governor's Reading Initiative, included six-week criterion-referenced assessments. District writing and mathematics assessments are administered twice yearly. OCR assessments are given approximately every six weeks.

To summarize, our system requires principals to use SAT-9 reading and mathematics, MUSD writing and mathematics and OCR assessments. These data are collected for all students in the district with the exception of SAT-9 data for kindergarten and 12th-grade students. The Academic Performance Index and related improvement targets for each subgroup are included.

A district template has been developed that summarizes these data as averages for each grade level represented in a given school. The template has spaces to record improvement targets for each data element and to document selected "student engagement strategies." Each of these elements are derived from site staff interactions with the collected student performance data. During the summer administrative retreat, these templates -- along with baseline data -- are shared with each site administrator. Additionally, time is spent reviewing the MUSD accountability system and what is required of them.

A process for teachers to analyze the data

Principals are asked in the fall to present the collected data to staff and to have teachers engage in an analysis process. Specific to the SAT-9 data, teachers are asked to review the content cluster report for their grade level in reading and mathematics. The content cluster report breaks down each of these areas into component skill objectives and indicates the number of items present on the test for each area. The report shows which areas have the most items correct and incorrect.

Using additional materials provided by Harcourt Brace, including the "Compendium of Academic Objectives," teachers in grade-level or department groups are asked to select an area where students are having the most difficulty and to come to agreement about a level of improvement for that area (an improvement target). They are also asked to develop student engagement strategies to be used by all teachers in that grade level or content area (for middle and high schools) designed to improve student learning in the identified area. This pattern of analysis, target-setting and agreements about teaching strategies are followed for each of the assessments.

Setting student learning performance targets

Using SAT-9 reading as an example, teachers may discover that within the reading content cluster area, "interpretation" has the most items and the fewest number correct. The current year's assessment may have yielded data such as the average number correct in the area of interpretation on the SAT-9 reading test is 15 correct out of a total of 24. Teachers would then agree on an improvement target for this area. That is, they would specify that on the following year's SAT-9 assessment, they believe that for their students, the average performance in this area would be 18 out of 24, or a 20 percent improvement. Again, this process of analysis and target-setting is followed for each of the assessments (SAT-9, Open Court, Writing and Mathematics CRT).

Student engagement strategies

Central to our assumptions in implementing these activities is that unless what is done affects what teachers do on a regular basis in their classrooms, this system will not lead to systematic improvements in our schools. Therefore, we ask teachers to select a set of teaching strategies that will be implemented on a regular basis by all teachers in a grade level or content area designed to specifically help all students improve in the performance areas identified through the above-described analysis of student performance data.

Continuing with our example, having identified "interpretation" as an area for reading improvement, all fourth-grade teachers may agree to a weekly reading and writing assignment that asks students to practice the skill of interpretation. Furthermore, teachers are then asked on a monthly basis, during staff meetings, to engage in a discussion of how this engagement strategy is being implemented and how it is affecting student learning. In other words, teachers are given a structured opportunity to discuss their craft and to share what they are learning about improving student learning in the specific areas identified as needing improvement. Principals are asked to summarize twice a year, in writing, what progress is being made both as evidenced through the ongoing assessments and the teacher dialogue about engagement strategies.

Professional performance plan

Utilizing the template described above, principals submit to the superintendent their baseline data, improvement targets and student engagement strategies. This document becomes the basis of their fall professional performance plan dialogue with the superintendent. Throughout the year, the template is added to and resubmitted to indicate progress toward the targets, and is used as part of the dialogue with the board of education.

Board dialogue

Prior to January, each principal engages in a 30-minute dialoque/ study session with the board of education and superintendent's cabinet. During this discussion, the principal shares the processes engaged at the site for analyzing student performance data and setting targets. Principals also discuss in depth their programs and how teachers are using specific engagement strategies to enable students to improve and therefore hit their improvement targets.

Beginning this fall, principals will also share how their site did in achieving the targets set out in the previous year. They will discuss with the board and cabinet their beliefs about why they were or were not able to meet targets, what was learned and how this has affected their strategies for working with staff in the current year.

This dialogue engages board members in the accountability system as active participants. The questions they ask and what they learn become part of the superintendent's feedback system for her and her cabinets' performance accountability. In the spring we plan to have principals make reports at a regular board meeting to summarize their school's efforts and subsequent performance on the assessments.

These components describe a yearly cycle of data collection, analysis, planning and reporting that will continue to evolve as needed and as we learn how to best utilize such a system. As a backdrop to these activities, the district also engages in a variety of goal-setting activities with administrators, the community and the board of education. These activities yield a document that describes strategic goals and objectives in five areas (learning for all students; safe, orderly, positive learning environments; quality staff providing quality service; school/home/community partnerships and communication; and acquisition and allocation of resources in support of goals 1-4).

As we enter into our second full year of working with the MUSD Student Learning and Accountability System we will have to test out, in our own inquiry, the degree to which the components and activities described above are manageable and more importantly, lead to real improvements in the teaching and learning processes of our schools.

One of the ways we will assess our accountability system is by assigning district-level administrators to each site. That is, for each of our 11 school sites we have a specified district administrator (including the superintendent) who will attend their meetings, participate in the analysis of student performance data, and work with the school staff as they implement the student learning and accountability system.

While it is essential that we engage in the public accountability systems mandated by the state and have a responsibility to share our outcomes with our public, we have perhaps a greater responsibility to ensure that all of this accountability and testing adds up to increased student learning. We are confident that this will only happen if teachers and principals feel engaged in activities they value and believe are in the best interests of students.

We believe that the accountability system described in this article is a start at finding the balance between compliance and deep systemic improvement. We also recognize that we are asking a great deal of our staff as these activities are added to an already full plate of educational enterprise. We will continue to test our system and are committed to modifying it to best meet the needs our community, our staff and most importantly, to help our students achieve the highest levels of learning possible.

(For more information or to receive copies of our data templates, strategic goals and objectives and other documents related to the Monrovia Unified Student Learning and Accountability System, please contact Joel Shawn at [phone].)

Joel Shawn is assistant superintendent of curriculum and instruction for the Monrovia Unified School District.