Growth measures for, systemic change: through periodic learning assessments, you can analyze which instructional programs are most effective, make student groupings and reallocate resources to areas of need.

Educators are becoming more aware of the limitations of testing that simply measures student achievement at a single point in time, such as benchmark tests, locally constructed formative tests, conventional standardized tests, and state assessments used to determine adequate yearly progress under No Child Left Behind.

Not surprisingly, school administrators are implementing assessments to measure individual student growth during the school year and from year to year. In states like South Carolina, Indiana and Minnesota, anywhere between 60 and 90 percent of the school districts statewide are using a formative assessment based on growth measures to make informed decisions about each student's education.

Accurate measures, combined with a vertical measurement scale, have provided an opportunity never before available to K-12 educational settings. Now not only do educators understand how their students are performing at one point of time, they can quickly see how much students have grown from quarter to quarter or from one year to the next. As a result, educators can understand and influence growth for all students, regardless of achievement status, age and class groupings. Likewise, growth measures point to how student achievement is aligned with district or state-defined content standards while ensuring instruction challenges each student appropriately.

The principal question is how growth data might be used to continuously improve the effectiveness of educational systems. Can we focus more clearly on each job function within a school district in ways that align those functions to improve student learning?

Access to accurate student growth data informs a wealth of decision making, including program and teacher effectiveness, the ability to challenge all students given their status, adequacy of instructional programs or resources, school staffing and scheduling, as well as the impact of environmental factors, such as bus schedules and classroom materials. With the use of accurate measures and timely access to the analysis of school/district progress, schools now can determine the amount and nature of academic growth that each student needs and then organize themselves to accomplish these learning goals.

Gauging Changes

At the most fundamental level, a growth measure provides educators a quantifiable way to gauge the difference in scores for a single student from one point in time to another. By comparison, the achievement level is the score that a student has at a fixed point in time, such as a standardized test score. The quality of a growth measure, however, depends on the integrity of the testing methodology and measurement scale.

Northwest Evaluation Association, a national nonprofit organization providing research-based assessments, has championed the use of computerized adaptive testing to measure growth in student achievement as the foundation for the Measures of Academic Progress, or MAP. It now is used by more than 2,200 districts nationwide.

MAP reports achievement on a RIT (Rasch Unit) scale, an equal-interval vertical measurement scale that enables educators to measure growth independent of grade level and to evaluate and compare performance data across years. The RIT is infinite, although most students' scores fall between the values of 140 and 300. The scale is equal-interval, meaning the distance between 170 and 182 is the same as the distance between 240 and 252. This allows educators to apply simple mathematical equations to the scores to determine the mean and median scores in a class or grade.

RIT scores are used by teachers to plan instruction around students' strengths and weaknesses relative to state curriculum standards. Educators can choose curricula that are aligned with NWEA's Continuum of Learning, and therefore it becomes easier to place students in well-targeted, differentiated instruction, which leads to improved performance and growth.

The aggregated results from periodic assessments, ideally implemented at nine-week intervals, provide a wealth of information for administrators. In an analysis of instructional programs, educators can use growth data to determine the effectiveness of one instructional program over another, make decisions about the groupings of children given their current achievement levels and instructional needs and reallocate instructional resources. Likewise, it is possible to see whether programs are adequately effective for all disaggregated groups of students or perhaps just one subgroup.

With an analysis of school data, juxtaposed to a virtual comparison group of like students and districts, administrators can make informed, objective decisions about teacher, counselor and staff performance. Tough questions about teacher preparedness, school staffing levels and student/teacher assignments can be addressed objectively.

Virtual Comparisons

The National Heritage Academies, which operate 53 public charter schools in five states, including Indiana, Michigan, New York, North Carolina and Ohio, is an example of an organization using the NWEA virtual comparison group reports. The academy schools began using MAP growth measures in all of its schools in 2003. Individual student scores demonstrate how each student is progressing and learning over time, and these scores provide teachers with guidance on where to place students in the curricula and how to adjust instruction based on student needs.

By selecting a virtual comparison group, including comparable students based on metropolitan service district characteristics, grade levels and socioeconomic status, the academy schools' administrators and teachers can see how student growth compares with students nationwide in similar surroundings.

To date, students enrolled in National Heritage Academies are showing growth at the 73rd percentile on the last fall-to-spring administration. More importantly, the comparison data, in combination with MAP test scores, has revealed specific instructional areas in both math and reading in need of improvement. Likewise, these schools now can obtain a clear picture of growth at each of its 53 schools, thereby identifying high-growth schools with replicable best practices and lower-performing schools where additional resources should be applied.

"The comparative data provided by MAP supplements the data we receive from our students' individual standardized state test results, allowing us to leverage this information to help all students achieve academic growth," says Robert Theaker, director of assessment and measurement at the National Heritage Academies. "Overall, we see where grades are hitting home runs, where instructional practice has an impact on achievement and where we can adjust resources to help both teachers and students. We can now target both our professional development and instructional practice efforts most appropriately."

From a building or district view, it is possible to learn whether time is used most effectively to generate student learning. Which schools are more or less effective? Are there environmental circumstances, such as long bus routes, meal services or facility conditions that have a negative impact on learning?

At the Beaufort County, S.C., School District, administrators have used growth measures to see how they can best use professional staff time. For example, the district discovered that when teachers did not have enough preparation time or adequate time to make thoughtful decisions about students and their instruction, there was a correlation with lower growth scores. The district thus changed structures regarding who does planning and when. Additionally, best practices were shared across buildings and the result has been improvements attributed to non-instructional factors.

The school district actively responded to its learning about time by implementing a calibration model for staffing, which ensures that adequate resources are in place to support learning. This model incorporates core, supplemental and intervention staffing requirements. By implementing this belief system, the district sought to increase the likelihood that staff are performing those roles most critical for student success.

A second strategy involved piloting a fulltime substitute teacher program to ensure instructional time would not be compromised when the regular classroom teachers were absent from the classroom for professional development purposes. These full-time substitutes were trained on district initiatives, familiar with all policies and practices and developed relationships within the district's schools. By addressing time in these ways, Beaufort County recognized the real issue related to time as a factor in determining student growth is about managing the time with students and staff well.

The Beaufort County schools combine results from the MAP growth measures with data acquired through a districtwide survey of professional and administrative staff, as well as food service employees and school bus drivers. The addition of data on process, as revealed in this survey, aligned with data on student growth, allows Beaufort administrators to make more informed decisions about school and district operations to produce greater student growth.

In short, an assessment built on growth measurement not only assures that educators can appropriately challenge all children and raise student learning, it also can provide the foundation for better decision making at a district level, thereby improving how schools are organized and programs delivered.

Answerability

The ability to organize educational institutions around individual student growth has to start with district leadership. School board members, superintendents and principals all play a role in setting policy, adopting programs and organizing the delivery of instructional services to meet the needs of children.

At the policy level, school boards must create a culture that is focused on the values and principles that foster constant improvement in learning. If the core values of the district are to organize and deliver instruction in a way that allows all children to be appropriately challenged, then specific curriculum, programs, schedule changes, teacher effectiveness or school performance can be assessed based on how well it fosters academic growth in students.

School boards, as well as district superintendents, then must own and manage a system of answerability. In the Horry County, S.C., school district, the superintendent and administrative staff would meet at least three times a year to examine student data regarding achievement, growth, growth targets, attendance, attitude and other criteria. (See related article, page 12.) The data are disaggregated across all of the important local factors, such as race, gender, program and status (high-achieving, average or low-achieving students).

Periodic reporting should help the school board to:

* Understand the relationship between student needs and resource allocation by building and program, and likewise, that teacher effort is appropriately focused.

* Have access to the context variables that are important to support effective leadership.

* Support staff development that is responsive to locally derived evidence.

* Support changes in school calendars and school schedules to assure the use of time maximizes learning.

Periodic reports not only keep a board current regarding school and district performance, they also inform how the district is ranked compared with like districts or with national averages. NWEA has created a data warehouse of longitudinal growth data. Although data are currently used by NWEA staff researchers and partner research organizations, one day the data will be open to individual school districts. Stripped of all student identifiers, these data then can provide a robust pool for creating virtual comparison groups. Districts can compare the costs and effectiveness of programs, organizational structures and the use of time or structure for grouping of students. In addition, the longitudinal data offer a quick comparison with state and national averages of student growth and achievement.

Robust Analysis

While the board fosters the culture and focus in a district, superintendents must manage both district policy and practice so that time, talent and resources are used to maximize learning. Few superintendents would argue with this mandate, and many are pouring their efforts into creating a model that will foster improved learning for all children.

Although a school district may recently have purchased a formative testing program, the data received through the administration of the tests may provide an insufficient measure of growth. Unfortunately, without a reliable growth measure in place, superintendents are ill-equipped to know what works and what needs to be changed. Which curricula are effecting the greatest growth? What are the best assignments for teachers, given their effectiveness lifting achievement in particular student subgroups?

All aspects of school or districtwide functions can have an impact on how much growth can be produced in students. In districts where outdated measures are in place, there is not quantifiable evidence to suggest what changes to make throughout the district.

Superintendents then become active champions for a culture that is organized around ways to optimize student growth. They own the mission of accelerating academic growth of every low-performing student and challenging every child with growth-provoking learning experiences.

For example, Linda Clark, superintendent of the Meridian, Idaho, School District, provides school principals and staff the license and encouragement to make structural changes to effect improvement in student learning. With this encouragement, educators can embrace change more eagerly, based on the evidence revealed in growth data from students. Clark cites several schools that have been successful in overcoming major odds to attain rapid and sustained improvement.

As the superintendent of Mesa County Public Schools in Grand Junction, Colo., J. Tim Mills created a culture focused on growth. Most evident is the manner in which school staff now engage parents in conversation about growth, resetting expectations about success in the school and how the district can measure progress.

Changing the conversation about education in the community, stimulated by the data available on how students are growing and what instruction, resources and staff are creating greatest benefit, similarly leads to systemic change focused on student growth in the schools.

Gerald Hill, superintendent in Glenview, Ill., joins his administrative team in systematically reviewing districtwide data on student growth. They encourage principals to review growth data with classroom teachers and other professional staff at the site level. By following up on their commitment to create growth in student learning, district staff are changing the culture through their communication and making sure everyone stays in alignment with district aspirations for raising student learning.

In districts where this model has been successfully implemented, superintendents and their executive teams use data from assessments to:

* See that all students receive instruction that is challenging.

* Allocate resources by program or school based on student needs.

* Assign teachers so that appropriate talent is assigned to each school based on the student needs of the school or program.

* Provide for staff development that is responsive to the evidence within each school setting.

* Meet with each school principal at least three times a year to examine student data and local efforts to improve.

The ability of superintendents to be effective in their role is dependent on timely and accurate reporting, robust analytics and student growth data over time. In turn, principals need the context variables that are important for leadership. This includes the necessary flexibility and support to respond to the rapidly changing needs of students throughout the school year.

Technology Platform

Certainly it is the quality of the growth measure as well as the timeliness of both the assessments and reporting that allow administrators to implement systemic change. However, the integral role of technology cannot be overlooked. Only through the advances in computer and software technology over the past few years have school districts been able to realize the potential for using growth data to create meaningful change.

Most testing companies are offering computer-based or online assessments, but many lack capacity to measure growth. Others are offering assessments that are computer-based adaptive tests, which present test items to students adjusted for difficulty depending on how students have responded to previous questions. Likewise, current technology returns test results immediately to teachers and administrators.

As a best practice, administrators need desktop access to online information and the ability to query data to see program effects in producing growth. Online query tools allow administrators to disaggregate the growth data at any level. With this insight, administrators can evaluate the effectiveness of school programs, policies and functions and resource allocation.

Administrators also can compare and analyze district, school and individual growth targets, spot district or school-level patterns and trends and quickly drill down through groups to pinpoint student data.

Newer technology also is allowing organizations like NWEA to import additional data into its longitudinal data set, to access all known district data, regarding attendance, schedules, personnel and finance. It is also possible to import non-traditional data sets, such as U.S. government census data, to more fully inform decision making.

Future Measures

Educators continue to have a healthy conversation about student achievement, school performance and the nation's ability to serve the educational needs of all children. Clearly school districts want to produce an environment where all children are appropriately challenged, able to realize growth in learning year after year and can achieve state-mandated benchmarks.

Benchmark tests, locally constructed formative tests, conventional standardized tests and state-mandated assessments only demonstrate where students are performing at one point in time. Without adequate information about the activities, programs, curricula and educational structure that produce the greatest growth, it will be difficult for administrators to effect and sustain growth in student achievement, that is to create the systemic changes that will produce long-lasting benefits to students and their learning environments.

RELATED ARTICLE: Growth data: answerability for continuous improvement.

BY GERRITA POSTLEWAIT

Imagine the school board has established these criteria for the superintendent's next performance evaluation. Next June the board members want to know:

* How much did students in our district learn this year?

* At what levels were students performing in reading, language arts, math and science when they started and how have they grown over the course of the year? How are our students advancing compared to other students in the country, given similar demographics?

* In which schools are students experiencing the greatest gains in achievement? Are all subgroups of students demonstrating adequate growth? Are student achievement gains visible in every classroom, in each grade level, in every school?

* Which curricular programs or instructional methods appear to be yielding the greatest return on investment in terms of growth in student achievement? Include Title I, special needs and other pull-out programs in your analyses of results.

In the past, it would have been difficult, if not unfeasible, to ascertain real-time answers to these questions, but now, thanks to new technologies, such as computerized adaptive assessments that measure and instantly analyze how much student growth is occurring for every student during each quarter, superintendents can address these concerns with confidence.

A Changed Conversation

A decade ago, Horry County Schools, a district of approximately 36,000 students located on the coast of South Carolina, became mired in the arduous tasks of defining "world-class" standards, constructing assessment items and personalizing instruction for each student. Teachers felt overburdened with developing, administering and scoring benchmark assessments and developing a system to organize and deliver personalized learning plans for every student.

Then, five years ago, Horry County partnered with the Northwest Evaluation Association, a nonprofit organization that aligns district or state learning skills and performance objectives with a computer adaptive assessment system. Within minutes of a student completing a test, the classroom teacher electronically receives a diagnostic analysis that tracks the amount of growth the child has attained during the previous quarter. NWEA has embedded its assessments into the curriculum so that when a teacher double-clicks on a student's name, he or she can instantly access an individualized learning plan for the following nine weeks, including the performance standards this child has mastered plus the specific skills and concepts the child needs to learn next.

Through this new sophisticated monitoring system, we acquired the means to measure and analyze growth, to engage all educators in richer, deeper, more focused data discussions, to personalize instructional delivery and to instill a sense of responsibility for student success among everyone in the school system.

School Usage

As a result of growth data, teachers and principals are thinking differently about their work. After each assessment, they review the performance of individuals, classes and grade levels of students to analyze patterns of growth and make decisions about which approaches are most effectively advancing students.

At the elementary level, several schools decided to split mathematics and/or language arts education into two time periods. One period focuses on teaching curriculum standards at grade level to meet state assessment requirements; a second period is set aside for students to be grouped according to proficiency level versus age or grade level. This process assures students receive instruction in math and language arts that delivers both the state-mandated grade-level standards and instruction tailored to each child's current level of performance.

In many cases all certified personnel in the school teach a group of students during the second, personalized instructional period to reduce pupil/teacher ratios. This customized distribution of resources around student needs has resulted in impressive achievement gains.

At the secondary level, a defining moment came while teachers were reviewing assessment data for Algebra I students. They determined that all students, especially those at risk of failing, would benefit from a more flexible use of time, so they increased the number of sections of Algebra I offered during a designated time block, thus allowing students to be regrouped frequently so they could move through the curriculum as they mastered essential concepts and skills. They also provided targeted support to students who needed it.

These types of thoughtful, innovative responses to identified student needs are not yet universally implemented in Horry County schools, but they provide a model for being systemically "answerable" for student success. The conversations about student growth data represent a partnership of mutual responsibility for student, school and district success. Leaders must remove barriers and provide support to help implement plans schools have crafted in response to identified student needs.

Leaders' Answers

Not only do teachers and their students benefit from timely access to growth data, but principals also gain a better picture of where and how students are performing in their schools. Within 72 hours of student assessment, NWEA provides administrators thorough analyses of growth at the individual, class, school and district levels. The data can be easily aggregated and disaggregated around many factors, providing leaders a basis to see growth of different subgroups of students in different classroom settings.

At the school level, faculty members meet with principals in Horry County periodically during the year to review data they have gathered concerning student learning that has occurred in their classes. By examining data showing student growth during a quarter or semester, principals and teachers can spot patterns and identify troubling trends, using the information they've gleaned to make real-time, strategic modifications as needed.

Principals meet individually with the superintendent at least three times a year to summarize what they've learned about student growth in their schools during the past quarter. Probing questions such as these are provided in advance of the conference: Review the evidence that shows that every student in your school experienced growth in academic achievement during the past nine weeks. What approach seemed to produce the greatest learning? Where are your greatest concerns, and how do you plan to address them? How can the district help?

Return on Investment

District staff in Horry County view their primary work as helping schools remove barriers that impede success for every child. Staff members in each department have accurate data to determine the extent to which their operations contribute to student success. Through the use of growth data, directors of specialized programs that provide targeted assistance to students determine the impact of their interventions on student achievement. We can parse out the gains of students who participated in specific instructional or curricular approaches in order to help determine the return on investment with respect to the funds expended for products, services or personnel.

District administrators also use the growth analysis tools to discern patterns of widely varying rates of growth from site to site. Sometimes the data reveal that schools previously thought to be high performing are actually producing far less growth for students than schools with more challenging circumstances.

In addition, a unique feature of the MAP assessment program allows our district to determine whether our students are performing compared to similar children nationally. NWEA is able to create "virtual comparison bases" that tell us whether our students are faring as well as their counterparts in other areas of the country.

Although the work is complicated and complex, the school district is committed to building a system that accepts responsibility for the continual progress of all students, for helping lower-performing students achieve more while at the same time making sure all students' needs are being met at every level.

Growth data enable educators at the local level to engage in conversations, reflections and actions that represent a partnership of mutual responsibility--answerability for the success of all students.

Gerrita Postlewait is a superintendent in residence with the Stupski Foundation, 2 Belvedere Place, Mill Valley, CA 94941 E-mail: gerritap@ stupski.org. She served as superintendent in Horry County, S.C., for 10 years.

RELATED ARTICLE: Taking a measure of the measure.

BY DOUGLAS D. CHRISTENSEN

Imagine for a moment that you have a 12-inch ruler in your hands. As an instrument of measurement, we depend on this ruler. We depend on it to be accurate, valid and reliable. The same would be true for a yardstick, a 12-foot tape measure or other similar measuring devices.

The 12-inch ruler, however, has its flaws. If I need to measure something longer than 12 inches, it is possible for errors to creep in. In fact, if I measure something where I have to lay the ruler end to end several times, the width of the line I use to continue my measurement will soon be a factor in the overall length.

And if I need to lay the ruler end to end say 100 times, fatigue of the measurer will soon enter the process. So our 12-inch ruler isn't universally accurate after all. For things that are 12-inches long, it is excellent. Nothing is better. However, measurements of more than 12 inches, and also less than 12 inches, are problematic and prone to estimation and error.

There is an enduring principle of measurement: Measurement freezes the measure and what is measured in place. Don't believe it? Think about the metric system vs. our system of inches, feet and yards. We are probably the only advanced country in the 21st century that does not use the metric system. And what if the 12-inch ruler was the only measurement tool for length we had? How long would it take before virtually everything became 12 inches long or some derivative of it? My guess: Not very long.

Accuracy Compromised

Let's go back to our 12-inch ruler What if what we had to measure was not a flat, straight-line measurement but was circular? Again, we have a valid, accurate and reliable measure in our 12-inch ruler, but it compromises its validity, accuracy and reliability when applied to situations for which it is not intended.

Remember [pi][r.sup.2]? To find the circumference of a circle (the distance around it), you multiply the radius squared times [pi]. If you remember from your high school math, [pi] (pronounced "Pi") is an estimation that has an infinite set of numbers to the right of the decimal place. Once again, we could probably easily measure the radius or diameter but using the algorithm to calculate circumference introduces error even though the tools used are accurate, valid and reliable.

One more example. What if we are trying to find out the distance between Lincoln, the Nebraska state capital, and Chicago. We could use our 12-inch ruler or a yardstick. Or we could use the odometer on our automobile. Which is more accurate, more valid and more reliable?

The odometer is probably least accurate due to being affected by inflation of tires, pavement heat and calibration at the factory. If I set out to find the distance to Chicago using my ruler, how much error will I introduce when I have to lay the ruler end to end, draw a line and repeat it a zillion times? How much error will I introduce as I get tired of bending over and drawing the line to move the ruler one more time?

Given the fact that my odometer is not precisely accurate, the factors of fatigue of the measure and fatigue of the measurer will likely not enter in. Which is the more accurate and reliable measure of this distance? I would opt for the odometer.

The real question here is not how many feet or yards it is between Lincoln and Chicago, anyway. What use is it to know how many feet it is? It may not even be relevant to know how many miles because what I probably want to know is how long it will take me to get there. While I can convert the feet and yards into miles and figure the likely speed at which I will be traveling, the odometer reading is much more useable and much more relevant to the question I am trying to answer: How long will it take to get there?

Dynamic Tools

So what is the point here? So much of traditional measurement is caught up in this analogy. We assume that standardized tests are like rulers in terms of their accuracy, validity and reliability. In some ways they are. They measure things that are "inch-like" or "foot-like" best but do not measure well the things outside of those dimensions.

Maybe we should focus less on the instruments of measure and focus more on the information they provide and how we use the information. Can we trust that the information we get is the ultimate question of accuracy, validity and reliability?

Measuring learning cannot be done with rulers. Measuring learning takes dynamic measures that change with the outcomes to be learned, with the instruction provided and with the students being taught. Measuring learning requires gathering information from multiple indicators of what students know and are able to do before we jump to the conclusion that learning has occurred (or has not occurred).

If we want rote content learning, then we can use our tools of inches, feet and yards to determine whether students measure up. If we want learning that is more about thinking skills, problem-solving skills and information/ technology mastery skills, then our traditional tools won't work. We need tools to match the things we are trying to measure.

The tools we need the most to measure real learning are most often not pre-manufactured. One does not go to the test store and buy one off the shelf. Dynamic learning requires tools that are built for the purpose and are available on the spot. Who best to build these tools? Teachers! And where's the best place to build them? Classrooms!

Doug Christensen is commissioner of the Nebraska Department of Education, P.O. Box 94987, Lincoln, NE 68509. E-mail: doug.christensen @nde.ne.gov. His column appeared previously in the Nebraska Council of School Administrators' newsletter.

Allan Olson is president of the Northwest Evaluation Association, 5885 S.W. Meadows Road, Suite 200, Lake Oswego, OR 97035. E-mail. [e-mail]