Data-driven districts: four districts that take different tacks using data to inform key decisions.

In business and industry, data are king. Information about customers, inventory, sales, rates of return and employee turnover is crucial. Good data help determine success or failure.

Education, of course, is a different animal. Its essential product is children, who aren't at all like mass-produced widgets or numbers on a spreadsheet. Kids are real, data are abstract--and for some educators that makes it a four-letter word.

"Not many educators buy into or pay much attention to numbers in graduate school," says Philip A. Streifer, a former superintendent and now an associate professor of educational leadership at the University of Connecticut. "Dealing with data is a subject that makes people uncomfortable. It causes a good deal of anxiety."

Yet the school reform movement increasingly demands just that. The call for greater accountability means administrators and teachers must show proof--tangible, statistically valid evidence--that what they are doing is working, that students are learning faster and better.

Rapid Spread

The old tools of education--intuition, teaching philosophy, personal experience--do not seem to be enough anymore. Virtually every state has put into place an assessment system intended to measure and validate student achievement and school performance.

The passage of the No Child Left Behind Act only raises the stakes further, adding new responsibilities for states, districts and schools to accurately and promptly collect, analyze and report data on student demographics and achievement. Federal law now requires annual state reports on student performance by gender, race, disability, income, migrant status and English fluency. States also must report how many students were not tested, in the same categories, and provide detailed information about the professional qualifications of those teaching the students, such as the percentage of classes taught by teachers lacking certification in their subjects.

"Right or wrong, external accountability is coming to everyone," says Katherine Gemberling, an educational consultant and former deputy superintendent in Montgomery County, Md. "You can't simply mandate educational quality and order up tests to make sure it happens. But the fact is, external accountability models exist because educators did not step up themselves and establish definable measurements of quality. ... Educators feel compelled--they are compelled--to look at anything that will help them show they're getting good results."

In other words, like good businesspeople and well-run companies, educators and school districts are being asked, if not expected, to prove their bottom line with hard, solid data. With passage of No Child Left Behind, school districts large and small have taken up the banner of data-driven decision making.

"We have seen an increased interest by districts in DDDM," says Geannie Wells, former director of AASA's Center for Accountability Solutions. "We saw the momentum building before No Child Left Behind and then definitely a heightened interest after."

But no revolution runs smoothly or easily. There are always missteps. Though districts and states generate huge amounts of data, many remain ill equipped to make the most of it, to effectively employ that information in policymaking or in reporting to federal education officials.

"Ownership of data has been a problem," Wells says. "Some data reside at the state level, the district level, the school level or even with vendors in proprietary formulas. Incompatibility of various data systems has been a problem. Often enrollment data can't be tracked with student achievement data or program participation data, etc. Inadequate funding and technical support for data collection have been challenging. And data from testing companies and states are often not disaggregated to such a level that they can be useful and are not available in a timely fashion."

Only 16 states, for example, have systems that give each student a unique identification number that allows them to be tracked and assessed over their entire school career. Only 17 states provide demographic breakdowns of student achievement by race or family income on school report cards.

Slow Progress

"Everybody isn't starting at the same place," Gemberling says. "Everybody doesn't have the same tools, whether that's the technological infrastructure or the people trained to handle it. ... You can't just crunch some numbers and expect that this will lead to effective, real-world decisions. How, for example, do you measure growth, teacher effect, student ability? What is evidence of quality? Good data about these things require more than just standardized test results."

Streifer knows those difficulties well. As superintendent of the Avon Public Schools in Connecticut in the 1990s, he came face-to-face with the hard realities of data--or lack thereof.

Like most school districts, Avon collected lots of information. But much of it was organized or reported in ways that rendered it hard to use or worse. "Data collection is a messy, messy business," Streifer says. "It's done in different formats, sometimes electronically, sometimes on cards or paper. Often it's incomplete. Teachers collect it differently, and not everybody has the same access to it, which means not everybody is going to be on the same page."

During his time at Avon, Streifer implemented a series of organizational changes. "The goal was to change district culture, to focus on how to use data to make improvements, raise achievement across the board, including programs like music, art and sports," Streifer explains. "I took a systems approach.

I looked at various operations of the district, how they planned, conducted program evaluations, the kind of data they looked at. I asked questions about the degree to which each program was supported by good organizational structures, what links existed to professional development."

Since leaving the superintendency, Streifer continues to promote data-driven decision making. At the University of Connecticut, he further developed his model, collaborated with his Avon successor, Richard Kisiel, on projects, and completed a book, Using Data to Make Better Educational Decisions, published earlier this year by AASA and Scarecrow Education.

Data-driven decision making, Streifer says, is the inescapable future of educational administration. One-fifth of the coursework in the educational leadership program at his university is devoted to the subject. "That's a huge departure from when I was trained but times have changed. People realize education can't be pushed forward any more based just on hunches. You've got to have good information thoroughly analyzed."

But getting good information thoroughly analyzed can be daunting and difficult. There are issues of cost, training and sheer complexity. Streifer says superintendents have told him they didn't think data-driven decision making would work for them because it looked too complicated or they feared their principals couldn't handle it.

But that's no reason not to try, he says. "You've got to have a healthy skepticism. It's possible to abuse or misuse data. You can draw some dangerous or wrong conclusions if you're not careful. But on balance, I think it's better to try than to not try."

What follows are snapshots of how four different school districts around the country have tried to use data in meaningful ways, each grappling with the real concerns and benefits of data-driven decision making.

Plainfield Public Schools

The Plainfield school system is a so-called Abbott district, one of 30 districts in New Jersey identified by the state following a series of landmark New Jersey Supreme Court rulings in 2001 as historically underfunded and of special need.

Plainfield serves roughly 7,600 students, plus another 900 3- and 4-year-olds in prekindergarten programs. Almost three-fourths of the student population are African-American with the remainder mostly Latino. Sixty-five percent of the students receive free or reduced-price lunches.

As a result of the Abbott decisions, Plainfield now receives additional state funding, but the money comes with heightened expectations. For example, the court mandated that every Abbott district pursue a standards-based education driven by state content standards and supported by per-pupil funding equal to spending in successful suburban schools. The court said that new or supplemental programs would be expected to "wipe out student disadvantages" that are often endemic in poorer urban districts.

But Joshua Starr, Plainfield's director of accountability, says district trustees and administrators didn't want to build the requisite accountability system based solely on standardized test scores. They wanted one that could be readily evaluated by external entities but still function well internally. "You have to have a system that external entities like the state or the public could use and understand," Starr says, "but also one that is embraced by those who have to work within it. We wanted a system that engendered the best aspects of internal accountability, that promoted critical inquiry based on accurate data by all stakeholders, that had regular checkpoints but in a nonthreatening environment for decision makers."

The initial challenge was simply gathering information--raw data that could be used to build ideas and programs and to make decisions or policies. Among those in the information management business, it's called "data warehousing," a term that aptly describes the fundamental idea.

"Basically, what you're trying to do is build a giant central repository of knowledge, a database that is accessible to everyone in the district who can make use of it to benefit students," says Wells. "The challenge with schools is that data typically reside in so many places, all of them different. It takes a huge commitment of time, funding and energy at every level to build an effective data warehouse. Some large districts have done so, but not many.

"I went looking for data and there were not any," says Starr, "not unless I wanted to print out reams of material, then sit with an abacus crunching numbers. So I contracted with E-scholar (one of numerous educational software companies with data management programs) to help us gather and organize data--student information, attendance figures."

Starr calls this kind of information "electronified," meaning it's already in a form that is relatively easy to collect and assess. However, its value can be limited. Attendance figures, for example, only reveal so much. To extract real insight demands as much information and as much diversity as possible.

"It's an ongoing process. You're always trying to add other kinds of data," Starr says.

Tougher but potentially more valuable data include things like staff development information or lesson plans. "Other kinds of material that are harder to collect are less obvious," he adds. "It all helps. If you use it right, it can identify patterns and trends."

For example, Starr says he can take simple data like attendance and test scores and combine them in different ways, broadening or narrowing the point of interest. "I can look at student achievement results on different tests, then break them down by gender, grade level, ethnicity."

Adding more particulars sharpens the picture.

"If certain students don't seem to be doing as well as they should, for example, we can change the parameters to look at common factors that might be affecting them, such as class size, how many are in the free-lunch program," he says. "It doesn't necessarily provide answers. Often it just leads to more questions. But you have to be asking the right questions to get the right answers."

And what's the right question? Starr says it's simple: "Does this effort ... increase the opportunity for kids to achieve?"

Answers vary, of course. And that can be a rub, he adds. Scrutinizing data, looking for patterns and trends, sometimes leads to multiple interpretations, a fact that causes some people much consternation.

That's when people fall back on test scores or newspaper stories as their preferred means of evaluation. "It's a constant struggle to push ahead. We've set up a communitywide task force on accountability with staff support. When schools do their end-of-the year reports, we present them to the community. We explain what was done, how and what to expect next. Still, it's a struggle to convince people that test scores are not the only indicator of school quality. They are just one indicator.

"For years, people at the school-site level have said, 'Don't judge us just on test scores. They don't show all that's going on,'" Starr says.

"The question is in what other ways can you present reality. People always ask 'What are the data?' We've learned to be flexible, to allow people to have success and show it in other areas. For example, a school might show that it had excellent results getting parents involved or that its kids were writing at a higher level. These things wouldn't necessarily be reflected in a single test score.

"We allow them to use whatever data best tells their story in relation to the rubric. This motivates people to collect data. It increases the use of e-mail and Excel to create spreadsheets. Every school does an end-of-year report. Generally they all contain the same kinds of evidence--attendance figures, test scores--but some schools get really creative. One middle school, for example, did a bunch of surveys of parents, teachers and students. Another counted all of the book titles in the school library to check on diversity of authors and subjects, then put out a report. If we at the central office had dictated that idea they would have balked, I'm sure, but because it was their idea, they pushed ahead on their own and it was wonderful."

Starr concedes not all schools or school leaders have accepted the system with equal enthusiasm. "Some schools have not embraced it in the way we hoped," he says. A good chunk of that reluctance can be attributed to varying degrees of comfort with the process. "Everybody is different. Everybody comes at it from a different background, different training. It's an ongoing thing. I have conversations every day with principals, teachers, even parents."

Collecting and dissecting data are not ends unto themselves, Starr says. They are processes that, hopefully, lead to better decisions. "The system isn't about where you are, but whether you're moving, progressing. We can't necessarily control student outcomes, but we can control issues like whether kids are safe, teachers are well-trained and there are adequate resources that will help students meet standards and expectations."

And all of that requires data.

Palo Alto Unified School District 

The 9,500-student Palo Alto school district sits on the northern edge of California's high-tech, high-profile Silicon Valley, serving the city of Palo Alto, part of Los Altos Hills and the Stanford University community. Seventy-six percent of the annual budget derives from property taxes, and the general fund expenditure per pupil in 2000-2001 was $8,943.

"We are obviously among the wealthier districts," says Cynthia L. Pino, associate superintendent for educational services, "but we also have the economic and ethnic diversity you find in other districts."

Given its surroundings and clientele, it is no surprise that Palo Alto long has touted the benefits of data-driven decision making. "When I came to the district in 1999, data-driven decision making was already part of the district's list of goals and a lot of effort had already been made," Pino says.

"But the data were mostly aggregated, monolithic collections of information that were hard to break apart or use when seeking particular answers. In recent years, however, Pino says the district has pushed hard to disaggregate the information, breaking it down by gender, socioeconomics, ethnicity and other categories.

In the late 1990s, the California state legislature passed laws outlining new pupil promotion and retention requirements. The laws were intended to discourage and eliminate the practice of social promotion based on factors not directly associated with the student's mastery of grade-level content. Specifically, students would be retained if reading performance was below grade-level standards in the 2nd and 3rd grades or reading, writing or math performance was below grade level standards in the 4th, 5th and 8th grades.

Pino says district officials looked at the new laws and decided they needed to broaden how they measured student progress. "We've never locked ourselves into norm-referenced tests," she says. "We like and want multiple measures not based on any one assessment. This includes teacher judgment."

Even before new promotion and retention requirements, Palo Alto had intervention plans in place to identify students whose achievement was below grade level. But the Palo Alto board responded by adopting a more rigorous policy, one that not only identifies students but ensures each is linked to a "quality intervention," says Pino, such as after-school tutoring and summer literacy labs.

Called "Kid by Kid," the program is clearly data-driven and data-intensive. It draws upon a host of informational sources: state-mandated tests, content standards tests, writing tests and various diagnostic tests that are given at the beginning of a school year to establish baseline abilities.

"One of the goals is to be able to monitor the child throughout the year," says Becki Cohn-Vargas, director of elementary education. "We want teachers to be able to look at what's effective and what's not."

It's a huge undertaking, one that the district has been steadily working at for several years. "It's an evolution," CohnVargas says. "You have to balance the kinds of information you want with how much you want to test. You've got to figure out what to do with the data, then train teachers how to use them."

Evolution works slowly. Some principals and teachers want more data. "They ask for more analysis, which can be hard to provide, but is a wonderful problem to have," Pino says. Others, though, feel they are being given more information than they need or the wrong information.

"We try to make the data useful and friendly," Pino says.

But obviously it's a work in progress. Individual school sites cannot directly access data. Instead, principals must request what they want or need from the central office, though Pino says new software eventually will make it possible to convert the system to a Web-based service.

"We've come a long way, but then the board and district have made this a priority for a number of years, long before I got here. They recognize the value of good data here. It's part of the culture."

Francis Howell School District

The Francis Howell School District sprawls over 250 square miles of eastern Missouri, northwest of St. Louis. The student population is big, too, 18,600 students, making it the fifth largest district in the state.

A few years ago, Francis Howell had big problems, too. For 18 months in 1997 and 1998, an outdated, inadequate computing system spewed inaccurate data about student attendance. Rather than counting kindergarten students based on instructional hours, as state funding formulas required, the system recorded them as full-time students.

"The district's mainframe was running old programs on old servers that were operating at way over capacity," says Superintendent Dan O'Donnell. "The information wasn't stored or backed up off-site. And nobody was verifying it, making sure it was accurate."

The result: The state overpaid Francis Howell by $7.5 million. Worse, district trustees and administrators had based spending on these erroneous projections, which suggested they needed a lot more teachers and resources. When district auditors discovered the problem in 1999, officials knew they faced a huge problem: The state would want its money back and the district, which had over-hired, would be forced to make painful cuts in personnel and services. The crisis prompted one superintendent to depart on a medical leave. His replacement retired less than a year later. Then in 2000, O'Donnell took over.

He and the school board negotiated a repayment program spread over five years. Then they took the first steps toward making the necessary cuts that would free up money for the state.

Bad data collection had gotten the district into trouble. O'Donnell figured good data collection could help get it out. He junked the old system, bought new servers, expanded storage and hired new information and systems personnel. Then he started collecting all kinds of data, running them through different test situations, "We started using data the correct way, asking how the information could help us decide what was best for kids, what cuts we could make that would do the least harm."

It was a painful paradigm shift. "Management in the district had been by instinct," O'Donnell says. "People made decisions based on what felt like the right thing to do. Often that meant just pushing more money at something."

One immediate improvement was accuracy.

"Under the old system, we couldn't even get accurate staff counts or faculty lists. They would include retirees or short-term subs, for instance, or inaccurate information on years of service, tenure status and salary step," says O'Donnell.

The problem of bad information extended all the way down to basic student attendance and enrollment information. "Not only was the data flawed, but it often took weeks to get a report out of the system and there was no verification process to test accuracy."

The district's revamped system corrected these problems and helped point to where cuts could be made. Many district personnel left through attrition. Most positions were not refilled. The few new hires, O'Donnell says, were required to show at least some familiarity and acceptance of data-driven decision making.

For those already on staff, the district began an extensive training program headed by a new chief information officer, Kevin Palmer.

"Part of that process included a training program for all staff in our central office, out to all building administrators and then to all faculty and staff on the program components they needed to deal with, such as grading, discipline, data use and analysis. Professional development is a key to the continued success of the system."

Things are looking pretty good at present too. In just three years, the district trimmed $23 million and paid off the state early.

But O'Donnell says that's not the end of the story. The availability of good data--and people who know what to do with them--means the district can now find ways to recover cut programs. "We had a good reputation and lofty goals, but we lost a lot of credibility when the public learned we couldn't manage our money. That's changed and we re now trying to rebuild that trust."

It might take awhile, he says, but the district will get there.

He's got the numbers to prove it.

RELATED ARTICLE: Providing Data to Your Board

Linda J. Dawson

Latent fear has surfaced among superintendents nationwide over one of the more current requirements in school reform: data-driven decision making by their boards.

Why would data-based decisions cause such angst among school system leaders? They cited the following reasons during focus groups that we conducted over the last three years:

* "What will board members do with data when they get them? They'll use them as a tool to harm me!"

* "If we start giving them data, will they want them to the classroom level?"

* "What happens when we have the inevitable 'dips' in performance?"

* "My job is to know the data--their job is to support me!"

Some of these fears are understandable. Others are misguided notions. However, in-the-trenches experience has taught far too many superintendents to be justifiably wary of giving board members student achievement data.

But the times are changing and the savvy superintendent now realizes that good data, presented in the context of board-adopted and districtwide goals for student achievement, can move a board to focus on core performance issues. Those include supporting administrative means to improve achievement while advocating for moral and fiscal support of district efforts throughout the community.

So how should superintendents present data to their boards? Here are four guiding principles:

* Insist the board establish districtwide goals or end results for student achievement.

Insist is a strong word--purposely. The superintendent plays a leader-servant role to the elected or appointed lay board. Board members often do not know what their role is individually or collectively. They will come on board and grab hold of what they think they understand and can do. Instead of dealing with these misguided missions, future-focused superintendents lead them into harnessing that energy into their No. 1 job--ensuring community-based values are established around expected student outcomes.

* Assist the board in receiving targeted, customized training on what data are and are not to measure whether student outcomes are being achieved.

Board members sometimes comment: "I don't need Statistics 101." "Numbers can be twisted to say whatever the administration wants." "How do we know if we're getting good data?"

Some superintendents will bring in an acknowledged expert to provide training on what data are and how to think about them in connection with districtwide student outcomes. You should ask the outside expert not only to define and explain terms but to provide tangible examples and case studies.

Ask the consultant to lead the board through scenarios in which members collectively analyze various graphs. Have them brainstorm on what questions come to mind in this hold-harmless environment, free of possible embarrassment at a public meeting. Finally, have the consultant bring them to closure on what data they will want to see and when to demonstrate progress on student outcomes.

A Conduct Code

* Insist the board adopt a code-of-conduct policy on data use.

Because the misuse of data is a possibility, bring the board a code of conduct that governs members' behavior and constitutes accountability to themselves and to you. Put the code into governing policy. Include such statements as "I will respect the confidentiality of privileged information;" "I will use data to represent all of my constituents honestly and equally and refuse to surrender my responsibilities to special interest or partisan political groups;" and "I will request data as a board, not as an individual, unless the information is readily available and will not redirect staff time."

* Provide the data on time, with an executive summary and in an understandable presentation.

Be honest in describing what the data say about achievement or the lack of targeted achievement. Deliver the data on the agreed-upon dates for monitoring. Logically this should occur throughout the year as data arrive in the district. Board members want to see the data, if possible, before public consumption and possible misinterpretation.

Arm board members with reality by giving them your executive summary analysis about gains and deficits and what you intend to do about them so they know the game plan. Caution: Do not invite or tolerate their dictating the means to improvement. That is up to the professionals to make happen.

Forego the "dog and pony" show presentation. Instead, focus attention on the understandable points and lead board members in thoughtful, analytical discussion. Make the presentation one that can be used as they interact with community groups.

Resources

* "Improving School Board Decision-Making: The Data Connection." A practical guidebook with an accompanying video or CD. Contact National School Boards Foundation at 703-535-1604.

* www.schoolboarddata.org. A website of resources, including sample policies, a print and Powerpoint training module, role plays for group presentation, sample data collection presentations and articles.

* On-Line Training. "Improving School Board Decision Making: The Data Connection." Contact NSBA at 703-838-6722.

Linda Dawson is director of the National School Boards Foundation Data Project and is president of the Aspen Group International, P.O. Box 1177, Castle Rock, CO 80104. E-mail: [e-mail].

The Lingo of Data-Driven Decisions

Data-driven decision making, like most specialized endeavors, has a language of its own, some of which can seem quite daunting. Below is a glossary of some key terms drawn from Using Data to Improve Schools: What's Working, published by AASA in 2002.

Data mining: The analysis of data for relationships that previously have not been discovered.

Data warehouse: A central repository for all or significant parts of the data that a system collects. A data warehouse typically is a database or collection of databases existing in virtual, rather than physical, space.

Decision support system: A computer program application that analyzes and presents data so users can make decisions more easily.

Disaggregated data: Data broken down by specific student subgroups, such as current grade, race, previous achievements, gender and socioeconomic status.

Formative assessment: Assessment in which learning is measured at several points during a teaching/learning phase, with the primary intention of obtaining information to guide further teaching or learning steps. Formative assessments include questioning, comments on a presentation or interviewing.

Longitudinal data: Data measured consistently from year to year to track progress, growth and change over time. True longitudinal studies eliminate any students who were not present and tested in each of the years of the study.

Perception data: Data that inform educators about parent, student and staff perceptions about the learning environment, which could also reveal areas in need of improvement.

Qualitative data: Data based on information gathered from one-on-one interviews, focus groups or general observations over time (as opposed to quantitative data).

Quantitative data: Data based on "hard numbers" such as enrollment figures, dropout rates and test scores (as opposed to qualitative data).

Summative assessment: An assessment at the end of a period of education or training that sums up how a student has performed.

The Will and the Way of Data Use

Lance Alwin

I am a peripatetic leader. As I walk about, observing and engaging in conversations with members of our educational community, I am absorbing, albeit unconsciously, mountains of raw data. Almost imperceptibly, this process will not only serve as the foundation to necessary and meaningful relationships with others (because of how the data were "gathered"), but the data itself will become the grist of future decisions relating to any number of matters.

Notwithstanding my personal process, data--how they are collected, how they are used and how they affect the acquisition and use of resources--are a big topic of concern in our 3,000student school district.

In our unique instance, we have come to care dearly about the impact that data collection and decision making are having and will have upon the culture of our educational community.

Changing Perceptions

Our rural district had experienced several decades of unsuccessful referendum efforts to build new schools.

Once we received training from AASA about using data to make decisions, we saw an opportunity to rebuild the critical community-school connections that must exist if we were to receive support from our community that, among other things, translated into needed resources. The result has been increased parent volunteerism in and engagement with schools, greater family participation rates in after-school programming and significant increases in local business partnerships. The basis for these improvements was data collected via surveys and focus groups wherein we simply asked the community members, "How can our schools be made to feel more welcoming to you and your needs as parents, families or community businesses?" To say the least, we were surprised to discover they possessed many answers, which up to that point we hadn't heard.

We knew that providing data to our public meant that they had to ring true. They didn't want more "double talk" from board members and administrators. The only way that would happen was if community members had a hand in creating that information and then were invited to talk about the results.

So we collected data at two distinct levels: at the individual school level and the community level. At the school level, for instance, we collected data on the school readiness of children, which in turn drives local preschool programming. At the district level, we collected data regarding community child-care needs, which fueled policymaking efforts with regard to the issue of school readiness.

As a result, these data felt personal to members of our community. They became the basis of our ongoing community conversation about the shared values and shared vision of who we are and how we do business. In a word, it created the "community of will" for how we would proceed as a school district.

A Means and an End

As the saying goes, the proof is in the pudding. Since embarking on this more community-sensitive, data-driven course of interaction, our community has supported two long-term capital improvement and operations referendums. They also have backed state and federal grant applications by our district for 21st Century Community Learning Centers, Student Achievement Guarantee in Education and Head Start programs, knowing full well that when those grant dollars subside, our district will need to find new funding streams to maintain those worthwhile learning programs for our children.

We continue to bear a trust within the educational community to provide information about how we are doing (accountability) that emanates from meaningful and worthwhile data. As we continue our journey into data-driven decision making, we have not lost sight of the fact that data and their purposes are not just a means to an end. Rather they are both a means and an end because we have moved our educational community forward in ways that strengthen the bonds of our shared vision and the relationships that serve that outcome. Walking into our middle school, one can find space dedicated to a thriving family resource center and an infant care center, both being the result of a data collection process that exposed our community's common ground with regard to shared values concerning children's readiness for school.

Another way to think about what we see in others and our school systems is seeing their respective potentialities rather than their deficiencies. We can engage in thoughtful and meaningful data-driven decision making that brings us back full circle to the business of what it is we do--educating. We can engage in practices and policy setting that affords others the opportunity to change by drawing them out from themselves and into community.

To achieve this we must ensure that our organizations and the people who work in them are places of reflection and have all the skills that this entails. That is why data-driven decision making is so important to the lifeblood of our learning community. It provides the grist upon which policies and practices of seeing potentialities rests and, in turn, nourishes our will.

Lance Alwin is superintendent of Antigo Unified School District, 201 South Dorr St., Antigo, Wis. 54409. E-mail: [e-mail]

A forum for becoming data savvy

Raymond Yeagley

As a superintendent, you may never have heard of the National Forum on Education Statistics. I didn't know much about it either until a few years ago when I was appointed to represent local education agencies from my state on this national body.

I've discovered the forum is a terrific vehicle for preventing me from reinventing many wheels or from having to revamp my district's data-collection processes to meet federal standards. With enactment of No Child Left Behind, the need for local school leaders to understand data collection and use has moved squarely from the "nice to know" into the "absolutely crucial" arena.

The forum, in conjunction with the National Center for Education Statistics, has a wealth of free, high-quality reports that can inform the work we do as school district leaders, especially as we try to meet new state and federal reporting mandates.

Some forum publications that have been particularly useful for my district include "Protecting the Privacy of Student Records" (a guidebook and brochure that clarify many requirements of the Family Education Rights and Privacy Act), "Technology @ Your Fingertips" (a process for getting the best technology solution for your schools) and "Building an Automated Student Record System" (guidelines to assist in selecting or building an electronic database for student records).

Nontechnical Style

The National Forum on Education Statistics is constantly working on new publications, many of them aimed at meeting the needs of school district leaders. For example, we anticipate a February 2003 release of a publication addressing the nontechnical aspects of design, content and policy development for school district websites and Internet use. It also will include a technical chapter on Internet security for network administrators.

Forum publications generally are not technical reports for statisticians. They contain clear, practical information to help school leaders collect and use data to improve the operation of their schools. A case in point is a recent publication, "Safety in Numbers," which provides information on using data to improve school safety. Last year my district struggled with a harassment issue that disrupted the smooth operation of our high school for almost nine months. Had we known of and used the procedures described in this new publication for obtaining and managing information about bullying, I believe we would have handled some aspects of the situation differently and may have prevented it altogether. We already are making changes.

Lack of data knowledge and effective data-management systems in local school districts can be costly in other ways. Some state and federal reports are so complex that it is easy to miss what should be obvious.

Recently I heard about a school district that had under-reported its special education population for several years. Last year, the district reported a disabled population of 6 percent when the actual number was more than 17 percent. It's hard to guess how many federal dollars have been lost for the children in that district as a result of incorrect reporting. A local understanding and use of the data quality procedures found in forum publications could have prevented the loss.

In addition, forum products have addressed other topics of relevance to school district leaders, including documents defining staff and student data elements and indicators, data quality guidelines addressing accuracy of education data, a facilities handbook that helps districts to improve management of their fixed assets, and security guidelines for protecting sensitive information within computer networks.

Easy Access

So how does one access and obtain the forum's products? The easiest way is through its website at www.nces.gov/forum. From this page, you can download a publication, order a free single print copy or purchase multiple copies from the Government Printing Office at a nominal price.

Interested superintendents can keep up on forum activity by periodically visiting the website. You also can receive automatic notices of new publications from the National Forum on Education Statistics and the National Center for Education Statistics by signing up for e-mail notices at nces.ed.gov/newsflash/.

At first, I was skeptical about the usefulness of forum publications at the local level, knowing that it was originally conceived as a federal and state partnership. However, in recent years the forum has expanded both its membership and its focus to address local needs. I invite you to take a look for yourself.

Raymond Yeagley is superintendent of the Rochester School Department, 150 Wakefield Street, Suite 8, Rochester, NH 03867 E-mail: [e-mail]. He will serve as chair of the National Forum on Education Statistics until July 2003 and is the first superintendent to hold the position.

Rio Rancho Public Schools

Eight years ago, the Rio Rancho school district was born, splitting off from the larger Albuquerque district. Local leaders and parents wanted greater control and self-determination.

Three years ago, the district began receiving student achievement data from the state, part of a broad-based educational reform and accountability effort. But the district was at a loss. It had no established system to effectively make sense of the numbers.

"The district had always had components of data-driven decision making, mostly in human resources and finance," says Larry Walling, an instructional data analyst. "But the district as a whole had never really looked at student achievement data or made organizational decisions based on them."

The Rio Rancho board determined that radical changes needed to be made and that someone from the outside should be brought in to implement them. "It wasn't a totally popular decision," says Walling, "because Rio Rancho is a very site-oriented district."

The board hired Manny Rodriguez, a former principal in Texas, as assistant superintendent. (Rodriguez has moved on to become the superintendent in Roswell, N.M.) "I came specifically to help with the accountability program, which was just getting started in New Mexico," says Rodriguez. "My job was to help move the district forward in terms of using data for learning and achievement."

One of Rodriguez' first moves was to hire Walling. The second was to introduce the Baldrige framework, a business-based assessment model created in the wake of a White House conference on competition and productivity and named after the late Secretary of Commerce Malcolm Baldrige. The Baldrige model called for, among other things, weekly classroom evaluations of students so that teachers could pinpoint learning issues and respond immediately. Independent consultants were brought in to train teachers and administrative staff on data collection and analysis and on what constituted best instructional practices.

Implementation was not easy. It involved four basic steps--planning, doing, studying effects and acting--with each step then broken down into segmented components, says Rodriguez. For example, the component on instructional process includes separate sections on how to break down complex data, develop timelines and create student intervention programs, such as enrichment classes or tutorials.

"The process is complex, but once you learn it, it's pretty simple to do," Rodriguez says. Walling acknowledges some initial shock among district personnel, but adds, "Once they got over the newness of it, they saw data could be very useful."

And not just in terms of monitoring student performance. Rodriguez and Walling also collected data about professional development. They surveyed teachers about their needs, then correlated those findings with student achievement, specifically in three target areas: reading, math and special needs.

As a result, new professional development programs were created and tailored to help both teachers and, ultimately, students. "It has worked out well," Walling says.

"It's been a big commitment. There's a huge investment in technology, especially if you're starting from scratch. But the school board has embraced the idea. They've gone through training themselves. And the community seems to support it too, especially business."

Scott La Fee is a writer for The Union Tribune in San Diego. E-mail: [e-mail]

Additional Resources

As the use of data-based decision making grows, so will resources to support it. This list should not be considered comprehensive or complete but rather a starting point.

Reports

"At Your Fingertips: Using Everyday Data to Improve Schools," MPR Associates, www.mprinc.com/pubs/1998.html

"Comprehensive School Reform: Research-Based Strategies to Achieve High Standards," WestEd, www.wested.org/csrd/guidebook/toc.htm

"Data-Based Decision-Making," National Association of Elementary School Principals, www.naesp.org

"Improving School Board Decision-Making: The Data Connection," National School Boards Foundation, 703-838-6722 or [e-mail]

"Making Good Choices," North Central Regional Educational Laboratory, www.ncrel.org/csri/tools/makegood.pdf

"School CSR Self-Assessment Tool," Northwest Regional Educational Laboratory, www.nwrel.org/csrdp/tool2.pdf

Books

Continuous Improvement Tools in Education by Richard Chang and Daniel Dalziel, American Productivity and Quality Center, Houston, Texas 

Designing and Using Databases for School Improvement by Victoria Bernhardt, Eye on Education, Larchmont, N.Y.

Results, The Key to Continuous School Improvement by Michael Schmoker, Association for Supervision and Curriculum Development, Alexandria, Va.

School Portfolio Toolkit by Victoria Bernhardt, Eye on Education, Larchmont, N.Y.

Using Data to Improve Schools: What's Working, AASA, www.aasa.org/cas/UsingDataToImproveSchools.pdf.

Using Data to Make Educational Decisions by Philip Streifer, Scarecrow Education, Lanham, Md.

Websites

AASA's website includes resources for school districts seeking to learn more about data-driven decision making. The links include software tools, federal government information, reports on value-added assessment and sample accountability plans. www.aasa.org/issues_and_insights/technology/

National Center for Educational Accountability, created by the Education Commission of the States, the University of Texas at Austin and Just for the Kids, has developed a site to expand states' knowledge on the use of data to monitor, analyze and improve student and school performance. www.ecs.org/html/project.asp?projectID = 26

National Center for Research on Evaluation, Standards and Student Testing, also known as CRESST, promotes the development and use of evaluation and testing techniques to assist with informed decision making. www.cse.ucla.edu

National Study of School Evaluation has compiled a comprehensive series of publications and services to support data-driven and research-based school improvement planning. www.nsse.org