Nitty-gritty data: with reams of numbers and test results at their fingertips, more district leaders are making decisions and positive changes across classrooms.

To teach one way or various ways?

That is the question swirling in a junior high school in New Jersey.

Now that data can be broken down and manipulated into a report within minutes, should teachers that use mostly project-based instruction and get good results stick with that, and should the teachers mostly using traditional instructional strategies stick with that?

"We can dissect student assignments and performance on specific tasks and that allowed us to see what types of instruction are taking place in the individual classroom," says Vice Principal Kathleen DeCristofaro of Burlington City Junior School. "The data could reveal that a teacher is using mostly project-based instruction and we can see student performance there and compare it with students in a class with mainly traditional instructional strategies. I'm not saying any one way is worse or better. Some people have a real talent for a particular strategy. But we also know that effective instruction is differentiated instruction, using multiple experiences.... But is it necessary to take a person who is extremely talented in one approach and make them more diversified and meet all the kids' needs?"

Or, she wonders, should the district place certain students who particularly respond to one type of teaching and keep them with that teacher?

These are a few questions City of Burlington Public School District officials are asking and just an example of some new territories districts are exploring with data.

Many school districts have started using data to drive decisions to improve student learning and achievement. Some aren't doing much with data, but a few school administrators are asking the right questions, making changes, according to Douglas Reeves, CEO and founder of the Center for Performance Assessment.

"It's no question that data-driven decision making is getting more and more important," Reeves says. "There are more emerging models that are strikingly more sophisticated."

Most data-driven decision making a few years ago was more about looking at end-of-year test results with little or no analysis to tie-in causes.

"It was an autopsy. I've never seen a patient get better because of an autopsy," he adds. "What you see now and it's not just in isolated areas but across the country, not only do you have annual analyses of test scores, but they are tracking various teaching strategies in the classroom."

And teachers aren't as afraid as they once were about exposing their students' scores and work, even when it shows students are failing. The key is to monitor data regularly, adjust instruction accordingly, and help administrators and teachers become experts in pinpointing what strategies work, says John Tarnuzzer, program consultant for Learning Through Technology Associates. "Do you have an organization that can change things fast enough? If the school is not a good learning organization where everyone is learning, it won't be able to use that data."

Karen Gould, assistant superintendent of Metropolitan School District of Wayne Township in Indianapolis, Ind., which created a database to drive instruction, says much time and energy went into it, but the rewards are clear. "We look at where we are now compared to four or five years ago and we are increasingly becoming more diverse in social economics, we have greater poverty and greater diversity and the trends are rapidly moving. And when you look at the data for student achievement and that's also going up, we know we're on the right track."

Right Data in the Right System

A data system at the Metropolitan School District of Wayne Township discovered that classroom instruction was not always aligned to state standards nor the state assessment test. Seven years ago, the Wayne schools created a comprehensive accountability plan, which earned a Magna Award from the National School Board Journal. Reeves and the Center for Performance Assessment helped lay out a plan, starting first with understanding the standards and what students needed to know. Then they created school-based assessments that aligned to the state test. Data from assessments assist teachers to select instructional strategies to enable each student to succeed on learning targets.

And each of the district's 15 schools has its own school improvement plan, such as improving literacy or raising ESL student achievement, based on district goals. The data also include soft data, like math or science competition participation among students.

In California, a desegregation lawsuit in the 1980s by Hispanic students, looking for more equitable education due to a racial imbalance in the north end of the San Jose Unified School District, led administrators there to start thinking data, says Marcy Lauck, supervisor of the Continuous Improvement Programs at San Jose.

To avoid mandatory busing, the district granted parents a choice to select any school for their children. With school choice and a need to monitor the desegregation order came the added focus on more detailed demographics and academic performance. Schools and teachers wanted better access and autonomy to data so they could study the impact of their programs regarding students' diverse needs.

Thus, the district needed a data warehouse. A two-year pilot began in 1998 revealing that every participating school found that easy access to data was critical, but they needed more training on technology tools and analysis skills to make better use of data.

So the district started using TetraData Corp.'s data warehouse, the EASE-E data analysis tool and Data On Demand, which refreshed data every night. Since last August, hits on the site have averaged about 200,000 per month, Lauck says.

In 2003-04, the district added Edusoft to provide benchmark testing capabilities and Renaissance Learning's Renaissance Place, which both allow for formative assessments to augment high stakes testing. "Sophisticated 'root cause' and cohort analyses that had previously been the domain of statisticians and researchers now became possible for schools to explore on their own with the use of the EASE-E analysis tools," Lauck says.

Data not only include attendance and student mastery of standards, but also the number of graduation-required community service hours high school students have finished as well as the number of books read and physical education results, she says. It houses 20 million records, covering nine years of data.

In Michigan's Kent Intermediate School District, which serves 20 school districts, 11 of which use a data warehouse with help from CELT Corp., getting it right the first time was huge. "One of the biggest things, thus far, that we've found is that it's really pushed local districts to do a better job of data collection," says Mark Maynard, school improvement technical assistance consultant for Kent ISD.

Maynard explains that in one district, a technology staff person had maintained the student management system as well as the lunch and special education data. The tech person didn't realize that for the Michigan Educational Assessment Program standardized test, he needed to show that certain children were getting free--and reduced-price lunch, and that made a key difference in whether the school made adequate yearly progress. "In the last six months, they really began to look at what a local district needs," Maynard says.

Wanting More

Hard data is driving bold changes in San Jose. EASE-E allows for 300 standard queries that administrators and school data teams can click on such as, what were the Algebra I grades compared to the Algebra I test? "Some standard queries pull up target students who, with targeted support, are good candidates to become part of the AYP proficient band," Lauck says.

And for the first time, administrators can look at classroom grades versus the California Standards Tests. Staff conversations sound like, "What does my classroom look like? Can I look at your class and you look at mine and get meaning from it?" says Emalie McGinnis, Title I resource teacher at Horace Mann Elementary School. "We're looking at trends data for the entire school, for all grade levels, none of that would have been possible if not for the incredible data warehouse."

In Kent ISD, one district's staff members are looking at defining assessment to decide if a child is ready to move from one grade to the next, Maynard says. There is a snapshot of where the child stands as far as MEAP results, that measure academic progress once a year, and then other more timely assessments, he says.

"The goal is that every teacher would have access to this and use data to inform instruction on a day-to-day basis," Maynard says. "It would be changing the culture. What we're really trying to focus on is creating reports instead of [the teachers] producing them. It's about pushing the right information in front of them so they can sit on it and look at it and drive instructional decisions. Eventually they will get to that point and pull their own information from that."

But for now, the school improvement team comprised of teacher leaders disseminates information to the teachers. "Teachers are definitely more interested and willing to look at the data, but the key is presenting it in a form that is easy to read," Maynard says.

Boston Public Schools' Special Education Information Management System, or SIMS. metamorphosed into MY BPS, a homegrown data program. MY BPS includes incident reports, high stakes assessment test results, report cards, medical records and transportation information. The system, which is working on compiling the data all on one computer screen, is created by a governance group comprised of people from technology, curriculum and the research and testing side. "What you try to do in the system is set up a data inquiry culture so that people are continually questioning the data," says Maryellen Donahue, director of research assessment and evaluation.

MY BPS Assessment allows for a series of select questions on student performance, such as, how did my students perform on the MCAS? How did they perform on the multiple-choice questions in English? And answers appear on a graph. It also allows fifth grade teachers to reflect back on her students' fourth-grade MCAS results, she adds. "The real change is that people want more," Donahue says.

Beyond Intuition

In the City of Burlington Public School District, the field was not only leveled. "I fell in love," says Technology Coordinator Joanne Tice. "At the end of a marking period or the year, I can select a school and get the reports any way I want. I can get the final grades with one push of a button and I can download it to an Excel spreadsheet and develop any sort of graph."

Burlington schools use Century Consultants' Star_Base School Suite, a Web-based student information management system, and Star_Insight, a data mining tool which disaggregates information. "It helps us make better decisions and create better goals," Tice says.

Teachers, school secretaries, guidance counselors and nurses insert data in the appropriate areas, which Tice then provides the data to principals or the superintendent or both. Principals meet with teachers in monthly faculty meetings or at the end of the report card period. The program allows administrators to whittle down from test scores or attendance, to see by school, by class or by teacher and grade, and then even go further by race or by gender. "This is a personal crusade to really use this data to see how the achievement gap can be addressed," says Kathleen DeCristofaro, vice principal of Burlington City Junior School. "Our white students do better with traditional practices and our black students do better with differentiated practices."

So when DeCristofaro wanted to know what impact discipline has on academics, she had Tice send a report with the number of 'incidents' according to specific teachers, and correlated that to academic data. Maybe the student needs to be in another class with different students or teacher, and maybe the student needs a behavior modification plan that needs monitoring.

"Before this, a lot of the decisions were intuition," DeCristofaro recalls. "It would be very subjective observations by administrators and teachers. Not that that isn't important, but this is a little more quantitative and it's refreshing that it backs up the intuition."

At Horace Mann in San Jose, a Program Improvement School with 70 percent of its population as English learners, Hispanic and low socioeconomic status, McGinnis and the principal re-organized the school's day based on never-before data analyses.

In 2003-04, the problem with the ELL and Hispanic students was evidently the language. "We thought we were teaching the kids the rules of writing but when we looked at the data we saw the instructional strategies were not being used by the students," McGinnis says. "We stepped back and looked at the standards. 'What is it that they are not getting?'" The district considered end marking, such as grammar, and how to get students to use it for overall thinking, McGinnis says. "We were doing this for a very general subgroup, and now we're doing it at the individual student level," McGinnis says.

By the end of the last benchmark exam in English/Language Arts, every first grader was proficient. Horace Mann achieved a 63-point gain on the state's Academic Performance Index, ranking in the top 10 schools in Santa Clara County for greatest gains on API. And last year, San Jose's API grew by 23 points, narrowly missing AYP targets for English learners and students with disabilities.

Even in a mostly homogeneous population like in Mountain Ridge Junior High School in Highland, Utah, knowing what kind of writing problems were tripping up students helped teachers adjust their teaching. "We have very little ethnic population compared to the rest of the nation, however, we have huge class sizes and quite a large special education population. Our state core curriculum is huge and very general and so it felt like we were running, running, running non-stop trying to cover everything and in the same breath teach to the test," says teacher Kate Ross. Before four schools in the district became a pilot using MY Access, a Web-delivered instructional writing program from Vantage Learning, Ross says she couldn't give each student's three- to five-page paper justice. She had no time. MY Access, however, gives students automatic feedback--while they type-on grammar and sentence composition and subject/verb agreements. The program also allows for administrators to see the first and last revisions of all work and plot it on a graph by school, by classroom and by teacher. "All those that need the most help can be helped," says Harry Barfoot, vice president of sales and marketing at Vantage Learning.

"In the mean time, I'm in the classroom and helping students on an individual basis," Ross says. And students are doing triple the writing that they were a few years ago. "After all these years, the students really get what it means to revise."

Middle-Child Syndrome Cured

In Wayne Township, administrators also learned less can be more. They had a slew of state standards and they had textbooks full of war battles and algebraic equations, but they didn't have time to teach it all. "What are the things that are absolutely necessary for students to be successful at the next level?" Gould asks.

Assistant Superintendent Jim Mervilde says the data, which are accessible on one screen, learned many kids were not proficient in reading and math in tenth grade, so they redesigned the middle school level. Every seventh and eighth grader has double blocks of language arts. Every seventh grader also has a double block of math. If students are not proficient in math by the end of seventh grade, in eighth grade they have a double block of math and one less elective class.

San Jose's Pioneer High School data team also investigated the academic history of students getting Ds and Fs in Algebra I. "Out of that came some significant recommendations to revamp the middle school math curriculum," Lauck says. Some of the middle schools held "Data Days" last fall where staff members used student achievement data to analyze their overall academic program and identify target students. Teachers will also gauge their target students' progress on district benchmarks to determine areas of reinstruction.

McGinnis, as well as other educators agree, if teachers set the bar high for learning, say for a student to jump two grade levels in reading, students will. "I think if you set the bar high," she says, "the students will work as hard as you demand."

Tips on Making Good Decisions With Data

Karen Gould of Wayne Township suggests that administrators go back to the beginning. "You have to go back to the standards and understanding them clearly, what kids are expected to know and be able to do and how to move forward and how you know you have the right data."

Administrators need to be informed of programs used in classes and to compare the performance of students in one class compared to another. "You have to have administrator buy-in," says Kate Ross, teacher et Mountain Ridge Junior High School in Highland, Utah.

Use accurate data. For example, the correct zip code must be accurate for the district. "And you really do need good tools to disaggregate data," says Jaanne Tice, city of Burlington Public School District technology coordinator.

Collaborate with vendors and with experts, such as technology people. "And don't underestimate the planning stage," says Kim Rice, chief information officer at Boston Public Schools.

RESOURCES:

CELT Corp.

www.celtcorp.com

Century Consultants, Star_Base School Suite, Star_Insight

www.enturyltd.com

ClassLink, LaunchPad, Information Console

www.ClassLink2000.com

Excelsior Software, Pinnacle

www.excelsiorsoftware.com

FileMaker, FileMaker Pro 8

www.filemaker.com

Northwest Evaluation Association

www.nwea.org

Pearson Digital Learning, SuccessMaker Enterprise

www.pearsondigital.com

STI, Special Education Tracking System

www.sti-k12.com

Sagebrush Corp., Viewpoint

www.sagebrushcorp.com

SchoolNet, IMS 

www.schoolnet.com

TetraData, EASE-E and Data On Demand

www.tetradata.com

Vantage Learning, MY Access

www.vantage.com

Angela Pascopella is features editor.