Big Initiatives in Small Places.

Research-based decision making enables smaller districts to apply world-class reforms

The pre-eminent psychologist Seymour Sarason complains in his recent book, Political Leadership and Educational Failure, that the education system is not geared to spreading a new idea, new methodology or a new practice.

If the system is incapable of spreading a new concept, how can decision-makers in education use valid research when considering new programs if ideas cannot be effectively communicated? My wife, a 6th-grade teacher, often reminds me how teachers feel about "new" ideas being thrust upon them every year with the promise they will lead to educational improvement.

The task of school improvement is not only daunting--it is replete with potential for failure if educators jump on the most recent fad. One need only consider the annual education buzzwords that have evoked community uproar. We went from new math to traditional math and problem-solving, from phonics to whole language and, in some cases, back to phonics, from preparation for life curriculum to standards-based curriculum, from self-esteem to assertive discipline to zero tolerance and on and on!

Everyone has an opinion on what constitutes a high quality school district, but rarely do purveyors of best practices back them up with research data. Serious investigation by staff members on many fronts is needed for reform efforts to move forward. Our district did just that when we used credible data to examine the math and science curriculum and, more recently, reading instruction.

Educators need to look at some business and medical models to construct, at the very least, a viable framework that uses research data to validate curricular programs before implementing them. Aircraft companies employ test pilots before their new planes become operational. Medicines undergo testing before they are sold. Shouldn't we demand more from vendors of textbook and educational materials? Why purchase from companies that produce materials for statewide adoption that bear no relationship to the needs or abilities of our students? Are their materials comparable to those used in high-achieving countries, states or even school districts?

Benchmarking Efforts

So how can superintendents improve curriculum adoption and materials selection processes? How can we transform our school systems so they become data-driven and information-based?

With several colleagues, I am pursuing the national goal of making U.S. students No. 1 in mathematics and science by 2000. We organized the First in the World Consortium, a group of 19 Illinois school districts, then discovered that little research exists to confirm whether we achieve that goal. Our persistence paid off, however. It led us to the Third International Mathematics and Science Study.

After working closely with U.S. Department of Education officials, we convinced the International Association for the Evaluation of Educational Achievement, the proprietors of TIMSS, to allow U.S. school districts to take the test. Our consortium included enough students to comprise a statistically significant sample so we gained data that enable us to compare our mathematics and science achievement and programs, as well as contextual demographic information about student and staff perceptions in grades 4, 8 and 12, to those of the highest-achieving countries in the world.

We now have a picture of how our students compare with the achievement of students in mathematics and science all over the world. TIMSS was the most comprehensive, rigorous, international assessment ever undertaken. It provides both a benchmark of our success or failure on student performance and a view of comparative life in our schools with our international counterparts. Our achievement data reflect overall consortium achievement and are not and will not be disaggregated by district. We did not consider the "horse race" information to be our top priority, but we did need a valid starting point to improve our math and science programs.

Information Sharing 

What is more relevant is that TIMSS enables us to analyze each district's mathematics and science curriculum, compare instructional strategies of teachers in the United States, Germany and Japan and learn what our students actually think about their behaviors in our schools.

We have used TIMSS information to bring teachers together to view videotapes of 8th-grade teachers in three countries. We were fortunate to have funding for a consultant trained in the videotaping project to facilitate our teachers' discussions and reactions to the diverse instructional strategies. One teacher in my district returned from the videotaping session and videotaped herself the very next day.

The consortium also organized teacher learning networks. Teachers have traditionally been constrained by sharing information only with colleagues in their schools or districts. The consortium, through the networks, has provided information from TIMSS about achievement, curriculum and teaching, and the teachers used that information to analyze their-own district's curriculum and to discuss it with other teachers. The networks, in essence, are cross-district collaborative groups engaging in pedagogical discussions about math and science instruction.

For example, consider that the best-achieving students in 8th-grade mathematics come from countries where most students have completed algebra and some geometry by the time they complete 8th grade. If your district cannot compare favorably to that benchmark, then you should consider raising expectations in your mathematics curriculum. Do you want your district to be internationally competitive? A caveat: Be sure you offer high-quality algebra and not a watered-down curriculum for public relations purposes.

Hidden Meanings

Arie van der Ploeg, a researcher from the North Central Regional Educational Laboratory, points out that TIMSS was designed to answer four key questions: What are students expected to learn? Who delivers instruction? How is instruction organized? What have students learned?

Van der Ploeg says that issues of curriculum, instruction and teachers' work are central to those questions. He notes that TIMSS planners realized the importance of distinguishing between what school systems intend to teach, what actually gets taught and what is learned.

NCREL's experience working closely with TIMSS analysts in Illinois, Minnesota and our consortium districts has illuminated several important lessons not so evident in much of the published commentary on TIMSS to date. Those lessons tell us that:

* A great variety exists in what countries and districts intended to teach;

* Less but still considerable variety exists in how countries and districts teach;

* The variability in student learning within countries is similar even if average achievement is quite different;

* What constitutes "best" practice is far from obvious, Excellent and poor results come to light in all countries and in all curricula and across many varieties of practice; and

* Students will not learn if they are not given the opportunity to learn.

Three key factors in improving student learning that TIMSS identified were: teacher engagement; challenging content for all students; and frequent communication among teachers about instructional issues.

If a "magic bullet" exists, van der Ploeg says, it comes from the challenge that new knowledge gives to established practice. To change that practice and accelerate improvement requires sustained effort and support from many individuals and resources.

Learning Communities

Dave Kroeze, superintendent of Northbrook School District No. 27 and chairman of the First in the World research committee, emphasizes the importance of teachers and administrators engaging in data-driven research efforts. He cites examples at the 4th and 12th grades where his teachers are conducting detailed analysis, using TIMSS data, to gain a deeper understanding of their curricular and instructional practices.

At the 4th-grade level, Kroeze says, staff in local districts are examining TIMSS test items in relation to their own schools' content and performance expectations. Are the districts challenging students appropriately? Does the curriculum sufficiently address scientific principles and concepts? The districts are comparing consortium achievement with other high-performance countries to pinpoint similarities and differences in approaches and instructional emphasis.

At the 12th-grade level, physics teachers are identifying TIMSS-released test items by model type and analyzing students' level of performance. These analyses are being done to determine the rigor and format of physics courses. The teachers also are investigating how students perform on experimental design and scientific process items. Finally, they are attempting to trace conceptual models through 4th, 8th and 12th grades.

The creation of teacher learning networks is consistent with a contemporary theme of using learning communities to improve organizations. In his book, Managing People Is Like Herding Cats, Warren Bennis states that the key to gaining a competitive advantage is the ability of leaders to create an adaptive learning environment that encourages the development of intellectual capital. What better model could be used to provide teachers with qualitative teacher information on beliefs about practices, quantitative information and standardized achievement data through informed practice with collaborative and reflective discussions?

Teaching Reading

Like many superintendents I found the debate over how best to teach reading and the diverse opinions of our staff and community perplexing. After three years of learning and working with the TIMSS model and leading the First in the World Consortium, I was convinced that posing a reading problem as a research question would bring us information to improve our practices.

At the same time, I wanted an objective and knowledgeable study and I wanted to assure our teachers I was supportive of their work. The issue was not that our students perform poorly, but rather how to improve our reading and language arts program.

To accomplish my goal, I asked the North Central Regional Educational Laboratory for help. (A regional education laboratory that can provide such services exists for every state.) The lab organized a team of specialists to work with every reading, special education and limited-English proficiency teacher in our district.

Rather than emphasizing random viewpoints only--a risky process when you review reading--we chose to analyze state and national district assessment scores, administer an opinion and practices survey to teachers, observe classroom instruction, interview teachers and analyze the district's reading/language arts curriculum.

These processes consumed the first year of the project and resulted in a document with findings addressing multidimensional factors of our reading program. Every teacher in the district received a copy of the study before leaving school for summer break. The study will appear on the NCREL Pathways Web page soon and should be useful for educators reviewing their reading programs.

During Year 2 we will use the research team to facilitate group discussions through a collaborative process on the study's summary, findings and conclusions. The ultimate goal is to revise the reading/language arts curriculum during the following year using data, staff input and best practices. We hope this process will reduce the potential of intense criticism from special-interest camps wedded to a particular philosophy of reading instruction.

Substantive Dialogue

Some findings are compelling already. Teachers acknowledge that more support is needed at the district level for staff development. Teachers also need more time to discuss reading and language arts instruction. Interestingly, this idea also has been a recommendation from a book group in our district and is cited as a world-class practice that leads to successful teaching in other countries.

Clearly, if we expect teachers to meet high standards, we must recognize the importance of training and provide opportunities for them to work collaboratively.

Working with teachers in small groups in a more informal atmosphere has been beneficial to me. Through book discussion groups, we engage in substantive dialogue and explore meaningful avenues for change to learn more about education practices.

Last year we discussed a book written by Don Chalker and Richard Haynes on world-class schools. Our teachers were excited to spend private time with Haynes, the author of a book they were using to discuss improvement, during one of his visits to the Chicago area.

Learning about world-class education practices in other countries led to creation of a plan for how they can be used to improve our teaching.

The teachers in our book group presented their findings to the board of education. Their work has already led to some new initiatives. One member organized a mentoring program for new teachers. The rigor of summer school classes was increased to improve student achievement and replace the more traditional recreational classes that could be offered by other community organizations. Our efforts were geared to helping students who needed the most academic assistance.

Intellectual Growth

What does all this mean? For one thing, we are working to improve our organizational culture and be cognizant of the need to grow intellectual capital. Teachers understand the importance of supporting their educational interests with research or best-practice information. Last year a parent group dealing with the issues of middle school organization and school size supported its positions with sound research findings when it presented its case to the school board. The culture of using research data has gained momentum in our district.

These projects do not have to be costly. Our participation in the First in the World Consortium has given us the most valid mathematics and science information available. Yes, we have made an investment in the project, but we also have been fortunate enough to have the dedicated commitment of many staff members to see how the TIMSS information can improve our programs and their teaching. Our students stand to benefit from a more enthusiastic, better-trained staff offering a higher-quality curriculum.

Book discussion groups require more of an investment of time than money. Discussing issues like world-class education practices solidifies school improvement efforts. This type of project also improves relationships between teachers and administrators in this nonadversarial venue.

Finally, the integrity of reform initiatives is enhanced. When research and data are used to support proposals, discussion tends to be more focused, "Feelgood" programs are ineffective and shallow. They only provide fuel for the fire ignited by critics. The time has arrived when educators need to identify successful practices that will lead us to high-quality education for the 21st century.

Paul Kimmelmon is superintendent of the West Northfield School District No. 31, 3131 Techny Road, Northbrook, Ill. 60062.

Additional Resources

The TIMSS model provides an excellent basis upon which to improve mathematics and science curricula, If budgets are tight, the good news is that plenty of information is available free of charge.

Just looking up TIMSS on the Internet will lead to more than you could possibly manage. The First in the World Consortium address is www.ncrel.org/fitw/homepage .htm.

The information from the consortium should prove useful because it is the only database in the world with identifiable school information that can be linked to performance.

Another bargain is the TIMSS Toolkit, available from the U.S. Department of Education. For $99 you get considerable information as well as the teaching videotapes comparing instructional strategies from Germany, Japan and the United States. The Toolkit can be efficiently used for staff development in districts on a limited budget.