Accountability at a crossroads: the nation needs school leaders who will make accountability decisions that are grounded in research, not popularity.

Educational leaders are an angry lot in the closing days of 2004. They are angry with federal and state regulators for the use of accountability as a blunt instrument of reform.

They are angry with legislators for limiting funding at the same time that they increased demands for services.

Although they are frequently too diplomatic to say it publicly, they are angry with parents and local board members for simultaneously demanding higher levels of performance from students all the while they demand the perpetuation of good grades for substandard work.

They are angry with students, whose cavalier attitude toward testing--"Does it count?"--can ruin the career of a dedicated administrator. they are angry with at least a few teachers and their union representatives who insist that with seniority comes the prerogative to avoid students who need them most.

And they are angry at themselves for occasionally buying the fantasies that achievement could be purchased in a box and that a brand name would be a substitute for extraordinarily difficult changes. They are angry that change is so clearly necessary but universally resisted. And they are really, really angry that even when they achieve success, measured by improvements in student achievement and equity, they are as likely to receive opprobrium as accolades.

These angry leaders are my friends and colleagues. I wish that I could give them what their private-sector counterparts receive. After all, what do we call people who manage multi-million dollar budgets, hundreds of staff members, thousands of stakeholders, and 24/7 job responsibilities? In the private sector, we call them chief executive officers with stock options and golden parachutes. In education, we call them administrators who are as likely to be evaluated based on the ambiguities of politics as they are on the effectiveness with which they improve student achievement (Reeves, 2004a).

Although I cannot offer the stock options, winter retreat on an island paradise or a golden parachute, I can offer a practical approach to four critical decisions that every school, district and state administrator must face. We cannot avoid a confrontation with these decisions, but can only choose whether our decisions will be grounded in research or will be the subject of one more fact-free debate in educational policy.

Decision #1: Mandates or menus?

There are essentially three types of decisions made in every school. "Level 1" decisions are within the discretion of every teacher. While the state standards govern what is taught, the selection of scenarios, along with decisions on grouping, grading, testing, rewarding, engaging and sanctioning students, are among the many decisions left to teachers.

"Level 2" decisions are collaborative, with teachers providing extensive input while the leader must make a final decision. Examples include the use of "power standards" (Ainsworth, 2003) and the selection of common assessments (Reeves, 2004b).

"Level 3" decisions are top-down--the leader makes the decision without any teacher discretion or collaboration. These decisions typically involve matters of safety, value or principle. After all, we do not vote on fire drills, cafeteria hygiene or fundamental principles. We do not vote on equity. We do not vote on fairness. We do not make a collaborative decision about whether accuracy in grading or compliance with legislative requirements is a good idea. We just do it.

The leadership question is this: At which level are educational decisions typically made? The common presumption is that the vast majority of decisions are top-down, the result of administrative fiat. The common complaint is that, as a result of standards and accountability, teachers are no longer permitted to have discretion or freedom (Ohanian, 1999; Kohn, 1999).

Common complaints are, however, commonly inaccurate. In the past year, I have conducted a simple experiment in which I asked teachers and school leaders across the country to hypothesize how educational leadership decisions are made. They presumed that about 75 percent of decisions were the result of a topdown decision.

Then I give the same group of teachers and leaders the opportunity to list as many decisions as they can within a fixed increment of time. They list every level 1 decision of teacher discretion, then with the same time constraint list all the level 2 and level 3 decisions they can identify. The results are strikingly different from the stereotype.

While the group presumed that 75 percent of decisions are level 3--"my way or the highway"--dictatorial decisions of school leaders, the reality is quite different, with only 20 percent of real decisions at that level.

I have repeated this experiment with many groups, including those consisting only of teachers, only of administrators, and a mixed group. While the percentages vary slightly from one group to another, the quantitative sequence never changes. In every case--bar none--the group identifies the majority or plurality of the decisions in their school as level 1--teacher discretion--not level 3--administrative direction. The least frequent decision is level 3--typically about 20 percent.

In fact, teachers not only tolerate but expect school leaders to make some level 3 decisions about issues of safety, value and principle. What enrages teachers, however, are the cases in which they are led to believe that a decision will be left to their discretion or will be collaborative (levels 1 or 2) but in fact were decided by administrative prerogative.

Therefore, leaders must be exceptionally clear about which decisions are level 1, 2 or 3. "This is a matter of safety, value or principle, and I can't negotiate it," is a fair and reasonable thing to say for the leader making a level 3 decision--provided that this statement is clear, honest and up front.

"This is a matter where you have complete discretion. I trust you," is an equally important statement for every school leader to make where there are level 1 decisions to be made.

"This is a matter where I need your advice and I pledge to listen to every one of you, but I've got to make the final decision" represents a level 2 decision.

If leaders wish to maximize teacher engagement, then they must be explicit about each decision level. They must, in brief, move from mandates to menus. This does not imply a return to the days of unbridled "academic freedom" masquerading as license for unlimited discretion. Each level of decision-making implies different levels of limits.

For example, the choice of how I might teach a state math standard is level 1--within the discretion of the teacher, allowing the teacher to use texts, models or any number of scenarios that will engage students. But the decision of whether to teach that state math standard is level 3--it must be done.

Architects of the boundaries

Effective leaders are not the micromanagers of the moment, but they are the architects of the boundaries (Reeves, 2002a). When decisions are within the boundaries established by the leader, there is an extraordinary amount of discretion by teachers. But if classroom decisions transgress the boundaries established by the leader--the boundaries of fairness, accuracy and effectiveness--then the decision-making structure changes. Therefore the leaders must be clear about boundaries: "Within these boundaries you have clear discretion and choice, but if you cross these boundaries, you no longer have choice."

When leaders are clear about menus--the choices within boundaries--that teachers have, then teachers, leaders and students have a greater opportunity for the collaborative achievement of satisfaction and success. When we embrace menus rather than mandates, a significantly higher percentage of the decisions that we make represent choices rather than submission to authority.

Students and adults alike are motivated by choice and empowerment (Goleman, 1998). Effective leaders gain engagement not through the abandonment of leadership prerogatives, but by a careful balancing of freedom and frameworks. As the architects of boundaries, these leaders provide neither micromanagement nor anarchy, but a sound and consistent decision-making framework for educators and administrators alike.

Decision #2: Compliance or moral imperatives?

Regardless of the outcome of the 2004 presidential election, there will remain a substantial number of educators for whom the words "because the United States Department of Education said so" is an unpersuasive basis for changing professional practices.

Let me be clear: Compliance is important, whether the issue is compliance with court rulings after the Brown decision, compliance with federally mandated protections for disabled students, or compliance with state and federal protections for the rights of employees, students and parents.

But compliance-driven leadership is ultimately limited to the least helpful leadership tool, punishment for bad behavior. I cannot think of any high-performing organization in any field that thrives on merely avoiding punishment. Success, motivation, resilience and personal satisfaction stem not merely from avoiding poor performance, but from doing great things and engaging in a profession in which we can literally change the world.

Although state and federal laws require the use of academic standards, our students, communities and colleagues are better served when leaders make the moral, rather than the legal, case for standards.

Standards are a moral imperative because they stand in stark contrast to the bell curve (Reeves, 2002b). There are essentially only two ways to evaluate student performance. One is the comparison of that performance to other students--the bell curve. Winners and losers are apportioned their rewards and sanctions not based upon the quality of their performance, but merely on where that performance stands compared to one's fellows.

The inadequate writer is judged satisfactory because he "beat" 51 percent of other writers; the totally proficient mathematician is judged unsatisfactory because she failed to beat a similar percentage of competitors. The bell curve thus validates poor performance and invalidates satisfactory performance, elevating ranking over common sense.

On things that schools and society value--such as licenses for teenage drivers, jet pilots and brain surgeons--we never accept the bell curve approach. Rather, we establish a standard of proficiency and the candidates either meet or do not meet the exam. The protestations of my 17-year-old son that he didn't hit as many parked cars as his friend down the road are unpersuasive to the driving examiner. My son must, thank goodness, drive proficiently or take the examination another day.

Every reader of this article has spent some time on the playground as a student or teacher. The language of the playground makes clear that students have a visceral sense of justice, as the words "That's not fair!" ring through the air. They know the rules and can explain them with exceptional clarity. "You can go here, but not there; you can do this, but not that." If the rules of the game are ambiguous or inconsistent, then students will not play the game.

In the academic context, standards are the rules of the game. While some colleagues may object to my analogy between games and academic performance, I would insist that games are precisely the correct context in which we should consider academics.

In games, we take fairness seriously and explode with righteous indignation if two different umpires have four different definitions of a strike, or if European judges allocate points differently than do judges from South America. Fairness requires consistency. No one talks about the academic freedom of the umpire or gymnastics judge. We just talk about fairness, a fundamental human value that is more present on the athletic field than in the halls of academia.

Legitimate criticisms of standards

The preceding paragraphs fail to address the many legitimate concerns of those who criticize standards. They say that there are too many standards; they are right. They say that standards are too ambiguous (or, in other cases, excessively specific); they are right. They say that standards are inappropriate for some grades; they are right. They say that standards focus only on specific skills rather than thinking and reasoning (and in other cases that they fail to address necessary specific skills); they are right.

In brief, almost every criticism of standards can be accepted as legitimate. The only question is this: Shall we use these criticisms as a basis for the rejection of standards or as a platform from which standards can be improved? If we accept the former proposition--the rejection of standards--then we are left with no alternative except a retreat to the bell curve.

Standards are, to be sure, imperfect and in need of significant improvement. The clarion call to educators and leaders is not the rejection of standards, but the improvement of them. This will be the work of our careers and the work of our professional successors.

Decision #3: Data or analysis?

The term "data-driven decision making" is a cliche, a term properly defined by the Concise Oxford Dictionary as "a hackneyed or overused phrase or opinion. A very predictable or unoriginal thing or person." Teachers and school leaders are drowning in data--test scores, percentiles, averages and the like. While we have abundant data, we lack meaningful analysis. Saying that "47 percent of students are writing at or above the state standard" is not as helpful as the following:

* 82 percent of Ms. Smith's students are proficient in the writing trait of organization.

* 23 percent of Ms. Smith's students are proficient in sentence structure.

* 75 percent of Mr. Bruce's sixth grade students are proficient in vocabulary.

* 18 percent of Mr. Bruce's students are proficient in reading comprehension.

* 69 percent of Mr. Kornfeld's fourth grade special education students are proficient in science, using the adaptations of extra time, small groups and dictation.

* 72 percent of Mr. Kornfeld's special education students are proficient in reading using the same adaptations and accommodations.

Building on strengths

The key to effective data analysis is a "treasure hunt" (Reeves, 2002b), a relentless search for best practices. Far too many attempts at data analysis are, from the perspective of teachers, a thinly disguised "gotcha!" in which they are exposed to ridicule and humiliation. As a general leadership principle it is faster and easier to build on strength and then compensate for weakness (Buckingham and Clifton, 2001).

In the "treasure hunt," the leader will cross-examine teachers, asking, "Why are you so good in vocabulary? What did you do that led to your success in measurement? How did you help your students excel in patterns and functions?" The honest answer by many teachers may be "I don't know," and the implicit addendum to that response is, "Nobody ever asked me to think about why I am good at my job--I've been too busy thinking about why I am doing a bad job!"

Searching for strengths

The search for strengths is no psychological exercise in false self-esteem. Rather, it is the key to successful leadership and organizational improvement. Almost every teacher has a strength, or a colleague who has a strength, frequently hidden from traditional data analysis. This might be a strength that is revealed not in a set of average test scores but in a sub-scale of a single test.

The answer to "Why are you so good in number operations?" might be the games that the physical education teacher plays. The answer to "Why are you so good in fractions?" might be the inspiring music teacher who helps students understand the concepts of whole, half and quarter with wonderfully creative exercises on the drums. The answer to "Why are you so good in patterns and functions?" might be the art teacher who uses color, shapes and imagination to illustrate the difference between arithmetic, geometric and exponential progressions.

Drawing meaningful Inferences

Data analysis is more than rendering a report. Anyone can read a test score and report it to the newspaper. Only the most insightful leaders and educators use test data to draw meaningful inferences about strengths and opportunities for continued improvement.

Adoption or implementation?

Education reform is littered with brand names. One imagines a future archeologist who, encountering an ancient school of the 21st century, will carefully brush back the detritus of layer upon layer of reform.

"Egad! I believe I've found the hieroglyphs that appear to say Open Court," exclaims the first archeologist. Digging further, they find other artifacts, revealing Everyday Math, Success for All, Hooked on Phonics, Cultural Literacy, Direct Instruction, Accelerated Math, McGuffey's Reader and a seemingly endless vein of one brand on top of the other, all breathlessly purporting to offer something new and different.

The archeologists of the future will notice what we too frequently fail to observe--that the differences in brand names had little impact on the students or schools. It was rarely the brand name that was the causal variable in school reform, but rather the degree of implementation that mattered.

Any of the brand names--including McGuffey's Reader--could be the basis of a successful reading program, provided it was implemented with the diligence, time, commitment and professional development that accompanies any successful program. Any of those programs would be abject failures when the school leadership did no more than deliver the mail to the classroom door expecting, as if by magic, that brand names yielded miraculous results.

The most important skill for any school leader is the articulation of expectations for adults. We must provide a straight answer to these questions: What does exemplary reading instruction look like? What does effective interdisciplinary writing instruction mean? What should a casual observer expect to see in an effective classroom with regard to content, student engagement and classroom management?

For all the talk in the past few years about instructional leadership, too few leaders can distinguish with any degree of clarity what they expect to see in the classroom. The failure of leadership to articulate expectations explains, in part, the woeful results of recent observations from more than 1,500 classrooms (Learning 24/7, 2004). These dreary conclusions include:

* Clear learning objective: 4 percent of classrooms

* Evidence of higher order thinking: 3 percent of classrooms

* Worksheets: 52 percent of classrooms

* Non-instructional activity: 35 percent of classrooms

* Academic dialogue or discussion: .5 percent of classrooms

* Lecture: 31 percent of classrooms

* Monitoring with no feedback: 22 percent of classrooms

* Students required to speak in complete sentences: 0 classrooms

* Evidence of bell-to-bell instruction: 0 classrooms

* Fewer than one-half of students engaged: 82 percent of classrooms

These observations are not simply a failure of teaching, but a failure of leadership. More than 20 years ago, Peters and Waterman (1982) suggested that "management by wandering around" was an effective technique. Wandering around without a map, however, is more likely to lead to more decades in the desert of despair rather than enlightenment.

The nation needs leaders who will make choices based not on popularity but effectiveness. In sum, we need leaders who will choose menus over mandates, moral imperatives over superficial compliance, genuine analysis over the recitation of meaningless data, and an assessment of the degree of implementation of educational programs over the mere adoption of a brand name.

References

Ainsworth, Larry (2003). Power standards: Identifying the standards that matter the most. Englewood, CO: Advanced Learning Press.

Buckingham, Marcus and Clifton, Donald O. (2001). Now, discover your strengths. New York: The Free Press.

Goleman, Daniel (1998). Working with emotional intelligence. New York: Bantam Books.

Kohn, Alfie (1999). The schools our children deserve: Moving beyond traditional classrooms and 'tougher standards.' Boston: Houghton- Mifflin.

Learning 24/7 (2004). Classroom Observation Study, in press.

Ohanian, Susan (1999). One size fits few: The folly of educational standards. Portsmouth, NH: Heinemann.

Peters, Thomas J. and Waterman, Robert H. (1982). In search of excellence: Lessons from America's best-run companies. New York: Random House.

Reeves, Douglas B. (June 6, 2001). "If you hate standards, learn to love the bell curve." Education Week, Vol. 20, number 39.

Reeves, Douglas B. (2002a). The daily disciplines of leadership: How to improve student achievement, staff motivation, and personal organization. San Francisco: Jossey-Bass.

Reeves, Douglas B. (2002b). The leader's guide to standards: A blueprint for educational equity and excellence. San Francisco: Jossey Bass.

Reeves, Douglas B. (2004a). Assessing educational leaders: Evaluating performance for improved individual and organizational results. Thousand Oaks, CA: Corwin Press.

Reeves, Douglas B. (2004b). Accountability for learning: How teachers and school leaders can take charge. Alexandria, VA: Association for Supervision and Curriculum Development.

Douglas B. Reeves is the chairman and founder of the Center for Performance Assessment. He is the author of 17 books and has twice been named to the Harvard Distinguished Author's series. He can be reached through his Web site, www.MakingStandardsWork.com.