Revisiting superintendent evaluation: do you and your school board members view it as an event or a continuous process?

Since the publication of our Superintendent Evaluation Handbook in 2003, I've had the opportunity to work with many board of education members and superintendents around the country. While there is widespread interest in the process of evaluating the superintendent among board members I encounter, I can't say the same for the superintendents.

Superintendents, for the most part, are pragmatic, believing that if satisfied with their own situation, the development of a more objective, holistic process of evaluation is unnecessary. In a recent study of 100 superintendents in three states, more than 75 percent said they were treated "fairly" in their most recent evaluation. Essentially they believe it is up to each individual superintendent to know the board's expectations and how their evaluations will be conducted. Conventional wisdom states that superintendents should "have the evaluation criteria and process written into their contracts."

But I've also met many superintendents who are not satisfied with the process and are having a hard time getting their school boards to see the problem in order to invest the time and energy to fix it. They receive vague judgments on their performance based on subjective impressions of board members.

These superintendents argue that such an evaluation does not help them grow professionally. During my days as a superintendent, I often would ask: "What does this rating mean?" and "What data were used to arrive at this rating?" Absent specific responses to these questions, it was impossible to plan for growth and improvement.

Too Subjective

Superintendents' performance evaluations continue to be too frequently conducted through a highly informal, subjective process, based more on impressions than data. The implementation of No Child Left Behind and other standards-based reforms created a focus on performance-based assessment for all school professionals, including superintendents.

Several states, like Illinois and Texas, require school boards to use student performance data in evaluating their superintendents. Student achievement data, focusing on continuous improvement, should certainly be considered in the process of superintendent evaluation. But a fair and unbiased evaluation of superintendent performance must be based on multiple sources of data that reflect performance in the many facets of the position. There is general agreement among superintendents and academicians who study the superintendency on the ever-increasing complexity of their jobs.

Creating a process with more objectivity is an attainable and important goal for both superintendents and the boards that employ them. One of the primary responsibilities of the board is to regularly assess the per-formance of the superintendent. Besides fulfilling this legal requirement, the process has the potential to serve a broad range of purposes, such as

* identifying and prioritizing school system goals;

* guiding the professional growth of the superintendent;

* defining the board's expectations of the superintendent;

* clarifying the roles of the board and superintendent;

* enhancing superintendent/board communications;

* enhancing the planning process;

* improving educational performance; and

* reviewing the overall effectiveness of the district.

Superintendents who see the added value that a comprehensive evaluation process can bring to the school district can prioritize its development and work with their board members to develop and implement such a system. Comprehensive models and guidelines can facilitate the development of a sound process.

A process that conforms to the National Personnel Evaluation Standards (a product of the Joint Committee on Standards for Educational Evaluation in 1988) reduces subjectivity and achieves these goals. The essential elements of these standards require a process that: adheres to legal and ethical standards, serves the best interests of the schools and students, is informative and useful, provides constructive feedback, is practical and produces valid, reliable data, free from bias.

An evaluation system that meets these standards is complex and multifaceted, but certainly not impossible to develop or implement. It requires a real commitment of time and energy to overcome the traditional obstacles.

Defining the Job

Determining job expectations is a logical and necessary initial step in designing a superintendent evaluation system. The recognition that a quality evaluation process must specify responsibilities (standards) of the job resulted in several sets of superintendent performance standards. Examples include the AASA professional standards for the superintendency (1993), NSBA roles of the superintendent (2000), and the Interstate School Leaders Licensure Consortium standards (1996).

But job standards only serve as the cornerstone of a performance evaluation system, ensuring the superintendent is evaluated based on what she or he was hired to do. While the national standards provide a framework for defining the role of the superintendent, they tend to reflect generic superintendent duties and responsibilities. If the national standards are used to assess performance, feedback is general, subjective and useless to the superintendent in directing professional growth.

Differences in school district size, complexity and priorities thwart attempts to standardize criteria for superintendent evaluation. Many state affiliates of AASA and NSBA provide evaluation models or templates to assist their members in developing an evaluation instrument based on standards. They all too often are checklists of generic job responsibilities that do not serve the goals of the process well because they ask board members to assess the superintendent's performance based on whether--and how effectively--the superintendent performed a series of tasks in each standard. Absent is real evidence for knowing whether the superintendent is performing these activities and little knowledge about what doing these activities well really looks like.

Agreeing on the standards in working with a board and superintendent to develop an evaluation process is just the first step in developing a quality process. The next step is generating a list of superintendent behaviors the board might observe as a result of the superintendent's work if the superintendent is meeting the standards effectively. These indicators of performance for each standard enable board members to be more objective in assessing performance and providing more specific, useful feedback. State affiliates like the Texas Association of School Boards recently revised their sample superintendent evaluation instrument in an effort to provide board members a list of performance indicators the members might see as a consequence of their superintendent's work.

Critical Elements

Unfortunately, job responsibilities and performance indicators alone are not enough. A decision of how performance will be assessed must be made. Convincing boards to adopt a behaviorally anchored rating scale (performance rubric), which provides descriptions of acceptable/unacceptable behavior for each job responsibility, is key to collecting more reliable data. A definition of all the terms used in the scale gives clarity and helps all board members know what ratings like "competent" or "satisfactory" mean as they make performance judgments.

Finally, the data sources that will be used to document and assess the superintendent's performance should be clear. I often ask board members how they know how whether their superintendent is doing a good job. Where do they get their information about the superintendent's performance in standards that cover, for example, instructional leadership or planning and assessment? How much of the performance do they directly observe? Does their information come from a broad spectrum of their constituents or the vocal few?

Most school board members admit their ability to collect good, reliable data that reflects a broad base is very limited. Yet the more data sources, the greater the chance the complexity of the superintendent's work will be fairly assessed. Because school board members generally are not trained adequately to collect reliable, valid data, the fairness of evaluations becomes a critical issue. Advocates of the 360-degree process have long advocated collecting data from all constituents. For example, parents and community leaders can provide valuable input for the board as they assess how well the superintendent communicates with the community.

Some data sources to consider include formal observation, informal observation, district goal achievement, student achievement gains and client satisfaction (survey data).

Several "natural" times exist for the board and superintendent to examine their current practice and modify it by adopting a process of superintendent evaluation. Of the school boards I have worked with, those who became engaged in the process to reassess and refine their expectations and the superintendent's job description prior to searching for a new superintendent have been most faithful to the process through time. By adopting standards and performance indicators, the board is clearly defining their expectations for the incoming superintendent. Therefore, throughout the subsequent search process, a candidate would have an opportunity to discuss the process up front and be able to have a clear understanding of how performance will be assessed in the future--with no surprises on either side.

A Hurried Task

As school district leaders, superintendents model the commitment to life-long learning. Their commitment to a comprehensive process that not only assesses the achievement of district goals but also provides data and other feedback about their performance demonstrates their desire to use such information for professional growth. Articulating the merits of a comprehensive process and prioritizing its development and implementation with the board is the first step.

Board members may come to their role with good intentions. But the lack of a sound orientation and real training in the most important aspects of their role handicaps them and impedes their ability to be assets to the school district in working with their superintendent and achieving their goals.

Considering the number of boards that lack the will and training to develop and implement a comprehensive evaluation process, superintendents should consider the evaluation process when negotiating initial and subsequent contracts. For example, a provision that the superintendent receives a copy of all forms used during the process by individual board members may provide a more realistic reflection of actual performance.

Because summative evaluation forms reflect a composite of individual board members' judgments, they may be more subjective, reflecting the sentiments only of the individuals who prepare those documents. The contract also should guarantee the right of the superintendent to respond to the evaluation in writing and make the response a permanent attachment to the summative document.

All too often superintendent evaluations are performed hurriedly in an attempt to satisfy a legal requirement or a policy mandate. If the evaluation is merely an event, it has little, if any, impact on the professional growth of the superintendent or improvement of the school district.

The success of the superintendent and ultimately the success of the school system are inextricably tied. If the superintendent is to receive a fair evaluation and if the evaluation is to contribute to her or his professional development, success and overall effectiveness of the district, then adequate time and resources must be devoted to designing, developing and implementing a comprehensive and quality performance evaluation system.

Resources

Michael DiPaola suggests these resources relating to his article:

* "Professional Standards for the Superintendent" AASA, Arlington, Va., 1993.

* The Keywork of School Boards Guidebook by K.W. Gemberling, C.W. Smith and J.S. Villani, National School Boards Association, Alexandria, Va., 1990.

* The Personnel Evaluation Standards: How to Assess Systems for Evaluating Educators by the Joint Committee on Standards for Educational Evaluation, Sage Publishing, Newbury Park, Calif., 1988.

* Superintendent Performance Evaluation: Current Practice and Directions for Improvement by I.C Candoli, K. Cullen and D.L Stufflebeam. Kluwer Academic Publishers, Boston, Mass., 1997.

* The Superintendent Evaluation Handbook by Michael F. DiPaola and James H. Strange, co-published with AASA in 1993 and available at www.rowmaneducation.com. The book contains sample evaluation system components.

Michael DiPaola, a former superintendent, is associate professor of policy, planning and leadership at the College of William & Mary, P.O. Box 8795, Williamsburg, VA 23187. E-mail: [e-mail]