Performance-based evaluation for superintendents: combining formative and summative approaches to address procedures, policies and products.

When I was hired as superintendent in East Longmeadow, Mass., in 1998, I followed a revolving door of three superintendents who had been dismissed after their first contract expired. The community was divided on many issues, especially regarding the superintendency and the direction of the school system.

Hired with a unanimous decision by the school board, my marching orders were to bring the community together, resolve differences and move the school system back on track.

East Longmeadow is a community of 18,000 residents in southwestern Massachusetts with a student population of 2,900. We are a suburban district, 95 miles west of Boston, that still maintains New England values and traditions. Family comes first, meaning education is held as a high priority. The community demands excellence and nothing less.

First Evaluation

My entire first year was spent conducting focus groups in the school district and in our community to flesh out a rationale for a school and community strategic plan. Years 2-4 brought the strategic plan to fruition and I am happy to say our community and school system now are on the same page and working together harmoniously.

During that first year, I also was responsible for working with the school board in developing a performance measurement instrument that would be used to evaluate me. The original instrument contained six performance areas and 47 criteria on which to be evaluated. This evaluation was summative in nature but lacked the capacity to recognize gradations in performance. Additionally, it did not contain any formative evaluation components for goal setting and improvement measures.

During the 2002-2003 year, the school committee appointed a subcommittee of the chair and one member to work with me in collecting current research on superintendent evaluations and pay-for-performance language. The committee chair and I attended the American Association of School Administrators and National School Boards Association conferences, where we found three sessions on superintendent evaluation and collected research and books on the subject through the conference bookstores. We assimilated all the relevant research into what we thought was the best of the best.

During the year, the Education Commission of the States issued a policy brief on No Child Left Behind that spoke clearly to school leadership accountability: "The newly authorized ESEA Act reflects and reinforces a major shift in thinking about the roles and responsibilities of school board members, district superintendents and principals.... (S)chool and district leaders are being held responsible for bringing about change and improvement." About the same time, authors Michael DiPaola and James Stronge published their book Superintendent Evaluation Handbook, whose many references added greatly to our research.

Finally, in the fall of 2003, the school board subcommittee and I presented our research findings and draft instruments to the full school board for review. After two months of in-depth review, discussion and minor edits to the draft document, the school board unanimously approved the research and the newly formed evaluation instrument, job description and school board policy on superintendent evaluation.

Lessons Learned

As we collected research, we found many problems inherent with current superintendent evaluations. Most importantly, we found that barely 25 percent of school boards conduct any form of superintendent evaluation. Of those that do, 75 percent do not evaluate the superintendent on an annual basis. Most evaluation instruments do not combine formative and summative evaluation components. Finally, most superintendent evaluation instruments lack "school and district improvement" as their main focus.

School board evaluation policies and processes were also troublesome, as indicated by our research. Most superintendent evaluations were narrative paragraphs written by various school board members that did not compare performance to any specific standards of performance. Many school districts lacked a policy on superintendent evaluation and superintendent job descriptions. The few we found did not align the job description with the evaluation instrument being used. School boards also admitted that their superintendent employment contracts did not specify the evaluation process to be used.

Finally, the research indicated inconsistencies with school boards in the evaluation process. Some boards rely on only the chair to evaluate the superintendent. Other boards ask each member to write an individual narrative evaluation without any summary consensus of opinions. The major problem with these approaches is that the school board needs to speak in one unified voice on the superintendent's performance evaluation.

Our Outcomes

Research suggests the use of both formative and summative evaluations. We decided the superintendent's evaluation would be a two-part evaluation. Part 1, the formative evaluation, consists of the annual goals, growth and management by objectives. The superintendent uses SMART goals (Specific, Measurable, Agreed upon, Results-oriented and Timely).

Part 2 of the evaluation is the summative part, in which the superintendent is evaluated against a set of identified criteria. Summative evaluation asks: "Is the day-to-day job of the superintendent being done?"

The new superintendent evaluation instrument is comprised of six performance area themes that AASA explored in its publication "Roles and Relationships School Boards and Superintendents" and which Michael DiPaola and James Stronge, in Superintendent Evaluation Handbook, now call "domains." Within these domains, our evaluation instrument specifies 30 performance criteria (standards) that identify specific areas of task behavior. We adopted performance indicators for each performance objective to show examples of observable behaviors to school board members. We listed three indicators for each of the 30 performance criterion. The indicators are examples of observable quality superintendent behaviors.

The school board evaluates the superintendent annually using the 30 stated performance criteria.

The Three P's

Our research can be divided into three easy categories that we call the three P's of superintendent evaluation: procedures, policies and products.

Procedures include evaluation timelines, dates and the annual evaluation cycle. These processes specify when the superintendent evaluation is to begin, its duration and when it will end. Procedures also specify what events are to take place, such as the pre-evaluation conference, the goals meetings with the school board, the tabulation process used by the school board, the final summary meeting with the school board and so forth.

Policies are the school board's written rules on superintendent evaluation, the school board's role, the procedures to be used and the evaluation instrument used to evaluate the superintendent. All of the above are crucial and must be stated clearly and specifically in school board policy. Individual policies that contain language pertaining to these items include superintendent evaluation, superintendent job description, employment contracts, professional ethics, district leadership and strategic planning.

Products are the tangible documents created by the school board and superintendent that are referenced in policies and in the superintendent evaluation procedure. These artifacts include the job description, the superintendent contract, the evaluation instrument, the self assessment and the school board policy itself. Additionally, the formative performance goals, created each year, become a product to be used annually as a document in the evaluation process.

Our school board has defined superintendent evaluation by encompassing the two issues of accountability and pay. The bridge between accountability and pay is performance. Measures of performance determine how much or how little a superintendent is paid from year to year.

Rating Process

Our school committee researched many of the currently prescribed performance appraisals. We selected the 9-point (Likert-type) rating scale because of the scale's ability to offer multiple gradations of performance. It also made the school board speak in one voice by reducing any effect of a sole rating that was not consistent with the majority of votes cast.

A 9-point scale greatly reduces the "stray bullet" effect on an overall rating of the superintendent. The school committee believes that if most members rate a performance standard one way, and a single member rates it a completely "out of sync" in another way, then the stray rating should be minimized. Conversely, if most of the board rates a superintendent in a similar way, good or bad, the rating stands as a consensus that speaks loudly and clearly.

The 9-point rating scale is itemized as follows: 9 = distinguished, 8 = exceptional, 7 = outstanding, 6 = very good, 5 = good, 4 = average, 3 = below average, 2 = poor, and 1 = unsatisfactory. One should not try to debate the subtle differences of the descriptors. Instead, it is important to recognize there are nine degrees of superintendent performance as documented and rated by the school board.

The notion of salary based upon performance is not new. However, a concrete and documented process is.

Our evaluation system uses a performance rating based on points. The points are grouped into four different pay-for-performance quadrants. The four quadrants represent a range in pay increases from zero percent to four percent based on the superintendent evaluation. The four quadrant titles and corresponding points are: Exemplary (203-270 points), Proficient (135-202 points), Progressing (68-134 points) and Does Not Meet Standards (30-67 points). Each of the four pay-for-perform-ance quadrants have "earned" salary percentages attached.

Our school board chose to use a 4 percent increase for the highest quadrant with zero percent for the lowest quadrant.

This year I received an exemplary rating so I earned a 4 percent increase in salary for the next year. The school board and I are convinced this process works. I now have fixed my sights on next year's process for continued districtwide improvement.

Six Performance Domains

These are the major aspects of the job of school superintendent, as described by Michael DiPaola and James Stronge in book for AASA titled Superintendent Evaluation Handbook.

* Policy and Governance: Job standards relating to policy, governance and political dimensions of the job.

* Planning and Assessment: Program evaluation, personnel evaluation, improvement plans and student learning.

* Instructional Leadership. Vision, motivation, support of mission, district goals, strategic planning, leadership of the superintendent, professional development of staff.

* Organizational Management. The daily operations and functions of the organization.

* Communications and Community. Effective communication internally and externally.

* Professionalism. Professional ethics and growth of the superintendent.

East Longmeadow's Evaluation Cycle

July: Superintendent develops new strategic formative goals.

August: Superintendent's formative goals are due to the school board.

December: Superintendent's midyear progress report to the board on formative goals.

February: Board reviews evaluation instrument, cycle and timelines. Superintendent completes self-evaluation and formative goals in a final summary for the board.

March: Board completes summative evaluation instrument. Board chair compiles all scores to derive performance raring.

April: Board chair meets with superintendent in post-evaluation conference. Board reports compiled performance rating of superintendent in public meeting.

May: Board grants performance pay for following year and possible contract extension.

June: Board and superintendent review school improvement plan evaluations.

Resources

Edward Costa recommends these resources on superintendent evaluation.

Books

Developing the Effective Principal: Hiring, Evaluation. and Retention Practices for the Superintendent by Jeff Stratton. Cheryl Cox and Jamie Whaley, Aspen Publishers. Gaithersburg, Md.

Superintendent Evaluation Handbook by Michael F. DiPaola and James H. Stronge, ScarecrowEducation, Lanham, Md.

The #1 Guide to Performance Appraisals and Effective Phrases for Performance Appraisals by James E. Neal Jr., Neal Publications. Perryburg, Ohio

The Performance Appraisal Question and Answer Book by Richard C. Grote. American Management Association, New York, N.Y.

Reports

"The Roles and Relationships of School Boards and Superintendents," AASA, Arlington, Va.

"Rewarding Excellence" by Cheryl Sullivan, NSBA, Alexandria, Va.

"Handbook on Teacher Portfolios for Evaluation and Professional Development" by Pamela Tucker, Eye on Education, Larchmont, N.Y.

Electronic

East Longmeadow Public Schools has published a PowerPoint presentation, "Making the Grade: Performance-Based Superintendent Evaluations," www.eastlongmeadow.org/ schools/supteval.pdf.

Edward Costa is superintendent of the East Longmeadow Public Schools, 180 Maple St., East Longmeadow, MA 01028. E-mail: ecosta@ eastlongmeadow.org