Testing superintendents: Missouri becomes the first to require a written exam for state certification, but some wonder why.

Frank Rowles had spent more than 20 years as a professional educator by the time he became superintendent of the Osceola, Mo., public schools last year.

He had experience as a building administrator, he had the knowledge, he had the degree--a specialist in education. But he hadn't worked as a superintendent in the Show-Me State, and that meant Rowles spent the Saturday morning before Thanksgiving taking a three-hour written exam aimed at proving he knows his stuff.

The School Superintendent Assessment has arrived in Missouri. Whether that's a good thing depends on whom you ask.

Upholding Standards

Missouri is the first and only state to fully embrace the exam, based on the 1996 school leadership standards of the Interstate School Leaders Licensure Consortium and administered by the Educational Testing Service. Proponents say the exam, first administered in October 2000, is a natural outgrowth of the standards, developed under the guidance of the Council of Chief State School Officers and adopted by 35 states.

The standards seek to establish clear expectations of school leaders. School superintendents shouldn't have just a passing knowledge of how teaching and learning works. Understanding pedagogy should form the basis upon which their subsequent experience is built.

"It's a statement of values about where the profession should be," says Joseph Murphy, an Ohio State University professor and the consortium's chairman. "The field should be about teaching and learning."

Just as students in 49 states take standardized assessments that judge whether they know and understand subjects from reading to science, those who wish to lead school districts should prove they know what they're talking about too, the test's proponents say. The School Superintendent Assessment, or SSA, consisting of nine scenario-based questions, attempts to weed out who does not have what it takes to lead a public school system. (See related story, page 56.)

With more than half the states using the ISLLC standards to guide the curricula of educational administration programs at graduate universities, the use of the SSA as a sort of exit exam should cause few ripples, says John Holloway, program administrator of the Teacher Quality Initiative at the Educational Testing Service, based in Princeton, N.J.

In 1998, ETS introduced the School Leaders Licensure Assessment, a six-hour written exam used by 10 states to license building principals. The SLLA, also based on the interstate consortium s standards, is a sort of precursor to the SSA, Holloway says.

"The thinking is, states that have this process would have candidates take the SLLA first," he explains. "Candidates would work toward their superintendent's certificate."

Dubious Requirement

That works in theory, and it's no problem for superintendent candidates coming up through a state's educational system. But that process can throw a wrench into the plans of sitting superintendents who'd like to move from a state that doesn't use the assessment to one that does.

So far, Missouri has no plans to issue waivers for experience to superintendents entering the state from other locales, despite their previous work record. If candidates don't hold valid Missouri superintendent certificates, they must pass the exam before or during their first year on the job.

That's what peeves Stephen Kleinsmith, who passed the SSA the first time it was administered in October 2000. (See related story, page 58.) Now superintendent of the 3,700-student Nixa, Mo., public schools in southwest Missouri, Kleinsmith, 46, came to Missouri from the 17,600-student Millard School District in suburban Omaha. With a doctorate of education in educational administration and leadership from the University of South Dakota and years of experience in Nebraska and Iowa as a teacher, elementary principal, high school principal and district-level administrator, Kleinsmith knew he was qualified for the job. Yet that didn't matter to Missouri s state education agency.

"I could have had multiple degrees, and I'd still have to take the assessment in the first year on the job," Kleinsmith says.

Even the test's supporters agree that requirement is a little dubious.

"The key question is, is this instrument better for those who haven't been a superintendent?" asks Jerry Cooper, a former Missouri superintendent and an assistant dean in the school of education at the University of Missouri-Kansas City. "If someone has been successful in other states, why be assessed?"

Missouri's Adoption

That Missouri is the first state to adopt the SSA as a requirement for superintendent certification doesn't surprise many in the state.

Following passage of the state's Excellence in Education Act, the state has required since 1986 anyone seeking a superintendent certificate to pass some form of leadership assessment, says D. Kent King, the state commissioner of education. Candidates from other states always were included.

For most of the last ] years, the state used a model developed by the National Association of Secondary School Principals. It required a two-day visit to one of the state's assessment centers for a round of tests, professional activities and simulated situations school leaders might face. The process was time-consuming and often didn't have much bearing on a superintendent's job, King contends.

Because of how long it took to administer and score the assessment, a backlog of applicants developed for state certification. By the time the Interstate School Leaders Licensure Consortium came along with its promise of devising a more relevant test, Missouri was ready for a change.

"The feeling was it was definitely time to upgrade to get an assessment that was reflective more of the administrator's need to get involved in instruction," King says.

Missouri joined ISLLC in 1994, adopting the standards a few years later. Statewide, colleges and universities with education schools and programs began changing their curricula to fit the standards. When the state adopted the School Leaders Licensure Assessment for new school principals in 1998, those teaching the future administrators felt their students were prepared to demonstrate knowledge of the standards.

Gauging Skills

Along with North Carolina, Missouri provided funding for the development of the School Superintendent Assessment. In October 2000, Missouri became the first state to administer the exam. State education leaders saw it as a the next logical step, following the successful use of the SLLA, says Jim Machell, an education professor at Central Missouri State University in Warrensburg.

"It's based on the same set of standards commonly embraced and accepted across the country," Machell says. "That's reasonable."

And it's a better test, proponents say. The SSA is designed to test whether its takers have a good grasp of the ISLLC standards. Like its cousins across the standards movement, the test gauges knowledge and ability, not seat time in a classroom, says Richard Andrews, dean of the University of Missouri-Columbia College of Education.

While test critics charge that it might deter potential superintendents from becoming certified as the state teeters on the cusp of an administrator shortage, Holloway doesn't think so. Nationally, there's no dearth of certified administrators. But those holding certificates aren't necessarily applying for jobs. In Missouri, for instance, 8,000 persons hold building-principal licenses. Only 2,200 are employed by the state right now, according to Doug Miller of the Missouri Department of Elementary and Secondary Education's Leadership Academy.

The administrator shortage points to a need to attain high-quality leaders at a time when it might be easy to hire less-than-stellar candidates, Holloway says. States could use the ISLLC standards and the tests based on them to feel confident about the applicants they hire, he adds. "They just can't water down their license expectancy because there's a shortage of candidates. It's got to be the reverse."

No Big Deal

Educators who've been raised in the test-happy educational environment won't bat an eye at taking what amounts to an exit exam.

That certainly was the case for Vici Hughes. The elementary school principal viewed the School Superintendent Assessment as just another step in her professional development. Someday Hughes, who is 46, hopes to become a central-office administrator, possibly in the 12,600-student Blue Springs School District in the suburbs of Kansas City.

Before becoming a building administrator, Hughes had taken the previous multi-day assessment the state required. In contrast, the three-hour SSA was a breeze, she says. She preferred the written exam to its predecessor.

Rowles, superintendent of the 504student Osceola School District in southwestern Missouri, wasn't relishing the thought of taking a test to continue on his career path. But as a University of Missouri-Columbia doctoral student, he found solid preparation in his graduate coursework.

"I would rather not take it, but I know I have to so I'm resigned to that fact," says Rowles, 44.

Will Testing Spread?

Yet questions linger among some in the state as to whether a written test really can determine whether an individual is superintendent material. Missouri's old superintendent assessment was cumbersome and expensive to administer, Cooper readily agrees. But it yielded the kind of rich data a written test never could, he says.

"I think the more data you have, the more likely success you're going to have,: he says. "When you have people in real situations and under the gun, I think it does cause people to act a little differently."

That's Kleinsmith's point, too. He said the questions stemming from the nine vignettes he encountered on the School Superintendent Assessment could easily be answered by anyone who understood how organizations work and how the leader fits in. The questions, he complains, were more academic than practical.

After taking the test in November, Rowles would agree. It was easy to see where the questions were leading test-takers, he says. The answers needed to reflect a caring attitude and the ability to seek out others' opinions, not autocratic responses that didn't involve others in the decision-making process.

"To be honest, I don't know if it really shows if anyone is a good superintendent or not," Rowles says.

Seventy-three percent of the 108 persons who took the first SSA administration in October 2000 passed the test. Of the 67 candidates who took the test in April 2001, 84 percent scored at least a 153, the passing score for Missouri. The maximum score is 200.

The state gave the SSA in November 2001 to Rowles and about 70 other candidates for superintendent licensure. Missouri also will field-test a portfolio assessment for school principals this summer. Other states are hoping to follow Missouri's lead, but educational politics work differently in each state, says Wayne Martin, director of the State Education Assessment Center of the Council of Chief State School Officers.

For example, ISLLC leaders had hoped North Carolina would begin testing its superintendent candidates soon, considering the state provided funding to design the test. But budget constraints are hindering the Tar Heel State's buy-in.

Adoption and implementation can be lengthy, beginning with approval from the state professional standards board or state board of education. Then the state must set its own standards upon which to base the superintendent exam. Finally, the state would contract with the Educational Testing Service.

Several states have completed the standards-setting phase or are in the midst of doing so, Martin says. They include Pennsylvania, Indiana, Nevada, Arkansas and Alaska. Delaware and Rhode Island are interested in the SSA but have not yet begun the test adoption process. Add to that number the states giving the ETS exam for site administrators, which includes Kentucky, the District of Columbia, Maryland, Mississippi, Tennessee and Virginia--and the SSA is off to a rolling start.

"Once you start (standard setting), it's like getting the raft into the whitewater," Murphy says.

This means it's a matter of time until Missouri doesn't stand alone. Soon fledgling superintendents in many places and sitting superintendents moving into another state will be tested to gauge their competence as a rite of passage, some predict.

Says Andrews, the education dean at the University of Missouri-Columbia: "If you're going to require certificates, you should have to take a test."

RELATED ARTICLE: What's Covered on the Superintendent Test?

Wondering what a three-hour written exam that measures the fitness of superintendent candidates consists of?

The School Superintendent Assessment devised by the Educational Testing Service has three parts. Each consists of constructed-response questions that measure the test-taker's understanding of the six Standards for School Leaders.

Part I: Evaluation of Actions (one hour)

Five vignettes present situations a superintendent might encounter. The question with each asks what action a superintendent would take next. The ETS test-preparation booklet provided this sample:

* A district has two elementary schools with a new K-5 school opening in the fall. It will be necessary to determine what students will attend each of the three schools, requiring the formation of new boundary lines.

Question: Identify and describe at least three critical factors the superintendent should include in a recommendation to the school board about the new boundary lines.

Part II: Synthesis of Information and Problem Solving (one hour)

Presented with a short scenario and a set of documents, the candidate must extract relevant information to address questions that pose complex problems relating to the scenario. This is a sample provided by ETS:

* The superintendent of Greenwood Elementary School District is responsible for aligning the district s curriculum with state standards in language arts and math. The state recently adopted new core curriculum standards in both subjects. The new standards--a response to public outrage over declining student achievement in language arts and math--mandate intensified programs in coverage of concepts and skills.

Greenwood is a community with a history of progressive and forward thinking in education generally and in its curriculum particularly. Many stakeholders are resistant to the newly mandated curriculum standards as an infringement on local control and the possibility that prized programs and services may be eliminated.

The provided documents include a letter to the state education commissioner from the school board president; a letter to the school board president from the state commissioner; an article from the local newspaper; a memo to the superintendent from the teachers' union; and a memo to the superintendent for the district director of curriculum and instruction.

Questions: What should be the superintendent's initial steps in enlisting the critical support of the school board in aligning the district's curricula with the state curriculum standards? Why is each step important?

What specific topics relating to implementation should the superintendent place on the agenda for the superintendent's council and review with the school board?

Describe essential elements of a plan for reporting to the school board and the public on progress being made toward implementing the state mandates.

Identify specific stakeholder groups involved in the implementation and the impact of the mandates on each group.

Part III: Analysis of Information and Decision Making (one hour)

Three document-based exercises consist of scenarios faced by superintendents and supporting documents. The candidate responds to two questions, using the information in the document. This is a sample provided by ETS:

* The school board in the Birch Grove Public School District has issued a new policy titled "Concepts and Roles in Community Relations: Goals and Objectives." The policy describes the board's nine expectations for its school-community relations program.

Questions: From the perspective of the superintendent, describe at least three essential elements of effective communication.

Identify and explain at least three steps the superintendent could take to develop programs, techniques and channels for implementing this policy.

A free booklet, "Tests at a Glance" for the School Leadership Series, is available from the Educational Testing Service to any educator who would like more information about the School Superintendent Assessment. The booklet provides test-taking strategies and sample test questions, along with answers and explanations.

Why I Object to Being Tested

STEPHEN KLEINSMITH

"Bring ideas in and entertain them royally, for one of them may be king."

Mark Van Doren 

This whole idea of requiring superintendents to take a competency test is at best a prince of an idea. At worst, it's an unnecessary and unwelcome burden for experienced school leaders who've already proven their fitness to serve in this important public office.

Having spent 22 years as a successful teacher, coach, elementary school principal, secondary school principal and central-office administrator, having earned my master's, specialist and doctorate degrees from accredited universities and holding superintendent certificates from both Iowa and Nebraska, I question the necessity of requiring superintendent candidates in Missouri to pass the School Superintendent Assessment.

I already had accepted a superintendency offer from the school board in Nixa, Mo., in the state's fastest-growing county, when I learned I would be forced to take a written exam to keep my job after one year. As of Jan. 1,2000, gone was the requirement of a comprehensive, two-day assessment of skills and knowledge, replaced by a single three-hour, timed test. Some would say that's not a bad trade-off and, for the most part, it wasn't.

What concerned me first was the uncertainty of the contents of this test--which had never been administered previously by the Educational Testing Service--and the lack of readily available test preparation material. Up to that point, I found the state s certification process both easy to understand and full of fair expectations. And I can understand the state's interest in promoting the admirable standards for school leadership developed a few years ago by the Interstate School Leaders Licensure Consortium.

What I found particularly frustrating was being caught between the old means of assessing administrator candidates and the new system. Missouri s previous administrator assessment was abandoned as of January 2000 as too time-consuming and costly and its replacement wasn't developed yet.

So when I learned that my requisite credential for working as a superintendent would depend on my passing an as-yet unreleased written exam about which little was known, I worried myself with doubts: Who is developing this assessment? Who will grade it? How will it be graded? What if I don't pass this stealth test?

Mounting Self-Doubts

As the move from my comfortable surroundings in Nebraska drew near, I second-guessed having asked my wife to quit her job as a school counselor in hopes of continuing her career in Missouri. Will my children be okay with uprooting them from their happy surroundings? What if I don't take enough time from my family or my new job to prepare for this test? The list of worries grew as I waited to find out when the test was to be offered.

Later that spring, I found out the date for the first administration of the School Superintendent Assessment would be Oct. 22, smack in the middle of our annual statewide school board/superintendents conference. The next test wasn't scheduled until the following spring.

I now see many of these concerns may have been unwarranted, but they were real concerns because they were mine, and they seemed so unnecessary at that point in my career.

I'm not sure that finding out that the director of development for this assessment was from Tennessee and the test results were to be scored in New Jersey was of any comfort. I guess Missouri wanted to involve the best and. the brightest minds nationwide to establish the highest level of test validity, reliability and quality.

I think they did that in the makeup of the School Superintendent Assessment, but that's not the point. What this test requirement is saying is this: Your advanced degrees from accredited American universities aren't fully appropriate arid sufficient to the task. Your administrative experience over many years is irrelevant. Your superintendent certificates earned in neighboring states don't count here. We don't t trust elected school boards to select a superintendent with the qualities needed to lead the local schools. (The Nixa board that hired me had just earned the "Outstanding School Board of the Year" award from the Missouri School Boards Association, an award given to only one school board in the state each year.)

Negative Ramifications 

Some of my fellow test-takers and other educators see this testing requirement as nothing more than an effort to appease the public. Is it an attempt to build public trust by supposedly holding public school officials more accountable? I wonder if anyone has considered the possible consequences of subjecting experienced school leaders to a test?

When will we know whether this requirement is helping or hurting those communities as they fill their administrative needs? Those who fail--and I know a few who did--are not allowed to serve as school administrators as they have been identified as ill-prepared or unable (until they finally pass this test). Will answers to a few essay questions tell state officials that the administrators being hired have mastered the important educational standards, knowledge and processes?

Will states like Missouri, already experiencing declining interest in the superintendency as a career pursuit, suffer further from this testing requirement? I worry about otherwise capable candidates who happen to be poor test-takers of a timed instrument on which one is required to write responses by hand? Might quality administrators from other states stay clear of states that add unnecessary state control in matters that should be considered a local decision? (I have some former colleagues who fall into this category.) Will this be viewed as another hoop to jump through? Who has time for that?

P.S. For what it's worth, I passed the test with ease the first time I took it.