Courtroom alchemy: adequacy advocates turn guesstimates into gold.

Beginning in the late 1960s, and accelerating unabated through to the present, plaintiffs have filed more than 125 court cases questioning the constitutionality of school district and school spending levels. In 2005 alone, high-court decisions were handed down in eight states, including Kansas and Texas, with a decision rendered in South Carolina that has national implications.

Cases in seven more states, including Kentucky, Louisiana, Missouri, and Nebraska, are now pending decisions concerning issues of adequacy in state funding mechanisms. And legal challenges to state funding mechanisms are not one-off endeavors. Arizona, California, Connecticut, Kansas, New Hampshire, New Jersey, New York, Ohio, Pennsylvania, Texas, and Wyoming are states in which there have been not one, not two, but as many as five or six legal challenges to legislatively determined spending levels.

Much of the litigation, particularly early on, centered on the issue of funding equity. As of 2005, funding mechanisms in 36 states had been challenged on the grounds that interdistrict spending was inequitable. Increasingly, however, cases have focused instead on the overall amount, or adequacy, of funding. Beginning in the 1990s, enactment in virtually every state of learning objectives and curriculum standards provided a new reference point for plaintiffs arguing that funding was inadequate overall. By 2006, the constitutionality of funding mechanisms in 39 states had been challenged on adequacy grounds (see "Judging Money", research, p. 68). Indeed, through the first half of 2006, funding mechanisms in only five states--Delaware, Hawaii, Mississippi, Nevada, and Utah--have been spared constitutional challenge.

Few would seek to deny American public school students access to the courts when inadequate school funding threatens their chances for achieving academic, and ultimately economic, success. But contemporary school-finance adequacy litigation goes far beyond seeking equity for the educationally disadvantaged. The movement is becoming a self-serving cause whereby plaintiffs have gained relatively uncontested judicial access to the policy process. Indeed, unsubstantiated claims and unreasonable requests contained in costing-out studies commissioned by plaintiffs have successfully circumvented democratic executive and legislative funding dynamics. The trend threatens to erode public interest in and support for K-12 education policy.

Plaintiff Victories Pick Taxpayer Pockets

Amounts awarded by courts are often substantial. Wyoming and New Jersey are fine examples. Wyoming plaintiffs have returned to court six times and have so far doubled Wyoming's per-pupil spending, elevating it from $5,971 in 1996-97 to an estimated $12,422 for 2006-07. Beginning teacher salaries, for those with master's degrees, rose in constant dollars from $24,402 in 1997 to $32,451 in 2004, a 33 percent increase. The average student-teacher ratio declined from 15 to 1 in 1993 to 13 to 1 in 2003. In spite of dramatic increases in spending, Wyoming student achievement levels in math as measured by the National Assessment of Educational Progress (NAEP) have either been stagnant or dropped relative to the United States as a whole.

While Wyoming is a poster child for litigant success, it is not all that extreme. New Jersey per-pupil spending, in response to Robinson v. Cahill and Abbott v. Burke, has been elevated in constant dollars from $4,688 in 1970, when the litigation began, to $13,229 in 2003. So-called Abbott districts, those that receive the largest share of new state funding, in select instances spend in excess of $19,000 per pupil, a figure that rivals day-student tuition at many of the nation's most prestigious independent schools.

How Much Is Adequate?

The underlying question seems reasonable enough: after all, if Johnny's school is underresourced, how can Johnny be expected to meet the state's expectations for learning? The problem is that no one knows with any degree of certainty how much money it takes for Johnny to meet state-derived learning standards.

Ensuring that sufficient resources are available for all students to meet state-specified learning standards is a laudable policy objective. Unfortunately, contemporary legal petitions for resource adequacy go far beyond the analytic capacity of present-day social science. The evolving concept of financial adequacy requires researchers to ascertain far more elusive relationships between education inputs, processes, throughputs, and outcomes. Researchers have simply not yet discovered answers to many of the questions regarding these relationships. For example, the amount of money or configuration of schooling resources needed to compensate educationally for impoverishment, disability, or language deficiency is simply not known.

Still, court cases proceed, and even proliferate, with the primary evidence coming in the form of adequacy cost studies. According to ACCESS, a project of the Campaign for Fiscal Equity, Inc., a total of 58 cost studies had been conducted in 39 states as of January 2006. Of these cost studies, state courts initiated 7, state government agencies initiated 34, and independent groups initiated 17. At least 20 cost studies in 14 different states were undertaken between January 2004 and December 2005, with a potential for at least 5 additional studies in 2006. Two adequacy cost-modeling methods are employed most often: the econometric or cost function approach and the professional judgment approach.

Cost Function Approach

In the recently decided (2005) West Orange Cove ISD v. Neeley case in Texas, plaintiff and intervening districts claimed revenues were insufficient to provide an adequate opportunity for students to meet newly elevated state learning standards. The evidence marshaled on behalf of the parties involved illustrates the weaknesses in the cost function approach to determining the cost of an adequate educational opportunity.

Generally speaking, researchers employing the cost function strategy specify a desired student performance level. They then seek school variables or student conditions such as household and community characteristics and school service levels (for example, teacher qualifications) that are statistically associated with that performance level. Researchers employ a cost function model to approximate spending associated with the desired outcome.

The heart of the evidence regarding adequacy in West Orange Cove was provided by two cost function studies, one submitted by the plaintiffs and one by the defense. The table in Figure 1 identifies similarities and differences between the plaintiff and defense studies. The studies relied on similar econometric models, data, and proficiency standards. Differences existed in the treatment of value-added measures, tests (SAT/ACT variables), and school-district efficiency controls.

While the plaintiff and defense cost function models initially appear quite similar, the subtle differences resulted in drastically different conclusions. The defense study estimated that $226 million to $408 million in additional state or local funding was needed for the entire state's student enrollment to reach a 55 percent passing standard. The plaintiffs' study estimated a need for $1.7 billion to $5.4 billion more in public funding. Such radically different estimates stem from the current state of cost function analysis. Varying assumptions, model specifications, and data deficiencies inform analyses that consequently lack objectivity, robustness, and precision.

This rendering is not intended as a criticism of the defense or plaintiff research. To the contrary, they have addressed a significant policy issue, and their studies advance cost-modeling strategies. Nevertheless, one must realize that seemingly innocuous assumptions made by each research team--none of which can be proven to be technically "wrong"--yield radically different results. In their current state, cost function analyses are simply inadequate for guiding changes in state education-finance policy.

Professional Judgment Approach

The modern introduction of the professional judgment approach into education finance began in Wyoming as a consequence of Campbell County School District v. State of Wyoming in 1995. The challenge resulting from Campbell was to provide the legislature, not the court, with an estimate of what it would cost to achieve a "visionary" school system, mandated by the court to be the best in the world. The absence of a statewide student achievement test limited the means by which the cost to Wyoming of such a high-performing school system could be determined. (It was not until the 1998-99 school year that the state began the annual administration of the Wyoming Comprehensive Assessment System [WyCAS].) Finance experts retained by the state modeled their practices after those of the Food and Drug Administration, Air Traffic Control Administration, and Veterans Administration, all of which have relied for years on professional judgment panels when faced with information deficiencies.

Typically, a professional judgment approach to costing-out adequacy relies on experienced professional educators to identify the resources necessary to produce desired outcomes. This presumes the ability of professional educators to exercise craft knowledge in design of instructional programs and auxiliary services sufficient to ensure an adequate opportunity for all students. Upon a panel's completion of instructional program design, economic and finance experts impute current market prices (for salaries, fringe benefits, and supplies) to create a final per-pupil cost figure. In the early days, circa 1995, researchers employing the professional judgment approach were careful to make sure everything was transparent and met as rigorous a standard as possible. Selection of panel participants, for example, was carefully done so as to be as representative and objective as possible. Solace accrued to those engaged in what was openly acknowledged as a porous means for determining costs from knowing that recommendations were being provided to an executive branch agency or a legislature--a deliberative body whose participants could take into account weaknesses in the evidence.

The professional judgment approach proliferated in education-finance lawsuits following Campbell. Approximately 18 professional judgment studies conducted by at least seven different research groups were tendered as evidence in adequacy litigation from 1996 to 2003. Many are devoid of methodological rigor and injected with self-serving biases, rendering results unworthy of legislative, executive, or judicial consideration. In the politically polarized domain of school-finance litigation, the professional judgment approach has evolved into a den of foxes guarding the henhouse.

Take, for example, a recent professional judgment study prepared by Verstegen and Associates for Young et al. v. Williams et al., a pending education-finance adequacy trial in Kentucky. The firm's evidentiary submission relied principally upon two organizations to select teachers, curriculum specialists, and administrators to serve on professional judgment panels that constructed the resource ingredients that would ultimately help determine the cost of an adequate education in Kentucky. The first organization was the Council for Better Education, Inc. (CBE), a coalition of school districts that first brought an adequacy suit against the Commonwealth of Kentucky in the late 1980s and then for a second time in 2003. The second organization was the Kentucky Education Association (KEA), the state's National Education Association (NEA) affiliate, whose mission targets "improved" education funding and which further identifies "enhancing" compensation and benefits as a principal reason for educators to join their association.

To the extent that panel participants knew their work was likely to be used to advocate for additional funding for the state's public schools (or that panel participants' predispositions reflect CBE's and/or KEA's mission and values), there is an obvious bias injected into the Verstegen and Associates study from its inception. For that reason, one might not be too surprised to learn that the report concluded between $892 million and $1.162 billion in additional funding was needed to achieve an adequate education. Furthermore, raising teacher salaries was identified as one of three "key resource requirements" for attaining educational adequacy.

We reviewed results from professional judgment studies in eight states released from 2001 to 2003. All eight studies were conducted by Augenblick & Myers, Inc. (A & M) or Augenblick Palaich & Associates, Inc. (APA). This review was limited to studies by these two groups for three reasons. First, professional judgment exercises organized by A & M and APA impose similar constraints on panel participants across studies and, in most instances, have relied on similarly trained staff members to facilitate panel deliberations. This makes cross-state and cross-study comparisons more reasonable. Second, they report the number of instructional personnel required per 1,000 "regular" students in nearly every professional judgment study conducted. This provides a common metric for comparison. Finally, A & M and APA have conducted the most professional judgment studies by any single group of consultants, thus providing the largest possible sample with a common metric for comparison purposes.

Figure 2 shows instructional personnel per 1,000 "regular" students recommended for elementary schools and high schools in several states. Projected numbers for instructional personnel vary significantly, even when the contexts are similar. For example, professional judgment panels in Maryland estimated, on average, that elementary schools need 116 instructional personnel per 1,000 regular students to meet state standards. Nebraska elementary schools require 90 instructional personnel per 1,000 regular students, whereas Montana only needs 75 instructors.

Unjustified variances such as those displayed in Figure 2 suggest, at a minimum, that there is no science involved in such estimations. The present state of education-finance adequacy cost modeling is highly vulnerable to assumptions made by researchers. Even within studies one can find great disparities. For example, the combined corporate team providing advice to New York plaintiffs in the Campaign for Fiscal Equity remedy phase, a joint venture of American Institutes for Research (AIR) and Management Analysis & Planning, Inc. (MAP), a Davis, California-based consulting firm specializing in education finance and litigation support (of which coauthor Guthrie is chairman of the board of directors), had contrasting approaches for adjusting differences between cost estimates generated by multiple professional judgment panels (see "Pseudo-Science and a Sound Basic Education: Voodoo Statistics in New York," check the facts, Fall 2005). The differences, distilled to their essence, were as follows. AIR aggregated cost estimates across multiple panels, essentially taking the highest amount recommended for each category. MAP contended that averaging the cost estimates from different panel submissions was more appropriate; taking the average yielded a lower recommended level of spending for each category. The different approaches resulted in estimates of additional funding needed that were billions of dollars apart.

Adequacy Is Jeopardizing Public Accountability

In the equity-based litigation of the 1970s and 1980s, researchers investigated and courts intervened in deep-seated education-finance issues of public policy significance, such as school funding schemes that exacerbated existing socioeconomic disparities. By contrast, contemporary adequacy research and lawsuits are increasingly pursued by narrowly self-interested plaintiffs as a means to bypass the conventional competitive political process and procure private gain at public expense. These very same "we need more money" groups ardently oppose operational innovations, such as competition and incentives, that may increase efficiency and productivity and help ensure that all children are provided the opportunity for an excellent education. Adequacy advocates seldom open the door to alternative public-policy solutions, such as income maintenance, housing subsidies, or health and nutrition improvement, for addressing the education problems of the disadvantaged.

Selfishness is not new to jurisprudence. Certainly, other parties at interest have pursued the protected calm and rational patois provided by the judicial process in place of subjecting themselves to the bare-knuckle pummeling of political debate. If this alone were all that was taking place, then there would be little or no news here. However, there is a deeper and more sinister dynamic at play in modern education-finance adequacy research and litigation. Self-interested advocacy has become a primary driver for adequacy research and, as the following example illustrates, now plagues the adequacy reform movement.

In August 2003, a 19-member task force commissioned by Minnesota governor Tim Pawlenty convened to make recommendations for revamping Minnesota's K-12 education-finance system. In addition to including prominent Minnesota school superintendents, a former state finance director, a state board of education president, and local community leaders, the School Funding Task Force engaged MAP (with which coauthor Guthrie was engaged). The resulting report's principal recommendations included (1) linking education funding to student learning; (2) elevating school accountability; (3) fostering community engagement and encouraging educator creativity; (4) continuing to value school choice; and (5) sustaining progress toward funding equity.

In September 2005, approximately 18 months after the School Funding Task Force report was released, the Association of Metropolitan School Districts, the Minnesota Rural Education Association, and Schools for Equity in Education contracted the services of APA to "examine the Task Force results and, using widely accepted methodologies, determine the costs necessary to ensure that each public school student is educated to meet the state's academic standards." APA's "continuation" of the School Funding Task Force's report estimated that $7.998 billion was required to realize educational adequacy for the 2003-04 school year, which implied that Minnesota's public school system was underfunded by approximately $953 million during that year.

It is worth noting that student outcomes measured by the NAEP scores are quite favorable for Minnesota. Only Massachusetts had a greater percentage of students at or above proficient in mathematics in 4th grade in 2005, while no state had a greater percentage of students than Minnesota at or above proficient in mathematics in 8th grade. When charting the average mathematics scale score and percentage of students eligible for free and reduced-price lunch in the 4th and 8th grades, we find that only nine or fewer states had a smaller percentage of students than Minnesota below "basic" proficiency. While areas in need of improvement surely exist, it is also difficult to believe an "inadequate" state is among the national leaders in student achievement by a widely accepted measure.

Salvaging Adequacy as Legitimate Public Policy

Clearly, cost modeling deficiencies, pseudo-scientific research practices, and advocacy testimony are tainting contemporary education-finance adequacy litigation. Yet readers should not interpret these criticisms as calling for an end to the adequacy reform movement. The principled cause of adequacy is legitimate. America's public schools surely would be enhanced if assured the optimal mix of resources, incentives, practices, and structures. Consequently, we set forth three recommendations by which adequacy-driven reform and cost modeling strategies can become more effective.

Invest in Research The policy system thirsts for more precise information regarding 1) the manner by which education resources can be deployed to gain greater student performance returns and 2) how much money is actually needed to ensure students an opportunity to learn what is expected. With a new generation of reform-minded philanthropists targeting high-profile efforts that will influence educational politics and policy, the time is ripe for a project that targets methodological advances and improvements for cost modeling strategies. A modest outlay, in the range of $10 to $20 million, could have dramatic potential if expended on convening the nation's top-flight statisticians, economists, and psychometricians to construct a research and methodological task force to buttress cost modeling strategies.

Raise the Standards If adequacy research and cost studies are to have a useful role, then steps must be taken to construct higher evidentiary standards. An independent organization such as the National Research Council, Council of Chief State School Officers, or Education Commission of the States should convene a panel of finance experts to construct adequacy research and cost-study standards. A clear set of evidence standards and screening criteria would be useful to legislative and executive bodies when appraising the objectivity and scientific reliability of cost studies that aim to determine the adequacy of state funding mechanisms.

Change the Venue Finally, the venue for adequacy deliberation should be the statehouse, not the courthouse. Decisions regarding adequate funding levels and designing means to provide all students with the opportunity to reach state standards of proficiency should reside with state legislative and executive branch officials. The judicial system is best suited to considering solutions for issues brought before the court. Legislative and executive branch deliberations are far better adapted for accommodating uncertainty, deconstructing complexity, and considering the tradeoffs inherent in education policymaking. Unlike the judiciary, legislative and executive bodies might also have the capacity to consider, alongside resource investments, innovations aimed at making sure those resources are used effectively.

James W. Guthrie is professor of public policy and education, Peabody College of Vanderbilt University. Matthew G. Springer is research assistant professor of public policy and education, Peabody College of Vanderbilt University. The unabridged version of this essay is available in Martin R. West and Paul E. Peterson, eds., School Money Trials: The Legal Pursuit of Educational Adequacy, forthcoming from the Brookings Institution Press.

Texans Make Mountains Out of Molehills (Figure 1)

Despite many similarities in the methods used by plaintiff and defendant
school-funding experts in Texas, the small disparities between them
resulted in major differences in the estimates of the cost of an
adequate education.

Methodologies Employed by Plaintiffs and Defendants

Both defendants and plaintiffs:
* Used the cost-function approach.
* Attempted to determine appropriate district expenditures per pupil.
* Considered how the state test scores for the district should affect
  its expenditures.
* Statistically controlled for the percentage of students who are poor,
  do not speak English, are in special education programs, and are in
  high school (high schools are more costly than elementary and middle
  schools).
* Used the same method for estimating teacher cost.

Modest Differences
* Defendants and plaintiffs used competing measures to determine the
  cost of achieving higher levels of student performance.
* Defendants used operating revenue, and plaintiffs used operating
  expenditures (a trivial distinction) to determine costs.
* Defendants and plaintiffs used different estimates to adjust for
  inflation.
* Defendants took geographic isolation of schools into account;
  plaintiffs did not.
* Plaintiffs considered the number of minority students; the defendants
  did not.
* To account for poor and non-English-speaking students, defendants used
  elementary-school figures; plaintiffs used district-wide percentages.

Case: West Orange Cove ISD v. Neeley
SOURCES: Imazeki and Reschovsky, 2004; Gronberg, Jansen, Taylor, and
Baker, 2004

Guess Work? (Figure 2)

Across states, adequacy case professional judgment panels vary widely in
their estimates of the number of instructional personnel required per
1,000 pupils.

Instructional Personnel Required (as estimated by professional judgment
panels)

                 Instructional Personnel
                    (per 1,000 pupils)
              Elementary School  High School

Indiana        63                60
North Dakota   72                58
Kansas         74                84
Montana        75                84
Tennessee      88                66
Missouri       90                88
Nebraska       90                79
Maryland      116                79

SOURCES: Professional judgment studies conducted by Augenblick & Myers,
Inc., and Augenblick Palaich & Associates, Inc.

Note: Table made from bar graph.