Curious Georgia studies economics: the tale of a researcher and a superintendent who debate the merits of measuring a school system's level of educational efficiency.

District and school administrators are challenged to provide learning environments that help students attain and surpass set levels of academic and social learning. In times of economic strife, discussions often turn to the efficient use of financial and human resources to maximize school districts' capacity to fulfill those objectives.

Most efficiency theories assume that public school administrators act similarly to managers of private businesses and thus pursue strategies aimed at minimizing costs. But school administrators know their organizations are structured to support management strategies characterized by budget maximization. Therefore, the cost-minimizing assumptions associated with traditional efficiency analyses are not well-suited for evaluating the efficiency of public schools and districts.

Measuring educational efficiency in public districts and schools requires alternative methods of analysis, as illustrated in this hypothetical case study set in Tennessee.

Setting the Stage

Georgia Madison has been the superintendent of the Rocky Top High School District for 12 years. Rocky Top is a large, comprehensive high school district in a fast-growing suburb. The student population is approximately 15 percent African American and 15 percent Latino. Approximately 30 percent of the district's students receive free or reduced price lunch. Student enrollment has been relatively stable for the last 10 years and academic achievement has hovered near the state average. Still, every four or five years, Rocky Top has a few National Merit Scholars or students who score over 1500 on the SAT or obtain some similar marker of outstanding academic achievement.

Recently, due to growth in the district, the school system has expanded services to accommodate an influx of new students. Accordingly, the county commissioners have issued bonds for new construction and have raised property taxes modestly to fund expanded instructional programs and administrative costs.

During her tenure, Superintendent Madison has worked through numerous education improvement programs designed to make her district a model environment for student learning. She manages a veteran staff of well-qualified principals in a district with low teacher turnover. Considering herself a good steward of taxpayer money, Madison agreed to allow a local university professor to evaluate her district's finance and budgeting practices.

Madison welcomed the opportunity. Professor Florida Sydney's research is focused on economic efficiency and productivity theories as they apply to public schools--a topic about which Madison has little interest and for which she has little patience. While she did not have much interaction with Professor Sydney during the research assignment, she asked that her staff be accommodating hosts, and they were. Sydney spent a great deal of time at the school district's financial office and in the records room before beginning to meet with the school district's instructional and administrative staff.

A Professor's Accusation 

As the academic year wound down, Professor Sydney scheduled a summary meeting with Superintendent Madison to review the preliminary findings of her budget evaluation project. To Madison's surprise, the professor expressed disappointment in Rocky Top's low level of educational productivity. Sydney supported her observation with the following:

* The administration at Rocky Top received and spent increasing amounts of revenues from both state and local sources for more than 10 years, yet showed no proportional increases in student performance.

* Persistent, sizeable achievement gaps exist between affluent and less-affluent students and between white and minority students.

Citing these two rather serious deficiencies as evidence, Sydney asserted, "Superintendent Madison, my evaluation of your financial management style supports the theory I have been analyzing: You are a budget-maximizing bureaucrat."

This sounded like an insult to Madison, who requested an explanation.

"Ever since the National Commission on Excellence in Education published 'A Nation at Risk' in 1983," Sydney began, "researchers have tried to determine whether money matters in education; and if it does matter, how? Some researchers assert that educational productivity can be improved by increasing economic efficiencies in the organization, management and operation of districts and schools."

Sydney continued: "One problem with these research efforts is that they used statistical and economic methods that typically are applied to determine levels of efficiency in the business world. Surely you would agree that your school is not the same as a profit-seeking business?"

"I definitely would agree with that," Madison replied.

"I would as well," Sydney said. "Businesses that seek to maximize profits also aspire to minimize expenses, which is the only way that profit margins can be improved. That is basic economic theory. As a public school superintendent, you obviously are not trying to maximize any profit. By the same token, you also are not seeking to minimize operating expenses. In fact, a larger budget actually helps you."

"Wait a second," Madison protested. "One of my responsibilities is to spend taxpayer money wisely in support of student learning objectives. I do not spend the district's budget on extravagant items."

"Perhaps," Sydney said, "but you also want to cement your own administrative power and ensure your continuity in this post. How do you do that? By accumulating resources to fund your--and your staff's--pet projects for school improvement. To your principals and teaching staff, acquiring annual budget increases is a sign of your power as an administrator. And the more special projects you fund, the more indispensable you become to Rocky Top.

"Economists refer to the idea of gaining larger budgets to become indispensable as an example of individual rationality. And the idea that you need larger budgets in order to look good for your principals and teachers is an example of economic survival. Therefore, in a rational pursuit to survive in your professional environment, you pursue annual budget increases. You are a budget maximizer."

The Superintendent Responds

"At Rocky Top, we concentrate on reducing the number of disciplinary referrals and expulsions. We work to decrease our dropout rate, to improve our graduation rate and to increase the number of students in Advanced Placement classes," Superintendent Madison explained. "We also promote good citizenship, high standardized test scores, as well as military and vocational placement. Heck, we even teach driver's education and parenting skills here.

"As you noted earlier, some of our test scores are increasing overall, although we still have some gaps that are not narrowing. These results show that because of the positive efforts of the instructional and administrative staff, the Rocky Top High School District not only uses its budget to achieve results; they also support the fact that I am not using budget increases to advance a desire for prestige and power."

Professor Sydney explained, "The theory of budget maximization does not hold that you will not achieve results; it just states that you achieve results inefficiently. Education finance and economics researchers are concerned primarily with two types of efficiency in the production of student learning outcomes: technical efficiency and allocative efficiency.

Technical efficiency is achieved when: (a) output levels cannot be maintained with lesser amounts of inputs; or (b) output levels cannot be increased while holding inputs constant. In other words, you want the biggest bang for your buck.

Allocative efficiency, loosely speaking, is achieved when your entire budget is spent in an attempt to deliver as many desired services as possible. If Rocky Top spent its budget only on efforts to achieve measurable student learning outcomes, fewer resources would be used and economic efficiency would increase. That is why economic theory claims budget maximizers--those who spend dollars on pet projects--tend to be inefficient managers."

"You make me sound like the stereotypical bureaucrat," Madison complained. "I am not here just to advance myself. I care about my students and this community!"

"I do not doubt that for a second," Sydney assured her. "However, Dr. Madison, you must acknowledge that you can only act on your commitment to students and the community while you hold the power of the superintendency. If you do not have a job, you cannot effect change."

Madison mused, "I am interested in knowing how to get better results and ensure that I am using budgeted dollars efficiently. However, the education game is not just about efficiency; as a matter of fact, some of my colleagues argue that we should not worry about being efficient at all. Furthermore, I am uncomfortable being classified as a bureaucrat. I would like to see whether there is some way I can challenge or modify the theory that you have presented to me. I have some time this summer that I can use to understand these concepts better. May I have a copy of your report?"

"Certainly," Sydney replied. "You may want to begin by reading William Niskanen's original research on budget maximization theory, Bureaucracy and Representative Government, for a good initial orientation."

Digging Deeper

During the summer, Superintendent Madison conducted her own review of the professor's findings. She learned that, consistent with Professor Sydney's application of budget-maximization theory, school districts are public organizations that promise a range of educational services and expected outcomes based on those services in exchange for a tax-supplied budget. But she also learned that budget-maximization theory claims that expenditure decisions made by public school administrators are not consistent with those made by managers of private firms for at least three reasons:

* School administrators cannot enhance the profit of their organization by creating internal efficiencies that lead to direct savings;

* School administrators are not motivated economically to create the impression of fiscal integrity in order to attract individual consumers; and

* School administrators are not motivated to minimize unit costs for a given level of quality in order to price the services of their organizations.

In addition, Sydney's report ignored at least one important theoretical concept: How educational efficiency and productivity should be measured. Sydney still measured the amount of inefficiency in public school districts as if they were businesses. But the application of these methods is unwarranted because, according to Niskanen himself, conventional measurements of economic efficiency cannot be applied to public agencies. Therefore, Sydney's analyses of Rocky Top's budgets and student learning outcomes using conventional economic measures of efficiency within a budget-maximizing framework may have generated erroneous conclusions.

Luckily, Madison had access to her school district's evaluation department, where a central-office staff member helped her recreate Sydney's economic and statistical results, in addition to a few others. When the professor arrived for a debriefing in the fall, Georgia Madison was more than prepared.

Seeing All Sides

Upon meeting with Professor Sydney, Superintendent Madison asserted that much of the research examining educational productivity issues has left unanswered a number of important questions about the nature and levels of efficiency in public school districts. For example, what incentives and constraints influence expenditure behavior? What individual or collective organizational objectives are maximized or optimized? To what extent and under what circumstances are individual or bureaucratic desires reflected in the organizational outcomes generated by public school districts?

These types of questions become particularly important to administrators given that trends in education seem to be exemplified by continued increases in organizational size and fiscal resources with only minimal increases in educational outcomes.

"Numerous researchers," Madison continued, "have discussed educational efficiency by using concepts embedded in the seminal educational finance research to discuss productivity issues in terms of dollar inputs that generated educational outcomes. Research by Eric Hanushek and Herbert Walberg, for example, found no significant statistical relationships between educational expenditures and student achievement. However, Larry Hedges, Richard Laine and Rob Greenwald found that increasing education spending did result in higher achievement when using more appropriate statistical methodologies.

"And finally," Madison stated confidently, "even though current economic research methods indicate that the relationship between educational resources and student learning is unclear, many academics believe there must be some type of economically efficient relationship between educational inputs and student outcomes and a way to measure it appropriately as long as resources reach schools, classrooms and students."

After listening to the presentation, Sydney responded, "That was thorough work. Naturally, I will need some time to review this output and ensure that you and I have approached the problem in the same manner." Rifling through a stack of computer printouts, the professor continued, "However, assuming that your work is correct, I am interested to know how you think we should determine whether you are doing your job efficiently."

"I'm glad you asked," Madison replied. "Here is what my colleagues and I did. For the student learning outcomes we discussed last spring--achievement, attendance, dropout rate, graduation rate and Advanced Placement course-taking--I plotted every high school in the district on a scatter graph. The horizontal axis represents funding levels per student; the vertical axis represents a student performance indicator. Then I drew a vertical line from average school expenditures and a horizontal line from the mean school outcome performance. Now, the graph is broken into four boxes. See? This modified graph is called a quadriform."

"Now on every outcome, we can see who is performing with what level of funding relative to every high school in the district. Some schools will use a lot of money to gain high outcomes and some schools will use only a little money to gain high outcomes. Some schools will have low outcomes but spend only a little money and some unfortunate schools will have low outcomes despite high expenditures of money. With this method, each school in the district is compared to its peer schools and not to some ideal level of efficiency or statistical equation."

"I am intrigued," the professor responded. "Let me ask you this: How would information like this change the way your schools or the district does business?"

"Well, the quadriform method allows us to see who our high-performing schools are and who may need additional help regardless of the management strategies used or policy goals pursued," said Superintendent Madison. "For example, in two categories, Goldendale High is a low-expense, high-performance school district. In your language, Professor Sydney, that would mean that they are efficient. In the other categories, they are high-expense, high-performance. Those are areas where I think we can be more efficient. This quadriform compares all of the high schools in our district, but we could use the same idea to compare all of the schools in a state, all of the districts in a state or even make national comparisons. What do you think, Professor?"

"I am not sure," Sydney replied, grabbing a marker and heading to the dry eraser board. "Do you have a few minutes to help me brainstorm? And when we are finished, can I have a list of the references used for your analysis?"

After a fairly lengthy discussion, Sydney and Madison agreed that school administrators always will need to confront economic, financial and budgeting issues, especially with the implementation of the No Child Left Behind Act. They also agreed that regardless of the analytic framework used, the ultimate goal of educational productivity research is to improve the quantity and quality of educational opportunities for children. And as such, alternative forms of educational productivity and efficiency measures need to be explored in tandem with more traditional methods.

Finance and Efficiency

Unfortunately, major challenges still complicate the study of educational productivity. These issues range from measuring inputs accurately to selecting the most appropriate statistical models to determine the role of innate intelligence in student achievement. Accordingly, attempts to improve traditional educational productivity now focus primarily on three areas:

* Understanding relationships between human resources allocation, individual preferences and organizational incentives;

* Developing systematic district-, school- and classroom-level data collection, management, reporting and dissemination mechanisms; and,

* Creating incentives that transfer individual and organizational productivity efforts into pursuits of educational policy outcomes.

Despite these efforts to improve the measurement of educational productivity, school administrators and researchers should explore at least three more conceptual ideas in the continued pursuit to measure levels of economic efficiency accurately:

1. Expanding the traditional two-stage production function relationship (i.e., dollars purchase services and services generate outcomes) into multistage models that more accurately portray the educational process (e.g., dollars purchase personnel; personnel provide services; students use services; and students generate outcomes);

2. Analyzing individual subgroups (e.g., by poverty status or language ability) using expanded statistical models and relationships that more accurately represent the educational process for these students; and

3. Investigating the effects of time-and other time-lagged effects--on economic models and relationships that more accurately represent the educational process for all students and subgroups of students.

In addition, these ideas should be reassessed while paying particular attention to their immediate, average and longitudinal effects on incremental, comprehensive and systemic education reform strategies.

Scarce Resources

Finally, it is important to remember that public school spending is conducted in a sociopolitical environment where organizations and individuals struggle over the capacity to distribute scarce resources. This conflict leads to negotiations, compromises and various organizational, political and personal goals that may oppose one another.

And acknowledging that non-economic forces influence educational productivity, school administrators and researchers also should explore at least two other relative efficiency measurement methodologies in addition to the modified quadriform described here: stochastic frontier analysis and distance function analysis. Both methods are used commonly by economic and public policy researchers to evaluate levels of efficiency for all organizations relative to the best-performing organization(s).

As Professor Sydney learned during her time spent in Rocky Top, the education game is not just about the money. The ultimate goal of educational productivity research should be to improve the quantity and quality of educational opportunities for children.

Additional Resources

For more information about economics and educational efficiency, the authors recommend the following articles, which they say are accessible and relevant for superintendents:

"Rethinking Educational Productivity and Its Measurement" by R. A. Rolle, in Measuring School Performance and Efficiency: Implications for Practice and Research edited by L. Stiefel et al., 2005, Eye on Education, Larchmont, N.Y.

"Out With the Old and in With the New: Thoughts on the Future of Educational Productivity and Efficiency" by R. A. Rolle in Peabody Journal of Education, 2004, 79 (3).

"Obstacles and Opportunities: Some Simple Economics of School Finance Reform" by W. S. Barnett in Educational Policy, 1994, 8 (4).

"The Politics of Educational Productivity" by W. L Boyd and W. T. Hartman, in Micro-level School Finance: Issues and Implications for Policy, edited by D. H. Monk and J. Underwood, 1988, Bellinger Publishing Co., Cambridge, Mass.

Anthony Rolle is an associate professor of education at Texas A&M University, 4226 TAMU, College Station, TX 77843. E-mail: [e-mail]. Eric Houck is an assistant professor of educational administration and policy at the University of Georgia.

Basic Quadriform Diagram

Quadrant 1:
Efficient
Low Input--High Output

Quadrant 2:
Effective
High Input--High Output

Quadrant 3:
Ineffective
Low Input--Low Output

Quadrant 4:
Inefficient
High Input--Low Output