Doubts About Decentralized Decisions.

Research Questions Positive Correlation Between Site-Based Management and Student Outcomes

Studies evaluating site-based management efforts concentrate almost exclusively on the changing methods. They express a general notion that once the processes of decision-making change, schools will be more effective instruments for educating children.

The studies, however, fail to look at the effects of site-based management on educational outcomes--either because the SBM advocates do not regard achievement as an important output measure or because there is faith that increased school discretion will increase student learning. As a result, little evidence exists to support the notion that SBM is effective in increasing student performance.

The few quantitative studies about SBM are not statistically rigorous, and the evidence of positive results is either nonexistent or weak.

Research Insights

We examined 20 relatively systematic studies for insights about the authority delegated under the specific SBM program, the inputs into the program, the explicitly stated objectives, and, most importantly, the outcomes.

* Authority Delegated.

The four major areas with increased discretionary power were curriculum, budget, personnel, and strategic planning. Eleven of the plans delegated increased authority to all four areas, six of the plans to three of the areas, and three of the plans to two of them.

With a few exceptions (strategic planning in Hammond, Ind., Chicago, New York City, Dade County, Fla., and Monroe County, Fla.; budgetary functions in Chicago and Monroe County; and curriculum and personnel in Chicago), the authority delegation, when given, was partial or advisory in nature. In these exceptions, it was complete.

* Inputs.

All but two of the 20 SBM plans involved additional fiscal resources in the form of directly expanded budgets to facilitate new activities and training programs for the participants. In most cases, new administrative positions were created to administer SBM. In two districts, more discretion was given to the schools, but not more dollars.

Despite a number of years of SBM experimentation, it is not possible to estimate the costs of site-based management. The omission of this information from the hundreds of descriptions and evaluations suggests that little emphasis is being placed on the resources required--and certainly not on the opportunity costs.

* Objectives.

All programs were explicit in having teachers more empowered and principals more independent in making decisions. Most were explicit in wanting parents and/or the community more involved.

Most stated in general terms that improved student achievement or learning was an objective, but five did not--those in Edmonton, Canada, Chesterfield, Mo., Cleveland, Memphis, and Riverside, Calif. Only three--in New York City, Chicago, and Philadelphia--had explicit, numerically stated student achievement targets. If one were to measure by space allocated, the statements of objectives in these studies clearly support the contention that the prime target of SBM has been organizational--changed governance, changed decision-making processes, and changed stakeholder relationships.

* Outcomes.

Six of the studies reported decisively there was more teacher and principal empowerment and other stakeholder involvement, seven reported evidence of some increase in autonomy, and the remaining seven had either no significant or negative results on autonomy or empowerment.

The results of site-based management on the most important outcome, student achievement, were not reassuring. Of the 20 studies, nine gave no results; three asserted positive results, but gave no achievement data; one asserted no or negative results, but gave no achievement data; two included achievement data and showed positive results; and five included achievement data and showed no or negative results.

The results of SBM, then, appear to be some increased sense of empowerment and involvement of the stakeholders (though not uniformly so), and virtually no evidence that SBM translates into improved student performance. Of particular concern are the four evaluations that used hard data on student achievement as outcomes and used adequate controls in the evaluation and showed negative or no significant benefits from SBM. Those studies examined site-based management in Dade County (in two distinct studies), Monroe County, Fla., and Philadelphia.

Study Summaries

Four studies on site-based management in local schools Warrant individual attention because student achievement was regarded as an important objective and received relatively careful evaluation.

In these evaluations, hard data were used to measure student performance, and some control factors-other than SBM-were introduced to isolate the SBM effects.

* Dade County, Ha.

The Dade County Office of Educational Accountability mounted a three-year evaluation of its SBM program. The first two years of evaluation focused on process characteristics such as how principals and teachers thought the implementation was proceeding and the attitudes of the stakeholders. The third year of evaluation continued addressing process, but focused on the impact of SBM on a wide variety of performance indicators, including student achievement and attendance.

In the 33 pilot schools] selected for testing, 5- to 12-person faculty councils (consisting of the principal, other administrators, and faculty) were set up as the decision-making body, supported by a number of faculty subcommittees. Partial authority for curriculum, budget, and personnel and virtually full authority for strategic planning were delegated to the faculty councils. More teacher empowerment was stated as the major objective.

Other objectives included increased discretionary resources to schools, increased evaluation of instruction, and increased community and parental involvement

Improved achievement of students was made an explicit objective only at a late stage of the project and the data for assessing this was also then made explicity--Stanford Achievement and Florida State Student Assessment test scores, days attended, the number of disciplinary referrals, and the number of dropouts.

Changes in the results in the 33 SBM schools were assessed in relation to changes in the performance of all non-SBM schools by simple comparisons of results, with tests of significance applied to some of the differences. No other controls were used. The results showed no significant change in test scores in reading and math at any level of schooling. Student attendance appeared to improve slightly more in SBM schools, but not decisively so. Suspension rates became significantly lower in] SBM middle and senior high schools in comparison to non-SBM schools, as did dropout rates.

Dade County's evaluation offers some support for the proposition that increased school-site discretion could improve student performance. Though no improvement in test scores was noted, improvement in attendance and dropout rates might reasonably be expected to translate into higher achievement over the longer run.

The policy force of these findings, however, must be tempered by the knowledge that the positive results are small-and, more aimportant, no effort was made to identify the characteristics of the SBM schools, other than having an SBM program, that might explain even these small differences.

An independent evaluation of Dade County's restructuring efforts was done by two researchers at Louisiana State University. They looked at 33 schools in Dade County, 16 of which were restructured and 17 of which were used as comparison schools.

This evaluation showed no significant correlations between student achievement and a variety of characteristics of more empowered teachers--teacher participation in what to teach, how to teach, and subject and grade assignment. This study is among the most sophisticated of the SBM evaluations, and it finds no positive effects of this restructuring.

Moreover, it adds definition to the evaluation issue by examining the link between specific "voices" of classroom teachers with increased empowerment and their effects on student achievement--and finds no linkage.

Monroe County, Ha.

SBM entered the Monroe County schools in the mid-1970s, and by 1979 evaluation studies were commissioned to evaluate some effects on process changes in the schools. Great emphasis was put on ongoing data collection and evaluation from the very beginning of the discussions on SBM.

School planning teams had complete control over developing a strategic plan, substantial control over allocating resources within an overall budget constraint, partial control over personnel (with considerable stress on the importance of performance measures), partial control over curriculum, and complete autonomy over the implementation of the curriculum. Teacher and principal empowerment were major objectives. Increased student achievement was a specific objective, and test data were regarded as the appropriate measurement instrument.

The student achievement results are presented more systematically in this study than in the other three. The closest thing to the use of controls is comparisons of Monroe County student performance with those of Florida and, in the case of SATs, with the nation. In comparing 1985-86 with 1988-89, the major conclusions to be drawn from the statewide data are these:

* reading scores increased slightly more in the state than in Monroe County for third grade students; showed the same level of increase for the fifth grade students and less for eighth grade students; and declined less in Monroe County than in the state for 10th grade students;

* writing scores increased for third graders in the state, though in Monroe County they declined; for fifth graders they stayed virtually unchanged for both; for eighth graders, Monroe County scores increased while state scores declined; and for 10th graders, county scores declined less than did those in the state; and

* math scores in the county were about the same for third graders; increased slightly more for fifth graders; and were higher in Monroe County for eighth and 10th graders.

Comparisons of SAT score changes yielded these results between 1984-85 and 1988-89:

* in the verbal scores, national averages declined 4 points, Florida declined 1 point, and Monroe County went up 2 points; and

* in math scores, national averages increased 1 point, Florida increased 4 points, and Monroe County declined 4 points.

All in all, the comparisons on K-12 statewide testing show no clear evidence of greater student performance in Monroe County than in the rest of the state. The comparisons on SAT scores suggest that Monroe County students did somewhat better than the rest of the state in the verbal test, somewhat worse in the math test.

* Philadelphia.

A longitudinal evaluation of the effects on student achievement of increased collaboration within a school was undertaken by the Center for Research on Effective Schooling for Disadvantaged Students at Johns Hopkins University.

Site-based facilitation of collaboration in 40 elementary schools in Philadelphia was the major characteristic of the plan for increased autonomy. Twenty other schools constituted a comparison group.

Increased collaboration and increased empowerment were regarded as major objectives, and surveys of principals indicated some success in these objectives--mostly with respect to selecting basic materials and purchasing instructional hardware.

Another major objective was getting Chapter 1-eligible students to attain the same average gain in reading scores as other students in the school district. Only in the second grade was there empirical evidence that increased collaboration translated into comparably increased reading scores. For the first, third, fourth, and fifth grades no significant effect was found.

Beyond that, the study examined the impact of seven components of the collaborative efforts on reading achievement. In only two out of the seven was a consistent positive effect found --teacher involvement in the deployment of human resources and a school's use of districtwide programs.

Chapter 1 funds of $900 per pupil were spent for this program. The impact of that expenditure would appear to be that teachers had more voice, but student reading achievement did not consistently improve.

Two Conclusions

A closer review of four studies using some controls and some specific information about student performance under site-based management yields two major conclusions.

First, several overwhelming obstacles stand in the way of evaluating the impact of SBM on student achievement. Virtually no empirical and statistical evidence exists in the literature. While there are many different designs to SBM programs, few identify student achievement as a major objective. Virtually no attention is paid to the question of how the process changes could affect student performance.

Second, the studies that have some numbers and some controls provide no significant support to the proposition that school-based management will increase student achievement. Clearly, that is not a conclusion in which one can have confidence. The data are inadequate, statistical controls are largely nonexistent, no repeated effort has been made to classify various types of SBM designs in some uniform way, and the time period may be too short to read results.

In conclusion, it should be noted that a major reason behind the current reform movement in education is the dismay with which the nation has reacted to the evidence of declining student performance. The longstanding pattern in educational evaluation of setting up many, many objectives--most of which are immeasurable--has continued into the review of SBM efforts.

Good results may flow from site-based management, but surely if it does not generate improved educational performance, it cannot be judged to be a success.

Anita Summers is professor emeritus in the department of public policy and management at the University of Pennsylvania. Amy Johnson is project manager at the institute for Research on Higher Education at the University of Pennsylvania.