The impact of TQM on highway maintenance: benefit/cost implications.
The early literature on TQM in the public sector -- which has tended to be expository (Carr and Littman, 1990; Sensenbrenner, 1991), skeptical (Milakovich, 1990; Swiss, 1992), or instructional in nature (Cohen and Brand, 1993) -- has given way to accounts of implementing TQM in various agencies and the lessons learned from these experiences (Cox, 1995; Maher, 1995; McNabb and Sepic, 1995; Rago, 1996). However, these reports do not address the issue of TQM's effectiveness, and attempts to do so have been inconclusive. For example, a review of a well-established TQM program in the Florida Department of Transportation found that employees had favorable perceptions of its impact on the department's operating efficiency, but no financial or operating data were available to confirm this impression (Bowman and French, 1992). A study of long-standing TQM efforts at the U.S. Internal Revenue Service, which did track hard data on inputs and outputs across pre- and post-TQM periods, found no significant difference in productivity before and after TQM implementation, although the author suggests that TQM might have helped the IRS retain the benefits of earlier productivity increases (Mani, 1995).
There are several reasons why evaluations of TQM effectiveness have not been forthcoming to date. First, many agencies' programs are their infancy and are concentrating on the early stages of quality improvement. Total quality management philosophy stresses the need for long-term commitment, emphasizing cultural change, skill development, and behavioral modification that would then lead to improved performance. Therefore, tangible improvement in service delivery may only be expected to come further along in the process. Moreover, TQM processes are intended to produce incremental, continuous improvement that will accumulate into meaningful change over the long run, as opposed to process reengineering, which is intended to produce dramatic improvement in the immediate future (Davenport, 1994). While agencies often document specific operational improvements and cost savings produced by quality process teams along the way, they often do not have measurement systems in place to track accumulated impacts of TQM programs over the long run.
In addition, public organizations are not always clear regarding their objectives in implementing TQM programs. In addition, these objectives may shift over time from employee involvement and development, innovation, teamwork, and healthier organizational climates, to improved service quality and customer satisfaction, increased productivity, and more bottom-line-oriented cost-effectiveness. Finally, the fact that such large-scale organizational interventions can rarely be introduced as controlled experiments, and in any case tend to evolve over time beyond original intentions, creates severe methodological challenges for evaluating their impact. Often constrained by a lack of record keeping regarding TQM programmatic activity from the outset, and inadequate or inconsistent outcome measures over time, attempts to evaluate the impact of TQM efforts on organization performance are often limited to little more than testimonials or anecdotal evidence.
Given continuing uncertainty about the true worth of TQM, a quality backlash is growing in the private sector that may well spread to increased concerns regarding public sector applications as well. The record is further muddied by the experience of agencies that proceed with very superficial versions of TQM -- sometimes referred to as total quality lip service -- or abort well-intentioned efforts in the early stages when tangible results in terms of quality and productivity do not materialize quickly. Thus, as noted by one of the principal authorities on public sector process improvement and innovation, Albert Hyde (1995), the impact of quality management on productivity growth in government remains a critical issue. Claiming that despite the skepticism, "few really doubt the primacy of quality as a major premise for improving performance," Hyde goes on to say:
In summary, there is every reason to believe that quality
management ... can be and will be a viable approach for
improving productivity and performance in the public
sector. Quality approaches centered around process
management, customer feedback, employee empowerment,
and supplier involvement are very compatible
with preferred public sector management styles. As for
productivity, the dichotomy that many see between
quality and productivity should lessen as quality
management adapts to the public sector and its service side,
while the public sector becomes more familiar with
quality management precepts and strategies (Hyde,
Nevertheless, Hyde still sees the impact of TQM as an open question, and in earlier work points out that the supposed principles underlying the approach are really no more than administrative proverbs until they are tested with rigorous evaluation designs and subjected to the measurement of results (1992). More recently, Wilson and Durant (1994) have pointed out that as the initial euphoria surrounding TQM wears off, it is critical to evaluate TQM programs systematically to determine whether they generate meaningful results. They advocate the use of theory-driven models to take implementation and contextual factors into account in interpreting results.
Purpose and Approach
This article responds to the need for systematic evaluations of total quality management in the public sector by examining the service delivery impact of quality improvement processes in the Pennsylvania Department of Transportation (PennDOT), focusing on its large highway maintenance program. Examining simple correlations, a preliminary analysis of this effort found that TQM activities showed modest positive associations with employee attitudes, quality ratings, and highway condition, as well as negative associations with grievances, sick leave usage, and injuries (Poister and Harris, 1996). Looking at updated data, the purpose of the research reported here is to identify more fully the relationships between TQM and service delivery measures in a system of variables, gauge the magnitude of impact on program effectiveness, and extrapolate estimates of the benefits and costs of these TQM activities.
PennDOT's Quality Improvement Processes
PennDOT is a large multimodal transportation agency that was transformed from a dysfunctional, unproductive, and demoralized organization in the late 1970s into a results-oriented agency with strong leadership, reinvigorated programs, positive employee relations, and a renewed public image by the mid-1980s. For the most part, the strategies initially employed to turn the department around were highly authoritarian and control oriented, but as productivity improved dramatically and then began to level off, top management turned increasingly to participative management and employee-involvement approaches (Poister and Larson, 1988).
PennDOT has been using a variety of TQM-related techniques to improve quality and productivity for well over a decade, reflecting a sustained commitment to employee involvement and empowerment that has grown through three different secretarial administrations. A quality circle program was initiated in 1982 and rapidly proliferated throughout field operations as well as central office divisions. Soon, the department began experimenting with employee involvement on a broader scale, encouraging an array of mechanisms ranging from teams within work groups to cross-division teams and department-wide task forces. All this activity was supported by a substantial investment in training for facilitators, team leaders, and others, focusing on group decision-making processes as well as simple techniques for analyzing work processes. At this time, the department also expanded employee recognition programs and challenged managers to support employee groups in their units and to involve them in improving operations.
In the mid-1980s, PennDOT adopted a standardized creative problem-solving process and began training managers and employees in these techniques on a massive scale. Quality breakthrough teams were formed throughout the department to improve the work process and resolve operational problems identified by management. By this point, PennDOT had become totally committed to employee involvement as a central strategy for innovations and organization development (Piel, 1991). More recently the department began to focus directly on customer service, developing indexes to track customer satisfaction over time and encouraging managers and employees to conduct customer surveys to monitor performance from the customer's perspective and identify needed improvements. Currently, reengineering efforts focusing on cross-divisional and cross-functional processes are being superimposed on the quality improvement foundation, and "partnering" strategies are being utilized to develop more cooperative efforts with the department's contractors and suppliers. Clearly, Penn has emerged as an early entry and recognized leader in adapting TQM-type approaches to the public sector, and employee involvement has been the central ingredient of these efforts (Harris, 1990; Kline and Young, 1994).
PennDOT's single most important operating program is its highway maintenance program, which is carried out by some 6,600 employees working in 67 county-level maintenance units across the state, accounting for roughly half of the department's total workforce. These highway maintenance units are headed by county maintenance supervisors who report to 11 district engineers who, in turn, report directly to the deputy secretary for highway administration. While top management sets overall priorities and monitors performance, this is an increasingly decentralized operation in which the county maintenance managers have considerable flexibility in developing their annual business plans and managing maintenance activities.
Particularly with respect to the highway maintenance program, then, Penndot lends itself to an analysis of the impact of TQM processes on service delivery for several reasons. First, unlike most public agencies, it has had well-established employee involvement and continuous improvement processes in place for several years -- time enough for signignificant results to have accumulated. Secondly, it is a product-oriented agency with readily measurable outputs. Third, given the overall decentralized and participative management thrust in the department, there is considerable variation among the county maintenance units in the extent to which they have emphasized TQM approaches. This variation facilitates statistical analysis of relationships between indicators of TQM activity and other measures of performance. Finally, PennDOT has been recognized as a leader among state transportation departments in using output and outcome-oriented performance measures, so the requisite data are available (Hyman, Alfelor, and Allen, 1993).
In the absence of any planned variation in TQM activity, this research uses cross-sectional data analysis to explore the linkages between PennDOT's employee-involvement/process-improvement activity and four categories of potential effects including employee attitudes, work quality, labor productivity, and road condition as a measure of overall program effectiveness. The units of analysis are the 67 county-level highway maintenance organizations. Data extracted from a wide variety of information systems maintained by PennDOT were aggregated to the county level and organized into a separate, integrated database for this analysis. By incorporating data from multiple points in time for some measures, the research design builds a longitudinal perspective into the cross-sectional comparisons among the county maintenance units.
Going beyond simple bivariate associations, this analysis employs causal modeling, or path analysis, to gauge the impact of total quality management on overall performance through a fuller system of variables. Figure 1 presents the causal model that was proposed initially to represent the expected relationships among these variables. Briefly, the percentage of employees exposed to training regarding employee involvement and continuous process improvement was expected to positively influence both the number of hours employees were engaged in teams working on process analysis and improvement and on employees' attitudes, which would also vary positively with team hours. The quality of work performed by highway maintenance crews in a county would be expected to be positively influenced by training in employee involvement and continuous process improvement as well as team hours and employee attitudes. Quality of work would be expected to improve more than predictable from quality ratings in prior years.
Current or recent labor productivity in a county maintenance unit would be expected to vary positively with labor productivity in prior periods, and to be negatively affected in counties where more difficult climate conditions (represented by the freezing index) can impede maintenance activities. Beyond these effects, labor productivity was expected to be positively influenced by training in employee involvement and continuous process improvement, team hours, and employee attitudes. As initially specified, the model is unclear about the direction of work quality and labor productivity. Although improved quality should be expected to strengthen long-term productivity, careful attention to quality standards may slow down immediate labor productivity. Finally, highway condition is represented by the highway maintenance backlog in the model. The current maintenance backlog would be expected to be higher in counties with higher traffic volumes and higher freezing indexes, as well as larger maintenance backlogs in the past. However, the maintenance backlog as a measure of overall program effectiveness would be expected to be lower in counties with more training, more team activity, more positive employee attitudes, higher quality ratings, and higher levels of labor productivity.
Measures and Data Sources
Operational indicators representing the variables in the model were constructed from data drawn from a variety of record keeping and information systems maintained by the department. Where necessary they were standardized to control for different scales of operation among the 67 county maintenance units. Regarding TQM-related activity, from examining a database maintained by the department's training section it was possible to determine how many individuals in each county had received any training in the employee involvement and continuous process involvement modules through the end of 1994. These numbers were then used to compute the percentage of employees in each county with at least some training in these areas. While PennDOT has not tracked the number of quality circles or quality breakthrough teams that have been formed over the past several years, it has maintained an accounting of all employee hours assigned to participation in them. Thus, it was possible to compute a standardized measure of the number of team hours per employee from 1986 through March of 1995 for each county.
Regarding employee attitudes, a survey conducted for PennDOT in 1993 by a university-based research group provided data on several attitudinal scales, including those representing job satisfaction and the individual's commitment to the department (Landy et al., 1993). Scores on these scales were summarized for each organizational unit to indicate the percentage of responses that reflected a positive attitude, and these summary positive scores for the job satisfaction and commitment dimensions were combined into a single index for the analysis reported here. While it should be mentioned that the average number of responses from the county maintenance units was only 16, this attitudinal index would still be expected to correlate positively with TQM training and activity levels across the 67 county maintenance units.
With respect to actual service delivery outputs, this analysis utilizes a measure of the quality of work performed by highway maintenance crews. PennDOT has had a systematic quality assurance program in place for several years that entails field audits of maintenance operations. In these independent reviews, inspectors from the central office bureau of highway maintenance evaluate the quality of work completed at a sample of job sites in each county covering a variety of maintenance operations such as mechanized pothole patching, concrete pavement joint sealing, shoulder cutting, and drainage pipe replacement. These field audits, which are conducted on an unannounced schedule in order to sample typical jobs, focus on both the adherence to prescribed operating procedures and the quality of work completed. The results are summarized annually for each county unit, with average scores on a 6-point scale ranging from "unsatisfactory" to "excellent," and were available from 1990 through 1995.
Labor productivity is measured in this study as the amount of work accomplished per production hour worked. While the outputs of highway maintenance activities are measured in various units such as tons or gallons of resurfacing materials applied to the roadway, miles of shoulders stabilized, or feet of guiderail replaced, it is possible to convert these outputs into the generic measure of task hours earned. Data on 10 common "big ticket" maintenance functions were extracted from PennDOT's county-level management objectives report for fiscal 1987 and 1995, including manual patching, mechanized patching, liquid bituminous surface treatment, skin patching, crack sealing, scratch coating, shoulder grading, shoulder cutting, drainage ditch cleaning, and drainage pipe replacement. Standard times, which have been established by the department for accomplishing single units of output for each of these functions were applied to the volume of output produced to determine the task hours earned, which were then summed over the 10 functional areas. These total task hours earned were then divided by the number of production hours reported on those functions to yield a ratio of task hours accomplished to production hours worked. In this measure of immediate labor productivity values greater than one indicate that county maintenance crews are exceeding established standards, and values less than one mean that crews are failing short of those standards. This analysis utilizes the labor productivity measure for fiscal 1987, the first year that standard times were included in the management objectives report, and for fiscal 1995.
The overall effectiveness of a highway maintenance program should be assessed by the resulting condition of the roads being maintained. In this analysis, highway condition is represented by the existing backlog of unmet maintenance needs required to bring all the roads up to specified serviceable standards. PennDOT's pavement management system involves the annual inspection of approximately one-half of the roads in the state's system -- some 22,000 miles of highway each year-using both direct observation and mechanical measurements to examine the condition of shoulders and guiderails as well as the pavements themselves. Off-road drainage structures are inspected every three years. While this process is used to measure pavement quality and identify numerous specific conditions and deficiencies in highway facilities, the bottom line produced by this pavement management system is an estimated dollar cost of unmet maintenance needs. These unmet needs, standardized by the number of miles of highway in each county in a cost-per-mile ratio, are incorporated in this analysis for 1995, the most recent year for which data are available, with 1989 as a base year for purposes of comparison. This analysis also controls for environmental variables that impact on a county unit's ability to decrease its maintenance backlog. These variables include (1) traffic volume as measured by the number of vehicle miles traveled per mile of highway in the county, and (2) the freezing index, which runs from approximately 500 to 1,500. Higher values indicate more severe winter conditions, more "freeze/thaw" cycles, and thus more maintenance problems.
The descriptive statistics presented in Table 1 provide a composite picture of the total quality management and highway maintenance activity elements incorporated in the overall model. On the average, about 49 percent of the employees in a given county maintenance unit had received some training in employee-involvement/continuous-improvement processes by the latter part of 1994, but that varied widely from just 4 percent in one county to 100 percent in another. The total team hours per employee invested in quality circle and quality breakthrough team activities from 1986 through early 1995 showed extreme variation -- with a mean of 402 and a standard deviation of 444 -- ranging from less than a single hour in one county to more than 1,600 hours in another. It is worthwhile to note that the mean of 402 hours per employee over roughly a 10-year period indicates that on average PennDOT employees in the highway maintenance units spent approximately 40 hours -- or one work week -- per year involved in a quality circle or quality breakthrough team.
Measures Mean Percentage of employees with training 48.55 Team hours per employee, 1986-1995 401.87 Job satisfaction-commitment 123.03 1987 quality rating 12.01 1994 quality rating 12.75 1987 labor productivity 1.05 1994 labor productivity 1.24 Freezing index 929.91 1994 vehicle miles 3,957.48 1989 unmet maintenance needs per highway mile $28,575.87 1995 unmet maintenance needs per highway mile $20,874.75 Standard Measures Deviation Percentage of employees with training 32.37 Team hours per employee, 1986-1995 443.54 Job satisfaction-commitment 27.69 1987 quality rating 1.51 1994 quality rating 1.54 1987 labor productivity .24 1994 labor productivity .38 Freezing index 279.78 1994 vehicle miles 3,125.14 1989 unmet maintenance needs per highway mile $15,066.60 1995 unmet maintenance needs per highway mile $8,552.30
The index representing employee job satisfaction and commitment to the department ranges from 60 to 183 (out of a possible 200) with a mean of 123 and a standard deviation of 28. The mean average quality rating was 12.01 in 1987 and up to 12.75 in 1994. While the small standard deviations indicate relatively compact distributions of these two measures, there were some divergent cases, with one county rated at a 6.84 and another at the highest possible rating of 15.00 in 1994. The labor productivity measure averaged 1.24 in 1994, up from 1.05 in 1987, reflecting improvement on the order of 20 percent. This measure ranged from a county with only 0.7 earned task hours for each production hour worked, to a county at the other extreme that accomplished almost 3 task hours for each production hour worked.
The substantial variation in weather conditions that affect PennDOT units' ability to maintain the highways is reflected in the freezing index, which ranges from 228 to over 1,500 with a mean average of about 930. The daily vehicle miles travelled per mile of highway in 1994 also varied widely, ranging from under 800 to more than 18,000 with a mean of just under 4,000. The average maintenance backlog was $20,875 per mile in 1995, a significant decrease from $28,576 per mile in 1989. Furthermore, the differences among county units in unmet maintenance needs per mile shrank over this period, as reflected by the much smaller standard deviation in 1995. In 1989 unmet needs ranged from $8,088 to $80,965 per mile, while in 1995 they ranged from $8,772 to $43,491 per mile. The large drop in the high figure from 1989 to 1995 suggests that some of the counties with the greatest maintenance needs per mile in 1989 reduced their backlogs the most over the following six years. Figure 2 confirms that the greatest decrease in unmet needs per mile was registered by counties in districts 11 and 12, which also had the greatest maintenance backlogs in 1989.
Path coefficients connecting these elements were calibrated by developing regression equations for each measure hypothesized to be influenced by other variables in the model and examining the standardized beta coefficients of these regressions in terms of both direction and magnitude. Tests of statistical significance were not employed because the analysis being performed was on the population rather than a sample of county maintenance units. In order to discrimination weak association from negligible effects, independent variables with standardized beta values of less than .05 in absolute value were eliminated and the regression models were rerun without them. The beta values of the resulting regressions were then used as the best estimates of the path coefficients in the overall causal model.
The fully calibrated model is shown in Figure 3. As is readily apparent, many of the [R.sup.2] values, representing percentage of explained variation, are relatively small, but all the relationships are in the direction predicted by the program logic, and taken as a whole the model makes sense on substantive grounds. Most of the path coefficients are modest in size; this is in keeping with the fact that the elements contained in the model are likely to be influenced by a diverse array of forces and the expectation that TQM processes will generate incremental, rather than dramatic, impacts.
The model shows that the number of team hours per employee spent on quality circles and quality breakthrough teams is greater in county units where more employees have received some training in these areas. The model also shows that employees' job satisfaction and commitment to the department is positively influenced by both the percentage of trained employees and the number of team hours worked over the past 10 years. The number of team hours in turn has a favorable impact on both labor productivity and quality ratings, although the relationship is notably stronger with respect to quality. Beyond being highly consistent with quality ratings in 1990, the 1994 quality ratings are positively influenced by employee job satisfaction and commitment as well as team hours. Similarly, in addition to being in part a function of 1987 productivity levels and the freezing index, 1994 labor productivity is also favorably related to both team hours and the job satisfaction and commitment measure. The percentage of employees who received at least some TQM training is not related to either the quality of work or crew productivity beyond its indirect impact through increased team hours and stronger job satisfaction and commitment.
Finally, unmet highway maintenance needs in 1995 are shown to be greater in counties with higher freezing indexes and/or more daily vehicle miles travelled per mile of highway to be maintained, as well as being positively related to 1987 maintenance backlogs. Beyond these effects, unmet maintenance needs per mile "tend to be lower in countles with higher quality ratings and fighter labor productivity rates, as expected, as well as those county units with more hours invested in quality circles and quality breakthrough teams and countles with higher levels of job satisfaction and commitment. Thus, hours invested in these TQM-type activities appear to lead to improved employee attitudes, improved quality, and improved productivity, which all lead to a more effective highway maintenance program as represented by lower unmet maintenance backlogs.
Assuming that the proposed model is appropriate in terms of actual causal relationships, Table 2 compartmentalizes the original correlations into direct and indirect causal coefficients and noncausal associations between each relevant pair of variables in the model. Of greatest interest are the relationships of team hours, employee attitudes, quality ratings, and labor productivity with unmet maintenance needs. For example, while the direct path coefficient between team hours and unmet needs is -.17, the indirect causal connection running through the quality and productivity measures is an additional -.05, resulting in a total causal correlation of -.22. This is less than the original correlation of -.33, suggesting that the difference of -.11 represents a degree of spurious association resulting from negative relationships between team hours and vehicle miles travelled and/or 1989 unmet maintenance needs.
The causal coefficient between the job satisfaction and commitment measure and unmet needs is also -.22, while both quality rating and the productivity measure have a total causal coefficient with unmet needs on the order of -.15. Two of the original correlations among the variables in the model appear to be distorted. For example, the original correlation between team hours and 1995 labor productivity is negative (-.15) while the causal relationship is positive (.05), suggesting a very weak tendency for productivity to increase as more hours are invested in team activity once the freezing index and 1987 labor productivity levels are taken into account. Generally speaking, these causal coefficients indicate the nature and degree of relationship between these measures, beyond the effects of other variables in the system, assuming that the model fits reality in terms of incorporating the relevant variables and specifying the direction of causality among them.
Because the path coefficients are based on standardized beta values, they can be interpreted in terms of the different measurement scales involved by applying them to the standard deviations of the variables in question. As shown in Table 3, the standard deviation of the team hours per employee is 444, while the standard deviation of the 1995 unmet maintenance needs is $8,552. The total causal coefficient representing the impact of team hours on unmet maintenance needs is -.22, meaning that on the average an increase of 444 team hours per employee would bring about a decrease of $8,552 in maintenance needs per mile times -.22, or $1,881. Thus, 444 team hours per employee is associated with a $1,881 reduction in 1995 maintenance needs per mile. Scaling the relationship down to the unit level, an increase of one more team hour per employee is associated with a reduction of $4.25 in 1995 maintenance needs per mile.
Table 3 Causal Interpretation
Measure Standard Deviation Team hours per employee, 1986-1995 444 Unmet maintenance needs per mile, 1995 $8,552
Total causal coefficient, Team Hours Per Employee/Maintenance Needs Per Mile: -.22 $8,552*-.22=-$1,881
444 team hours per employee: $1,881 reduction in 1995 maintenance needs per mile
1 team hour per employee: $4.25 reduction in 1995 maintenance needs per mile
Applying this relationship to the actual scale of operation in PennDOT's highway maintenance program yields estimates of the reduction in unmet needs generated by total quality management activity, which can then be compared to the estimated cost of these activities. Table 4 shows both mean averages and sums across the 67 county maintenance units for the number of employees, training hours, team activity hours, highway miles, and 1995 maintenance needs. Looking at the means, the typical or average county has 98 employees, 798 hours of TQM training, 37,810 hours of team activity, 638 miles of state highways to maintain, and $14,230,000 in unmet needs. If one team hour per employee is associated with a reduction of $4.25 per mile in maintenance needs, then in the typical county this savings would amount to $1,046,140. This is computed as follows:
Measure Mean Sum Employees 98 6,595 Training Hours 798 53,494 Team Hours 37,810 2,533,279 Highway Miles 638 42,716
Typical County Impact Maintenance needs savings = (37,810/98) * $4.25 * 638 = $1,046,140 Training and team activity cost = ($25 * 798) + ($20 * 37,810) = $776,150 Implied benefit/cost ratio = 1.35
Statewide Impact Maintenance needs savings = (2,533,279/6,595) * $4.25 * 42,716 =
$69,734,506 Training and team activity cost = ($25 * 53,494) + ($20 * 2,533,279) =
$52,002,930 Implied benefit/cost ratio = 1.35
(37,810/98) team hours
per employee x $4.25 x 638 miles.
This savings is equivalent to approximately 7 percent of the backlog of maintenance needs in the average county.
Assuming training costs of $25 per participant hour (including program costs as well as the cost of the participants' time), and assuming a cost of $20 per hour spent in quality circle or quality breakthrough team activity, the total estimated cost of this training and team activity in the typical county would be $776,150. Thus, the $1,046,140 savings in the form of maintenance backlog reduction would outweigh these program costs with a benefit/cost ratio of 1.35. Similarly, for the department as a whole, the estimated reduction in maintenance needs would be $69,734,506, computed at a savings of $4.25 for each of 42,279 highway miles for each team hour per employee (2,533,279/6,595). Compared to the estimated training and team activity cost of $52,002,930, the statewide benefit/cost ratio for PennDOT's TQM activity in highway maintenance would be the same, 1.35.
Thus, while the statistical correlations between team activity hours and county units' quality ratings, labor productivity measures, and maintenance backlogs are quite modest, the overall impact of these TQM-related activities appears to be significant with benefits in the form of maintenance backlog reductions exceeding program costs by more than a third. While PennDOT occasionally has documented specific cost savings and operational improvements produced by individual quality circle and quality breakthrough team projects, these effects are often scattered. Heretofore, the department has not been able to estimate the aggregate impact of its TQM activities on the effectiveness of the highway maintenance program. The results generated by this analysis suggest that the accumulation of incremental benefits of PennDOT's TQM processes over the years have indeed made a substantial positive impact on the bottom line of service delivery in its largest operating program.
The causal modeling technique employed here is essentially a heuristic tool that postulates a model of the causal relationships among a system of variables and then uses statistical analysis to estimate the magnitude of these "causal" impacts, assuming that the model provides an appropriate fit to reality in the first lace. Given the passive nature of the nonexperimental research design used in this research, the results reported here are obviously open to alternative explanations. Although the analysis is strengthened by the ability to incorporate several of the measures at more than one point in time, the lack of experimental controls precludes definitive conclusions regarding the real impact of PennDOT's total quality management activities on service delivery.
The principal threat to validity with this cross-sectional research design is selection bias. Especially in the early years, TQM approaches were promoted within the department on a voluntary basis. Skeptics pointed out that those counties opting to emphasize TQM training and team activities might be well on their way to improving quality and productivity and reducing their maintenance backlogs already. Thus, the positive correlations resulting from this analysis might not necessarily reflect a true causal impact of TQM activity on service delivery.
On the other hand, measurement limitations in this analysis may actually underestimate impacts. For example, the ratio of task hours accomplished to production hours worked is a somewhat narrow definition of labor productivity because it fails to consider the degree to which the total hours available for employees to work are used in a productive fashion. If quality teams had found ways to reduce downtime through improved equipment maintenance or better work scheduling, for example, this would have increased the percentage of total hours actually used for production work without effecting the measure of task hours per production hour. Similarly, the quality ratings used in this analysis provide a limited measure of service delivery quality because they are heavily based on adherence to standard operating procedures. If, as is most certainly the case, quality teams focused more on improving these procedures to produce longer lasting work, on applying more effective maintenance treatments to certain kinds of problems, and on stressing preventive maintenance activities, the effects of their efforts would not show up significantly in the quality ratings relied on here but might well be reflected in improved road condition and reduced maintenance backlogs. Indeed, this might explain why team hours impact directly on unmet maintenance needs more than indirectly through quality and productivity. Thus, the measures of both quality and productivity that were available for this analysis may have produced understated results regarding the impact of TQM activities on actual quality, productivity, and overall effectiveness.
This research was designed to gauge the impact of TQM activities on service delivery in a product-oriented government organization. Most likely, the results should be taken at face value. While based on fairly modest statistical associations, the findings suggest that PennDOT's TQM activities in the highway maintenance area are associated with a 7 percent improvement in overall effectiveness over the long run, with benefits exceeding costs by 35 percent. Thus, the results reported here are consistent with the expectation of incremental impacts accumulating over time and provide corroborating evidence that in addition to improvement in organization culture and processes themselves, TQM activities can impact favorably on the bottom line of service delivery. Ultimately, of course, the effectiveness of TQM should be measured by the extent to which it enhances customer satisfaction with services provided. The linkage between Penndot's TQM activities and customer satisfaction with the highways it maintains, gauged by a large-scale survey of Pennsylvania motorists, will be examined in the next phase of the research reported here.
Berman, Evan M., and Jonathan P. West (1995). "Municipal Commitment to Total Quality Management: A Survey of Recent Progress." Public Administration Review 55 (1): 57-66.
Bowman, James S., and Barbara J. French (1992). "Quality Improvement in a State Agency Revisited." Public Productivity & Management Review 16: 53-64.
Carr, David K., and Ian D. Littman (1990). Excellence in Government: Total Quality Management in tbe 1990s. Arlington, VA: Coopers & Lybrand.
Cohen, Steven, and Roger Brand (1993). Total Quality Management in Government: A Practical Guide for the Real World San Francisco: Jossey-Bass.
Cox, Raymond W., III (1995). "Getting Past the Hype: Issues in Starting a Public Sector TQM Program." Public Administration Quarterly 19: 89-103.
Davenport, Thomas H. (1994). "Managing in the New World of Process." Public Productivity & Management Review 18: 133-147.
Harris, Richard H. (1990). "Five Roads to Excellence." Journal for Quality and Participation 13: 54-58.
Hyde, Albert C. (1992). "The Proverbs of Total Quality Management: Recharting the Path to Quality Improvement in the Public Sector." Public Productivity & Management Review 16: 25-37.
_____ (1995). "Quality, Reengineering, and Performance: Managing Change in the Public Sector." In Arie Halachmi and Geert Bouchaert, eds., The Enduring Challenges in Public Management: Surviving and Excelling in a Changing World. San Francisco: Jossey-Bass.
Hyman, William A., Roemer M. Alfelor, and Joan A. Allen (1993). Service Efforts and Accomplishments Reporting: Its Time Has Come: Highway Maintenance. Washington, DC: Governmental Accounting Standards Board.
Kline, Robert M., and Noreen C. Young (1994). "Our Road to Quality Is Paved with Employee Involvement." Journal for Quality and Participation 17: 20-24.
Kravchuk, Robert S., and Robert Leighton (1993). "Implementing Total Quality Management in the United States." Public Productivity & Management Review 17: 71-82.
Landy, Frank L., Robert J. Vance, P. E. Tesluc, and John M. Stevens (1993). A Critical Assessment of PennDOT's Employee Involvement Processes. University Park, PA: Center for Applied Behavioral Sciences, Pennsylvania State University.
Mahler, Julianne G. (1995). "Evolution of a Quality Management Program." Public Productivity & Management Review 18: 387-396.
Mani, Bonnie G. (1995). "Old Wine in New Bottles Tastes Better: A Case Study of TQM Implementation in the IRS." Public Administration Review 55(2): 147-158.
McNabb, David E., and Thomas F. Sepic (1995). "Culture, Climate, and Total Quality Management: Measuring Readiness for Change." Public Productivity & Management Review 18: 369-385.
Milakovich, Michael E. (1990). "Total Quality Management for Public Sector Productivity Improvement." Public Productivity & Management Review 14: 19-32.
National Governors' Association (1992). Total Quality Management Initiatives in State Governments. Washington, DC: National Governors' Association.
Piel, Robert (1991). "So What Is an Employee Involvement Coordinator?" Journal for Quality and Participation 14: 42-44.
Poister, Theodore H., and Richard H. Harris (1996). "Service Delivery Impacts of TQM: A Preliminary Investigation." Public Productivity & Management Review 20: 84-100.
Poister, Theodore H., and Thomas D. Larson (1988). "The Revitalization of PennDOT: A Case Study in Effective Public Management." Public Productivity & Management Review 10: 85-103.
Rago, William V. (1996). "Struggles in Transformation: A Study in TQM, Leadership, and Organizational Culture in a Government Agency." Public Administration Review 56(3): 227-234.
Sensenbrenner, Joseph (1991). "Quality Comes to City Hall." Harvard Business Review 69: 64-75.
Swiss, James E. (1992). "Adapting Total Quality Management (TQM) to Government." Public Administration Review 52(4): 356-362.
U.S. General Accounting Office (1992). Quality Management: Survey of Federal Organizations. Report No. GGD-93-9BR. Washington, DC: U.S. General Accounting Office.
Wilson, Laura A., and Robert D. Durant (1994). "Evaluating TQM: The Case for a Theory Driven Approach." Public Administration Review 54(2): 137-146.
Theodore H. Poister is professor of public administration at Georgia State University. He has published widely in the areas of public management and evaluation research and has particular interests in the field of transportation policy and management. Poister has conducted applied research and consulting projects for numerous public and nonprofit organizations, including the Penpsylvania Department of Transportation.
Richard H. Harris is director of the Operations Review Group at the Pennsylvania Department of Transportation, where he has also served as an assistant highway maintenance manager. He has private-sector management experience in the trucking industry and is a colonel in the Air Force Reserve. Harris chairs the Governor's Total Quality Management Advisory Council in Pennsylvania and is active nationally in the quality and employee-involvement movement.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||total quality management|
|Author:||Poister, Theodore H.; Harris, Richard H.|
|Publication:||Public Administration Review|
|Date:||Jul 1, 1997|
|Previous Article:||Regulatory enforcement: accommodation and conflict in four states.|
|Next Article:||Economic espionage: security missions redefined.|