Printer Friendly

Evaluating Social Programs at the State and Local Level: The JTPA Evaluation Design Project.

Evaluating Social Programs at the State and Local Level: The JTPA Evaluation Design Project. Edited by Ann Bonar Blalock. (W. E. Upjohn Institute for Employment Research, 1990), 410 pages. Reviewer: Kathleen Hunter Sloan, Ph.D., Associate Professor of Public Administration, University of Hartford, Barney School of Business and Public Administration

The efficiency and effectiveness of government social programs in relation to their legislative purposes have long been a concern not only to program managers but also to accountable public policy decision-makers and tax-paying members of their constituencies. Social programs created to address the goals of combatting unemployment and underemployment among the less skilled and more socially-disadvantaged groups have been raised questions of efficiency and effectiveness. The disturbing funding, coverage, and administrative problems in state unemployment compensation programs, established as part of the national economic security system under the Social Security Act of 1935, present widespread problems.(1) In the light of changing employment opportunities in the labor force in the United States, books that attempt to improve analytical capabilities for judging the effectiveness of such programs merit serious attention. Evaluating Social Programs at the State and Local Level: The JTPA Evaluation Design Project is such a book.

The ambitious goals of a number of new social programs established by Congress in the 1960's and 1970's stimulated social scientists' efforts to develop the field of evaluation research. The federal government supported the need for more objective program evaluation and funded the cost of the mandated evaluation of a number of employment-related programs. During the 1980's, as policy-makers shifted program administrative responsibility from the federal to state and local government levels, they also shifted the mandatory responsibility for program evaluation to those governmental levels. It is the need for state and local governments to undertake comprehensive and objective program evaluation that led to the Washington State Employment Security Division's initiation of the JTPA Evaluation Design Project and, subsequently, to the publication of this book.

Ann Bonar Blalock, the editor and one of the contributing authors of the book, identifies its purpose as the encouragement of program evaluation as a program management and policy tool for those officials in state and local government who are responsible for implementation, operation, and evaluation of social programs. The book resulted from her participation, and that of the other authors, in the JTPA Evaluation Design Project, which was developed to evaluate the State of Washington's job training programs under the Job Training and Partnership Act passed by Congress in 1982. The JTPA replaced the Comprehensive Employment and Training Act (CETA) program, which had created public employment training by local governments. The new act stressed the involvement of the private sector in job training and gave state governments the responsibility for program design, management, and evaluation.

The Job Training Partnership Act received widespread support because of the intended involvement of the private sector at state and local levels in a "privatization" effort. The JTPA called for the establishment of Private Industry Councils (PICS) to work with state and local government officials in an attempt to both identify and remove barriers to job training for the target population of disadvantaged youths and adults and of dislocated workers. The new policy directions of the JTPA also created a need for careful assessment and evaluation of the program as implemented in the states.

The book is divided into three major parts: the first consists of an introductory chapter, in which the editor discusses a general approach to program evaluation and sets forth general principles and approaches to the topic, rather than describe the JTPA evaluation specifically.(2) The second part, "Complementary Approaches to Evaluating Programs," contains three quite detailed and informative chapters on "Evaluating Net Program Impact,"(3) "Evaluating Gross Program Outcomes,"(4) and "Evaluating Program Implementation."(5) In each chapter the authors discuss the conceptual framework for the type of evaluation and the methodological issues involved, and then use the JTPA as an example of applying such concepts and methodologies. The third part, "Organizational and Political Issues in Evaluating Programs," contains two chapters: "User-Centered Evaluation Planning"(6) and "Strategies for Supporting Comprehensive Evaluations,"(7) in which the authors discuss planning, evaluation strategy, and information dissemination and use.

The chapters on the conceptual and methodological basis for evaluation of net program impact and gross program outcomes are particularly well-detailed, and they discuss their respective topics in a thorough and comprehensive manner. The distinction between applications of net impact and gross impact analyses to the JTPA evaluation and their complementary aspects is clearly delineated. Noting that while "both models are designed to address program effectiveness questions," the authors point out that they differ "in terms of the types of evaluation questions that can be answered, the range of outcome measures of interest, and the types of comparisons being made."(8) For example, net impact analysis is limited to adults only and relies exclusively on administrative data sources; thus it is limited in both the number of personal characteristics included and in the small number of outcome measures to examine. Gross impact analysis, in contrast, "can be used to address certain relative effectiveness questions for youths served by the JTPA" -- in part, because it can include "an expanded set of labor market outcomes" and add more personal characteristics of program participants.

The conceptual approaches and the methodologies discussed may be of greatest interest to JRI readers, and those concerned with the possibilities of research-based evaluation should find this book particularly useful. The authors clearly are knowledgeable, and their experience in development of appropriate methodologies, and data sources is evident. Disappointing, however, is their method of using the JTPA evaluation as a case example, since they limit their examination of the program to its design and fail to provide any examples of application of the design and the specific methodologies to the results of the program in Washington state.

Since other analyses of implementing the JTPA indicate that few of the national goals or standards established in the legislation have been met, the lack in this book of applications that would help in analyzing the outcomes and processes of a well-designed evaluation program's effectiveness diminishes its usefulness for the academic reader. Nuckols, for example, contends that the result of the JTPA, after three years' implementation, is that it has not only reduced training funds but also shifted "resources from those most in need of training to those most likely to attain employment without training."(9) It is the apparent lack of awareness that implementation of this specific program may not be producing intended results that limits the usefulness of this book, since the underlying purpose of evaluation studies is to provide informed guidance to assessment of specific programs.

This book, essentially, is a succinct conceptual and methodological guide to program evaluation in general; notwithstanding this restricted focus it is useful. Although it provides brief descriptions of the legislative framework for overall design of the JTPA program, the reader should not look for evidence of the program's effectiveness, whether in the state of Washington or elsewhere, nor expect a substantive discussion of JTPA in relation to other programs, such as those that some states have developed to provide enhanced assistance and training to unemployment insurance claimants. In short, the usefulness of this work lies in the review and discussion of general evaluation design, not in the analysis of JTPA or in any comparison with similar programs intending to overcome the problems of unemployment and underemployment.

(1)See e.g., George E. Rejda and Kyung W. Lee, "State Unemployment Compensation Programs: Immediate Reforms Needed," Journal of Risk and Insurance, L VI, No. 4, 1989, 649-669; Diana Runner, "Changes in unemployment insurance legislation during 1989," Monthly Labor Review, January 1990, 64-69. (2)"Evaluating Programs," 3-40. (3)Terry R. Johnson and Ernest W. Stromsdorfer, 43-130. (4)Carl Simpson, 133-227. (5)David Grembowski, with Ann Bonar Blalock, 229-297. (6)Deborah Feldman, 301-351. (7)Ann Bonar Blalock, 353-372. (8)p. 121. (9)Dan Nuckols, "Public/Private Partnerships As Implementing Strategy: The Job Training Partnership Act," Journal of Economic Issues, XXIV, No. 2, 1990, 645-651, 649.
COPYRIGHT 1991 American Risk and Insurance Association, Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1991 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Sloan, Kathleen Hunter
Publication:Journal of Risk and Insurance
Article Type:Book Review
Date:Jun 1, 1991
Previous Article:Personal Risk Management and Insurance.
Next Article:The Economics of Safety and Physical Risk.

Terms of use | Copyright © 2017 Farlex, Inc. | Feedback | For webmasters