Assessing drug abuse programs: benefits from partnering with researchers.

Authors' Note: Points of view expressed in this article do not represent the official position or policies of the U.S. Department of Justice.

More and more, policy-makers are persuaded by data-driven results. Questions about a program's performance are a routine part of budget hearings at all levels of government, and funding decisions are routinely conditioned on evaluations and outcome data. When public officials are armed with findings based on scientifically sound, empirically valid information, they are better able to demonstrate accountability to legislators and their constituents.

Most departments of correction lack the money and staff to conduct research and evaluations of their programs. But some department administrators are nevertheless getting the information they need by teaming up with outsiders. For example, when administrators for the Pennsylvania Department of Corrections wanted an objective, systematic review of the effectiveness of their drug and alcohol treatment programs, they turned to researchers at Temple University because partnering with outside researchers gives them a host of benefits. (1)

Benefits of Outside Researchers

Partnering with outside researchers brings the Pennsylvania DOC greater credibility and firmer ground to stand on when it wants to make changes. In-house evaluations (those conducted by the department) can sometimes be criticized as self-serving and constrained by internal politics. An in-house evaluator, for example, may feel unable to speak critically. On the other hand, an outside reviewer brings a fresh eye to the department. Gary Zajac, the manager of research and evaluation at the Pennsylvania DOC, believes independent researchers are more likely to "tell it like it is."

Another advantage for Pennsylvania in partnering with a university involved managing money that came from an award given by the National Institute of Justice to Temple University. Temple managed the funds. This arrangement freed the DOC from the complications of the state contracting processes, which for most states can be a considerable burden. The department avoided writing a statement of work, issuing a contract, and then monitoring the contract and doing all the paperwork required when a public agency contracts for services.

Basically, the Pennsylvania DOC studies and evaluates its programs in partnership with outside researchers because doing so gives the department:

* Objective, credible information about program performance;

* Expertise not available in-house;

* No-cost research that assesses programs and tailors programs so they fit inmates' needs better; and

* Critical feedback on performance.

And all of these benefits come with the ability to participate in planning the evaluation and analyzing the findings.

For the research partner, teaming with a correctional department provides the following benefits:

* The opportunity to conduct research of interest in a real-world setting;

* The opportunity to make a difference in a public agency;

* The opportunity to gather original data and source material for authoring scholarly publications;

* The support of practitioners and policy-makers for research (This support usually results in stronger proposals and hence a stronger likelihood of being funded.); and

* Opportunities to train students, especially students in advanced degree programs.

Guiding the Partnership

The department's partnership with Temple University is based largely on the DOC's needs and is geared to producing knowledge the department can use to improve its treatment programs. However, the partnership also examines questions of critical importance to the research community such as what kinds of treatment work best for which kinds of inmates and under which conditions?

When the partnership was formed, a 14-member advisory committee was created to guide the process. Committee members included corrections policy-makers, data systems specialists, clinicians and treatment specialists, and researchers from Temple. In developing the project, DOC officials and staff and Temple researchers have had equal say. Together, they define the partnership's goals and oversee all aspects of the research.

The advisory committee is ultimately responsible for the direction of the project. It makes major decisions about the project, authorizes major actions needed to facilitate the evaluation, monitors progress, reviews findings and makes recommendations to policy-makers about how best to use the findings to make positive changes in the programs.

The advisory committee's role is quite active in the early stages of a project: think through the overall plan of action, develop a preliminary research design in consultation with the research partners, identify a funding source and participate in the grant writing process.

Once the grant is awarded, there is a great deal of startup work to do. Inevitably, something in the proposed plan needs to be adjusted. Zajac calls these adjustments "speed bumps." Because evaluations assess human behavior and because human behavior is unpredictable at times, research cannot always precisely follow the proposed plan. For example, when the DOC and Temple University partners wanted to assign inmates to experimental and control groups so they could conduct a well-designed randomized experiment, they found that experienced staff resisted assigning certain inmates to the control group because they intuitively felt that these inmates would benefit from being in the experimental treatment group. Handling these types of human factors is a typical challenge and one the partners worked hard to resolve through honest, open debate and discussion.

Building Knowledge

In broad terms, the aim of the partnership was to create high-quality drug and alcohol treatment programs. One of the first projects was to develop a database about the drug and alcohol programs in all 24 of the DOC's facilities (now 26). The team gathered data about program content, structure, clients and staffing and examined such program components as client eligibility and selection, nature and amount of services provided. Altogether, 118 programs of four types were covered.

After the descriptive assessment was complete, the researchers conducted a more in-depth process evaluation of treatment programs at several state correctional institutions. A subsequent study used a quasi-experimental design to study therapeutic community outcomes at five state prisons. (2) Although follow-up periods were relatively brief (less than two years), therapeutic communities significantly reduced recidivism by 11 percent compared with the control group. Finally, in a third study, the partners selected a specialized treatment prison where they would conduct a rigorous, randomized outcome evaluation of therapeutic community drug treatment. The evaluation of the effectiveness of the alcohol and drug treatment services at SCI-Chester is now under way and findings are expected in 2005.

The Pennsylvania DOC's partnership with researchers expands its capacity for in-house research and evaluation and gives the department the ability to make decisions and develop policies that are better driven by readily available data. The department has strengthened its commitment to revise some of its drug treatment programs and policies, adopting greater program standardization and giving more attention to screening and assessment. Perhaps most important, the partnership has resulted in an active research agenda that has led to successful proposals to conduct more in-depth outcome evaluations. (3)

There may be no better testimonial to the success of the partnership than the opinions of the steering committee members themselves, who the researchers interviewed at the end of the partnership's first year. The committee members believed the partnership was focused and productive. They agreed that everyone had a chance to participate, and they saw that progress was being made toward achieving the partnership's goals. Perhaps most important, they felt that the partnership enhanced the capacity of the Pennsylvania DOC to conduct its own evaluation research.

(1) The report, Building an Effective Research Collaboration Between the Center for Public Policy at Temple University and the Pennsylvania Department of Corrections: Final Report, 2002, by Wayne N. Welsh, is available for free downloading at www.ncjrs.org/pdffiles1/nij/grants/197067.pdf.

(2) Available online at www.ncjrs.org/pdf files1/nij/grants/197058.pdf.

(3) The NIJ-funded outcome evaluation of therapeutic community programs at five DOC institutions found that this type of treatment produced lower rates of rear-rest, reincarceration and drug relapse, and better psychosocial functioning. The final report, Evaluation of Prison Based Drug Treatment in Pennsylvania: A Research Collaboration Between the Pennsylvania Department of Corrections and the Center for Public Policy at Temple University, 2002, by Wayne N. Welsh, is available for free downloading at www.icpsr.umich.edu: 8080/NACJD-STUDY/03540.xml.

RELATED ARTICLE: Research Projects at Pennsylvania Prisons

The Pennsylvania Department of Corrections, which houses approximately 40,000 inmates at any given time, has several current and recent major research partnerships:

* Temple University is evaluating alcohol and other drug programs.

* A researcher affiliated with the Correctional Education Association has conducted process and outcome evaluations of the department's educational programs and is conducting an outcome evaluation of the department's community orientation and reintegration program.

* Pennsylvania State University has completed process and outcome evaluations of parenting programs.

* The University of Cincinnati has evaluated the Quehanna boot camp, the young adult offender program and other programs.

* LaSalle University has evaluated young adult offender programs.

* The Urban Institute conducted a process evaluation of the community orientation and reintegration program.

* Vera Institute of Justice has conducted process and outcome evaluations of the residential substance abuse treatment (RSAT) program.

By National Institute of Justice Staff