Clinical psychology program improvement on the Examination for Professional Practice in Psychology.

The present study was conducted in the context of previous research on the validity, correlates, and stability over time of clinical psychology program graduate scores on the national licensing exam, the Examination for Professional Practice in Psychology (EPPP). The purpose of the present study was to determine the characteristics of programs that demonstrated great improvement on the EPPP. Nineteen clinical programs that had dramatic improvement from 1988-1991 to 1997-1998 on mean EPPP scores were identified. Letters were sent to departmental chairs for their explanation for the increases. The themes of greater scientific rigor and selection of better quality students were salient.

**********

The examination for Professional Practice in Psychology (EPPP) is the national licensing examination that is used in almost all U.S. states and Canadian provinces. It has demonstrated validity, e.g. graduates of regionally accredited programs obtaining higher mean scores than those of regionally unaccredited programs (Templer, Tomeo, Harville, & Pointkowski, 2000). Clinical psychology programs whose graduates score higher on the EPPP have higher admissions standards, a higher ratio of faculty to graduate students, greater research orientation, and approval of the American Psychological Association (Yu, Renaldi, Templer, Colbert, Siscoe & Van Patten, 1997; Templer & Tomeo, 1998).

The relative EPPP scores of clinical psychology program graduates has stability over time. Templer, Couture, Martinez, and Tomeo (1999) reported a correlation of .80 between mean EPPP score in 1988-1995 of graduates of clinical psychology programs and mean scores of these programs in 1997.

The purposes of the present study were to (1) identify clinical psychology programs that demonstrated dramatic improvement over time; and (2) to explore possible explanations for these improvements.

Method

The means and ranks of the 154 clinical psychology programs that had EPPP mean scores for both the 1988-1991 (Educational Reporting Service, 1992) and the 19971998 (Educational Reporting Service, 1999) time periods were determined. A six-year interval was employed because six years constitute approximately a "generation" of graduate students.

Table I contains the 19 most improved programs. The criterion originally used was N of at least 6 for both time periods and an increase in 31 or more percentile points. There were, however, two programs that did not meet the N criterion but were included because of dramatic improvement. Bryn Mawr went from the 18th to the 83rd percentile. University of Pennsylvania went from the 43rd to the 99th percentile. The order of Table I was randomly determined rather than representing degrees of improvement.

Table I and a letter describing the purpose of the study were sent to all 19 departmental chairs. The letter said, "I would be most appreciative if you could, in the bottom half of this letter, give your opinion about the reason(s) for this dramatic increases and return the letter in the stamped envelope provided. I am aware of the fact that a few of the program's Ns are very small so that chance fluctuations explain some of the improvements. However, if impressions based on those small Ns mesh with those based on larger Ns, their credibility will be strengthened."

Results

Of the 19 letters sent out, there were responses from 12 programs, usually from either the departmental chair or the director of clinical training. One respondent stated she did not know how to explain the improvement. Another respondent indicated that no conclusions are warranted because the findings are an artifact of the small N. A third respondent indicated that the number must be incorrect because their program graduates fewer students than the numbers seem to indicate. The third comment may be, at least in part, a function of the present authors neglecting to say in their cover letter that the number pertains to number of exams taken rather than number of graduates taking the exam. Most persons who fail the EPPP probably take it again. The present authors accept the above comments with the same seriousness and respect accorded to the comments giving reasons for the increase. These comments are:

1. Institution of in-house training clinic (previous supervision by community supervisors). Orientation toward empirically validated treatments and structured diagnostic interviewing. More thorough competency exam.

2. Since now APA accredited, accepting stronger applicants. Hired stronger faculty. Stronger curriculum with larger number of required courses. Cover fundamentals. Integrative comprehensive exam.

3. Proseminar taught by experimental psychology faculty. Additional course in multivariate statistics. Recruitment of students with research experience. Slight emphasis toward science end of scientist-practitioner model.

4. Small number of students in programs. Smart students.

5. Better selection ratio. Admitting more science oriented students. More science based training in clinical courses. Increased research emphasis.

6. Better quality students. Faculty hiring in key areas. Curriculum improvements. Adding or strengthening courses in ethics and psychopathology and quantitative methods.

7. Students taught better because of three new faculty and new director of training (Respondent noted that the new director of training had previously been at another top 19-improvement program).

8. Better admission standards (GRE & GPA). EPPP directed teaching. Students encouraged to get EPPP study material and practice tests.

9. Accepting higher quality students. More explicit training in cultural diversity and methodology and statistics. Teach ability to think clearly. High scientific standards.

Discussion

The most dominant theme in the explanations of the departmental chairs and directors of clinical training that responded is that of increased scientific rigor. There is repeated mention of such changes as greater research orientation, more scientific training, higher scientific standards, more required coursework, hiring stronger faculty, more courses in statistics and methodology, teaching ability to think clearly, stress on fundamentals, orientation toward empirically validated treatment and structured diagnostic interviewing, and more thorough competency exams.

Another dominant and related theme pertains to student quality. EPPP improvement was attributed to more capable students, admitting more science oriented students, recruitment of students with research experience, and admitting a fewer number of students.

It must be borne in mind that the above stated reasons for improvement are opinions rather than research demonstrated reasons for improvement. Nevertheless, the composite of opinions has internal consistency and is congruent with the above cited research showing programs with greater scientific orientation, more stringent admissions requirements, and smaller graduate students to faculty ratios, tend to have higher mean EPPP scores. As viewed from a research perspective, the present research should be viewed as a pilot study that has generated plausible hypotheses. Unfortunately, there is apparently no previous literature on EPPP improvement to form a basis for decisions. From the perspective of faculty wishing to improve the EPPP scores of their clinical psychology graduates, it appears that they should increase the scientific rigor of the program and increase the quality of the students admitted.

Table 1
EPPP Program Ranks and Percentiles of 19 Most Improved
Programs, as Sent to Departmental Chairs

                      1998-1991

School                Number of   Rank    Percentile
                      Students

U. of Colorado           37       125.5       19
at Boulder

U. of Utah               28        102        34

Columbia U.-             45        137        11
Teachers' College

Vanderbilt U.            28       83.5        46

St. John's U.            50        108        30

Fairleigh Dickinson      39        98         36
University

Purdue University        23       94.5        39

U. of Miami              52       94.5        39

VA PolyTech              33       106.5       31
Institute

Bryn Mawr                15        127        18
College

U. of Georgia            43        101        34

Duke U.                  16       60.5        61

Case Western             24        77         50
Reserve University

U. of Windsor            26       79.5        48

U. of Pennsylvania        5       88.5        43

                      1997-1998

School                Number of   Rank   Percentile
                      Students

U. of Colorado           16        46        70
at Boulder

U. of Utah               10        51        67

Columbia U.-             33        79        49
Teachers' College

Vanderbilt U.             9        29        81

St. John's U.            48        31        80

Fairleigh Dickinson      19        34        78
University

Purdue University         9        35        77

U. of Miami              24        36        77

VA PolyTech              13        23        85
Institute

Bryn Mawr                 5       25.5       83
College

U. of Georgia            14        18        88

Duke U.                  11        13        92

Case Western             12       16.5       89
Reserve University

U. of Windsor            15       11.5       93

U. of Pennsylvania        4        2         99

References

Educational Reporting Service (1992). EPPP performance by designated doctoral program in psychology. New York: Author.

Educational Reporting Service (1999). EPPP performance by designated doctoral program in psychology. New York: Author.

Templer, D. 1. & Tomeo, M. E. (1998). Examination For Professional Practice in Psychology: Clerical errors corrected but inferences unchanged. Psychological Science, 9,83-84.

Templer, D. I., Couture, J., Martinez, M. S., 8,: Tomeo, M. E. (1999). Mean score stability over time of clinical psychology programs on the Examination for Professional Practice in Psychology. Perceptual and Motor Skills, 88, 1217-1218.

Templer, D. I., Tomeo, M., Harville, M. & Pointkowski, S. (2000). California psychology licensing exam pass rates of graduates of accredited and unaccredited programs. Journal of Instructional Psychology, 27, 134-135.

Yu, L. M., Rinaldi, S. A., Templer, D. I., Colbert, L., Siscoe, K., Van Patten, K. (1997). EPPP scores as a function of clinical psychology graduate program. Psychological Science, 8, 347-350.

Donald I. Templer, Lori Tyler, Adam A. Nelson, Scott Winstanley and Cammy Chicota, California School of Professional Psychology at Alliant Internationl University--Fresno.

Correspondence concerning this article should be addressed to Dr. Donald I. Templar, California School of Professional Psychology at Alliant Internationl University--Fresno.