An analysis of doctoral programs of health education.

Abstract: This study describes the data used to rank doctoral programs of health education based on the productivity of the faculty and the scholarly activity of doctoral students. No other data set of this type currently exists. Summative information is provided for each of the eight variables used to rank the 28 programs. The data show a wide degree of variation for each of the variables. Further analysis demonstrates that many faculty and programs are successful in obtaining external funding, publishing in refereed journals, and mentoring doctoral students. The data provide what would be needed for programs to improve their ranking.

**********

Although rankings of higher education institutions and programs are commonplace, it wasn't until recently that a ranking of doctoral programs in health education was completed (Notaro, O'Rourke, & Eddy, 2000). Academic quality rankings are available for a number of disciplines or fields of academic study. Data used to develop these ranking often are not made available to the reader. This study describes the data used in "Ranking of Doctoral Programs of Health Education" (Notaro, O'Rourke, & Eddy, 2000). Provided is a profile of program faculty scholarly productivity and student indices of scholarly activity. Administrators, faculty, prospective students and organizations concerned with professional preparation may find the results informative.

Rankings are an effective device in any subject area used to compare items that are difficult to measure. Certainly, academic programs are an area difficult to compare. Vast numbers of universities with an array of similar programs warrant a ranking system. Despite the multitude of articles providing academic quality rankings across universities or in individual disciplines, there is not one accepted ranking methodology. The literature evidences four major types of contemporary methods to rank academic programs.

* First is ranking academic quality by reputation (Roush, 1995). In this method, individuals knowledgeable in a discipline or esteemed in their field are asked to rank order programs.

* Second are academic quality rankings based on faculty productivity. Common measurements of faculty productivity include the number of articles published, the number of citations one's work receives in the literature and monies generated from grants and contracts (West & Rhee, 1995; Taubes, 1993; Webster, 1986; Tauer & Tauer, 1984).

* Third are rankings based on student achievements. These include such measures as SAT and GRE (standardized) test scores and placement of students in leadership or academic positions upon graduation.

Fourth are rankings based on institutional resources. These include measures such as the number of volumes in the library, student-faculty ratio and the amount of resources expended per student (West & Rhee, 1995; Webster, 1986). This study included data for two methods: faculty productivity and student achievement, as both can be employed to measure faculty productivity and doctoral student activity which is the measure of academic quality in this study. Eight variables, four pertaining to faculty and four regarding doctoral students, weighted by scholars and leaders in the field of health education were used to rank the programs (see methods and data collection procedures for a detailed description of the study). Reputation, although an often used measure of academic quality and a valuable indicator of perceived performance, was not the focus of this study because, by nature, it is highly subjective and of questionable usefulness (Notaro, O'Rourke, & Eddy, 2000).

SIGNIFICANCE OF THE STUDY

The significance of this study is twofold. Data from this study would appear to be useful to university administrators, faculty, students, and organizations concerned with professional preparation in health education. Administrators may find academic rankings useful in the allocation and acquisition of resources (Miller, Tien, & Peebler, 1996; Scott & Mitias, 1996; Webster, 1992). Faculty have an interest to see where one's program or institution "stands" against other similar programs (Scott & Mitias, 1996; Katz & Eagles, 1996; Lowry & Silver, 1996; Goodwin, 1995). Many students utilize rankings to determine where to continue their academic study (Miller, Tien, & Peebler, 1996; Scott & Mitias, 1996; Webster, 1992; Morrison, 1987). Similarly, organizations interested in professional preparation may find the results useful. It is anticipated that the presentation of these data will provide benchmark measures for those wishing to enhance program quality.

Secondly, just as this study is based on an extensive review of the ranking literature, the use of multiple indices employed to rank programs can enhance efforts to improve research methodology in future ranking studies in health education as well as in other disciplines. As the methodology evolves, the authors invite comments from the field.

PURPOSE OF THIS STUDY

This study provides a description of the data for each of the eight variables used to rank doctoral programs of health education. The variables used to rank programs measure academic productivity by: articles and citations in key selected journals, editorships in the same selected journals, external funding for research and contracts, scholarly activity of doctoral students, doctoral student/faculty ratio, faculty mentoring and placement of doctoral students, and doctoral student support including assistantships for teaching and research. Previous research have identified leading journals of health education which would be appropriate for this study (Laflin, Horowitz, & Nimms, 1999; Price & Robinson, 1999; Everett, Casler, & Summers, 1994).

METHODS AND DATA COLLECTION PROCEDURES

A detailed description of the methodology is available by Notaro et. al, 2000. That study used eight variables with weighted values determined by an initial survey of scholars and leaders in the field of health education. These eight variables represent a consensus of scholars and leaders in the field of health education with regard to the key quality indicators of doctoral level health education programs. Data for the eight variables were collected for the calendar years 1993-1997 by a survey of doctoral programs of health education, computer index searches, and library searches.

Data from the second survey were collected from department heads or coordinators of the doctoral programs of health education. Twenty-eight of the 43 programs participated in this study for a response rate of 65 percent. The department heads or program coordinators of the doctoral programs of health education were asked to provide data on the following five variables with the weighted values in parenthesis: External funding (18.1%), mentoring and placement of doctoral students (13.2%), doctoral student activity (11.8%), student faculty ratio (11.5%), and student support (9.0%). Computer index searches were used to determine the number of articles published (17.3%) and citations received (9.9%) by program faculty in 81 preeminent journals of health education as identified in the literature (Laflin, Horowitz, & Nimms, 1999). Publication and citation counts include only those accessible through the computer search mechanism and may not reflect actual publication and citation counts. A library review of the 81 journals was conducted to determine the number of editorships (9.2%) by program faculty. Data were tabulated and analyzed using the Statistical Package for the Social Sciences (SPSS). Descriptive statistics are provided for each of the eight variables.

LIMITATIONS OF THE STUDY

The limitations of this study are generally typical of ranking studies in other disciplines. To control for size, each program was limited to submitting, prior to data collection, the names of five faculty and 10 doctoral students as reported by the program or department heads. The journals used in this study were a set of 81 journals identified by Laflin et al. (1999) as preeminent journals in health education. The articles and citation indexes were searched by the authors' names as provided by the program or departments; and, all authors listed in an index for an article received full credit for a publication or citation because some sources list authors alphabetically making it difficult to determine which author had the greatest contribution to an article. Also, the results of this study are limited to the 28 doctoral programs of health education participating in this study.

RESULTS AND DISCUSSION

Results are presented in the order that variables were weighted as determined by scholars and leaders in health education. The weightings are provided as percentages in parenthesis after each variable heading.

EXTERNAL FUNDING (18.1%)

External funding was determined by the approximate total dollar value (direct and indirect costs) of health education grants and contracts where the faculty member served only as a principal investigator or co-principal investigator (not as a subcontractor), for the five year period January 1993 through December 1997, that contributed directly to the following; (a) Faculty and graduate student health education research, (b) Health education community intervention activities or demonstration projects, (c) Health education innovations, training, continuing education and related activities, (d) Other (respondents were asked to explain). If one grant/contract contributed to more than one category, the funds were to be either divided or allocated to the most related category. Funds were not to be double counted. Grants and contracts whose primary purpose was non-health education related were not included.

As indicated in Table 1, a wide range (from a high value of $20,500,000 to a low value of $18,000) of external funding was noted. The top five ranked programs all reported more than 10 million dollars funding while the lowest six programs reported less than $500,000 in external funding. The wide variation was noted in measures of central tendency with the mean ($5,042,076) being higher than the median ($3,248,074) due to the presence of several programs that received substantially more funds than many of the other programs.

ARTICLES PUBLISHED BY PROGRAM FACULTY(17.3%)

Articles published were collected from the 81 journals identified by Laflin et.al., (1999). The numbers of articles published were based on the sum of the five faculty members provided by each program that were retrievable from the 81 journals by the use of automated library indexes. Books were not used for several reasons. Books are not subject to peer review to the same extent as peer reviewed articles. However, to assess the effect of books on rankings, a separate analysis was conducted correlating academic publishing productivity of articles alone to articles and books combined. The variables were nearly perfectly correlated (.98) indicating excluding books from the analysis would not significantly effect the rankings. The indexes used were ERIC, MEDLINE, Psych Info, and Web of Science.

Table 1 evidences a wide range in articles published with a maximum number of 32 and a minimum number of one article. The top six programs published 20 or more articles with two exceeding 30 articles. The bottom 12 programs published fewer than 10 articles. The mean number of articles published by the programs was 13.0. While good overall faculty performance is dearly one explanation of the higher producing programs, other explanations could be considered. First, the presence of a "star" faculty with numerous publications in a program may give the appearance of total program activity that is in actuality limited to one individual. Second, some faculty publish in more popular areas of social concern (such as the effects of alcohol and smoking). The higher number of publications may, in this case, more reflect the popularity of the topic rather than its scholarly content. Also, programs with more publications may simply have more faculty. Although this study limited the number of faculty to five from each program, the larger programs may have more faculty to handle administrative duties resulting in faculty from large programs having more time to devote to research. Finally, the number of publications may also reflect the mission of the program in that research institutions would be expected to have more publications while programs with a teaching focus would more likely have fewer publications.

MENTORING AND PLACEMENT OF DOCTORAL STUDENTS (13.2%)

To measure faculty mentoring of students and quality placement the department heads were asked to provide a list of up to 10 of the top students who received their doctorates during the five-year period January 1993 through December 1997. The number of articles published and citations received was determined in the same fashion the faculty counts were derived from the indexed journals from the 81 health education related journals identified by Laflin et al. (1999). In addition, placement was determined by the number of the 10 doctoral students who were employed at Carnegie Research I Institutions or at national level health institutions/organizations such as the National Institute of Health, the Center for Disease Control and Health Promotion, and the National Arthritis Foundation. Each component (articles/citations and placement) were weighted equally at 0.50. Articles and citations received a weight of 0.50 in the same proportion relative to each other as the weight applied in survey one (articles to citation ratio of 1.74). This resulted in the 0.50 being comprised of .317 articles and .183 citations. The number of doctoral students employed at Carnegie Research I Institutions or at national level health institutions also received a weight of 0.50. The weighted scores were combined with the greatest total being the highest raw score.

A wide range of results are noted in Table 1 for all three of the elements of this variable. The range of articles had a high value of 18 and a low of zero with a mean of 5.0. The top six programs had 10 or more articles published by doctoral students while the bottom eight programs had one or less. Citations also exhibited a wide range from a high of 158 to a low of zero citations making the median (19.5) a more valuable indicator of central tendency than the mean (37.3). The top three programs received over 100 citations while three programs had no student citations and another five programs had less than 10 citations. The number of articles and citations of doctoral students may reflect the number of faculty to collaborate on projects as well differences in research or teaching missions of various programs. The number of the 10 doctoral students used in this study that were placed into a Carnegie Research 1 University had a wide range from a high of six to a low of zero students. However, only five programs had four or more students placed in Research 1 Universities with only one of those with five or above. On the contrary, 16 of the 28 programs had zero or one student placed into a Research 1 University. The mean value (1.4) was slightly lower than the median (1.8). In part, the absence from R1 universities may be due to most professional preparation programs in health education not having a doctoral program.

SCHOLARLY ACTIVITY OF DOCTORAL STUDENTS (11.8%)

To measure the activity of doctoral students in health education in research, teaching, and service while a graduate student, the self reported survey data was collected from the department head or program coordinator that included two mutually exclusive responses which were (1) the percentage of students who had sole responsibility for teaching two or more health education classes, and (2) the percentage of students who had sole responsibility for teaching one health education class. Students teaching two or more classes received a weight of 0.50, while students responsible for one class received a weight of 0.25. The remaining two items were (3) the percentage of students that were appointed by the health education department with a paid assistantship to assist health education faculty with: teaching a class, research projects, or service projects received a weight of 0.25, and (4) the percentage of students that served on a university, college, school or departmental committee as a representative of the department or program also received a weight of 0.25. The weighted scores were combined with the greatest program total being considered the highest raw score.

Each element of this variable exhibited a wide range of results (see Table 1). The percentage of students who had sole responsibility for teaching two of more health education classes ranged from zero to 100 percent. Despite this wide range, only three programs had more than 75 percent of their students teach two or more classes in contrast to the 11 programs that had no students teach two or more classes. Similarly, a wide range existed for programs that had students teach one dam of health education with a high value of 60 percent and a low value of zero percent. Only three of the programs had 50 percent or more students teach one dam of health education while 16 of the 28 programs had no students teach one dam of health education. This variation may be evident because doctoral students would normally be teaching undergraduate students and the percentages of this activity may be linked to the undergraduate mission of the various programs, particularly as some programs of public health do not have undergraduate students.

The percentage of students with assistantships for teaching and research ranged from a high value of 100 percent to a low value of zero percent. These results were evenly distributed with only six programs having 80 percent or more students with assistantships while only six other programs had no students with assistantships. Finally, students serving on committees had a wide range from zero to 100 percent. However, only one program with 100 percent of its students serving on committees was the only program to exceed 50 percent. This may be a reflective of a small number of doctoral students in programs. The remaining 27 programs ranged from five programs at zero percent to three programs with 40 to 50 percent of the students on committees.

STUDENT FACULTY RATIO (11.5%)

The student/faculty ratio was self-reported data and the raw score determined by dividing the approximate annual number of total doctoral students (not just graduates) by the approximate annual number of total full time equivalent (FTE) faculty per year. The lowest student/faculty ratio was considered the highest raw score. To determine the total number of doctoral students, full time students were given a weighted value of 1.0, and part time students were given a weighted value of 0.5. The two weighted values of doctoral students were combined. To determine the number of FTE faculty, the amount of time dedicated to the health education program of full time and part time faculty were combined. For example, six faculty with two at 100 percent time, two @ 50 percent time and two @ 25 percent time in the health education program would equal 3.5 FTE faculty. Results are presented in Table 1.

The mean number of students per faculty member (2.2) was slightly lower but close to the median (2.6) due the narrow range of values for the student faculty ratio. Four programs had less than one student per faculty member while two programs had seven or more students per faculty. The low range may be due, in part, to the existence within departments of bachelor and master degree programs. Although it would seem that a lower student faculty ratio would be better for student interaction with the faculty it may, in some cases, not be a positive indicator. Too few students may result in less chance for fellow doctoral students to interact, learn, and socialize with each other.

CITATIONS RECEIVED BY PROGRAM FACULTY (9.9%)

Citations received were collected from the 81 journals identified by laflin et.al., (1999). The numbers of citations received were based on the sum of the five faculty members provided by each program by the use of an automated library index. Citations were obtained from the Social Science Citation Index in the Web of Science index that contained 68 of the 81 journals, which accounted for 84 percent the total journals. Citations from the remaining 13 journals were not included.

The wide range of citations received (see Table 1) makes the median (156.5 citations) a more valid measure of central tendency than the mean (271.2 citations). The faculty of the top two programs were the only faculty to receive over 1000 citations, while the faculty at the bottom nine programs each received fewer than 100 citations. The wide range of citations was an expected finding based on the wide range of articles published as these two variables are highly correlated (Notaro, O'Rourke, & Eddy, 2000). This simply means that if faculty don't publish they can't be cited in the literature. Faculty that publish in more socially popular fields may receive more citations than others publishing in new areas of study. Therefore citations may not be an accurate measure of the quality of re search and contribution to advancing a field of study but may, in part, be more related to the popularity of the article relative to current topics in the mass media or on agenda of the university's administration. In addition, the presence of a star faculty with numerous publications may result in a program having a large number of citations that may not be reflective of the overall productivity of the programs faculty. Finally, it also may be that programs with larger numbers of faculty may be able to spend more time on research activities as the administrative duties can be spread over a greater number of researchers.

EDITORSHIPS (9.2%)

Editorships by program faculty were collected from the 81 journals identified by Laflin et.al., (1999). Editorships were obtained for 92 percent of the 81 journals by reviewing the appropriate journal editions. Results are presented in Table 1.

Despite the wide range of editorships from a high value of 32 to a low value of zero, the similar value of the mean (9.9) and the median (9.5) shows the results for editorships were not as skewed as other variables. All the programs except the lowest seven had at least five faculty editorships with the top two programs exceeding 30. Editorships were moderately correlated with articles published (.31) which may mean that faculty who publish more frequently are more visible to journals and may result in the opportunity of an editorship (Notaro, O'Rourke, & Eddy, 2000).

STUDENT SUPPORT (9.0%)

To measure student support for assistantships and research, three items of self reported information were used with the first two items being mutually exclusive; (1) Percent of doctoral students receiving a 50 percent or more teaching or research assistantship received a weight of 0.50, (2) Percent of doctoral students receiving a 24-49 percent teaching or research assistantship received a weight of 0.25, and (3) Percent receiving internal or external financial dissertation support received a weight of 0.50. The greatest total weighted score was considered the highest raw score.

Many programs provide 50 percent or more assistantships, and if they didn't many provide 25-49 percent assistantships to program doctoral students. Much less common is dissertation support as only six programs provided 50 percent or more of the doctoral students this funding. Student support correlated moderately with mentoring and student placement (.57). This would seem to show that students that receive support are more involved with program faculty through teaching and research activities. This interaction appears to lead to more opportunities to collaborate on research projects and receive recommendations for placement into faculty positions. Student support was also moderately correlated with student activity. This is not art unexpected result as student support in assistantships often involve activities that include teaching and research duties.

CONCLUSION

How might this data from this ranking be used? What are some of the implications of the availability of this data? As mentioned previously, although ranking studies are abundant in a number of disciplines, the data used to determine those rankings often are not made available. This study provides a descriptive analysis of the data used to generate a doctoral program ranking. Results indicate a wide range of findings for each variable. Programs of similar nature (in this case doctoral programs of health education) appear to vary greatly in the amount of external funding as well as the scholarly activity of program faculty and doctoral students. These programs also may vary significantly by their mission of research and teaching, as well as internal support from the academic institution in which they reside. The data from the study may provide some insight to the mission and focus of these doctoral programs. The information may suggest areas to strengthen efforts, or to downsize or potentially consider eliminating programs. The data can also be used to attract and retain program faculty and doctoral students by enabling these individuals to select a program that is a better match for their particular academic interests.

More specifically, these data may be used as "benchmarks" that can serve as goal levels for programs to which programs can aspire. "Benchmarks" developed from the data are significant achievements by programs that could serve to guide future activity to enhance program development and future ranking considerations include the following:

1. Departments of Health Education are obtaining funds from external sources.

2. The work of faculty from doctoral programs of health education are frequently cited.

3. Student/Faculty ratios for doctoral students in health education are in the range to allow for effective mentoring.

Finally, a general review of ranking data may enhance efforts to improve research methodology in future ranking studies. It is not, not has ever been, nor will be in the future the intent of the authors to make evaluative decisions about any program. In the end, it is the programs and readers of this information that can interpret and utilize the findings of the study for their own purposes.

Table 1. Descriptive Analysis of Ranking Variables

Elements                      Weight      Mean        Median

External Funding               18.1%
  Funds                                $5,042,076   $3,248,074
Articles Published             17.3%
  Faculty Articles                        13.2         13.0
Mentoring & Placement          13.2%
  Articles (31.7)                          5.3          5.0
  Citations (18.3)                        37.3         19.5
  R1 Placement (50.0)                      1.8          1.4
Student Activity               11.8%
  2 Classes (50.0)                        --           --
  1 Class (25.0)                          --           --
  Assistantships (25.0)                   --           --
  Committees (25.0)                       --           --
Student/Faculty Ratio          11.5%
  Ratio                                    2.56         2.25
Citations Received              9.9%
  Faculty Citations                      271.2        156.5
Journal Editorships             9.2%
  Faculty Editorships                      9.9          9.5
Student Support                 9.0%
  50% or more (50.0)                      --           --
  25-49% (25.0)                           --           --
  Dissertation funds (25.0)               --           --

Elements                         High         Low     H/L Ratio

External Funding
  Funds                       $20,482,000   $18,000    1137.9
Articles Published
  Faculty Articles                 32          1         32.0
Mentoring & Placement
  Articles (31.7)                  18          0           18 *
  Citations (18.3)                158          0          158 *
  R1 Placement (50.0)               6          0            6 *
Student Activity
  2 Classes (50.0)                100%         0%         100 *
  1 Class (25.0)                   60%         0%          60 *
  Assistantships (25.0)           100%         0%         100 *
  Committees (25.0)               100%         0%         100 *
Student/Faculty Ratio
  Ratio                             7.14       0.36        19.8
Citations Received
  Faculty Citations              1309          2          654.5
Journal Editorships
  Faculty Editorships              32          0           32 *
Student Support
  50% or more (50.0)              100%         0%         100 *
  25-49% (25.0)                   100%         0%         100 *
  Dissertation funds (25.0)       100%         0%         100 *

-- Means and median were not calculated as results were given as
percentages

* To avoid ratios of infinity, low scores of 0 were given a score of 1
to calculate ratio.

REFERENCES

Everett, S.A, Casler, S., & Summers, J. (1994). Frequency of topics, institutions and authors of four health education journals: 1982-1992. Health Educator, 26, 11-17.

Goodwin, I. (1996). PhD programs II: NRC's new pecking order sets off crowing and puts some danders up. Physics Today, 48,67-69.

Katz, R.S. & Eagles, M. (1996). Ranking political science programs: A view from the lower half. Political Science & Politics, 29(2), 149-154.

Laflin, M.T., Horowitz, S.M., Nimms, J.K. (1999). A researcher's guide to health education journals. American Journal of Health Behavior, 23(3), 210-232.

Lowry, R.C. & Silver, B.D. (1996). A rising tide lifts all boats: Political science department reputation and reputation of the university. Political Science & Politics, 29(2), 161-167.

Miller, A.H., Tien, C., & Peebler, A.A. (1996). Departmental rankings: An alternative approach. Political Science & Politics, 29(4), 704-717.

Morrison, R.L. (1987). Review of academic quality ranking of american collages and universities. Journal of Academic Librarianship, 3, 37.

Notaro, S.J., O'Rourke, T.W., & Eddy, J.M. (2000). Ranking of doctoral programs of health education. American Journal of Health Education, 31 (2), 81-89.

Price, J.H. & Robinson, K.L. (1999). Perceptions of quality of journals related to health education. Psychological Reports, 84, 259-266.

Roush, W (1995). Grad school rankings rankle. Science, 269, 1660-1662.

Scott, L.C. & Mitias, P.M. (1996). Trends in rankings of economic departments in the U. S.: An update. Economic Inquiry 34, 378-400.

Taubes, G. (1993). Measure for measure in science. Science, 260, 884-886.

Tauer, L.W. & Tauer, J.R. (1984). Ranking doctoral programs by journal contributions of recent graduates. American Journal of Agriculture Economics, 66(2), 170-176.

Webster, D.S. (1986). Academic Quality Rankings of American Colleges and Universities. Springfield, IL: Charles Thomas.

Webster, D.S. (1995). Rankings of undergraduate education in U.S. News & World Report and Money: Are they any good? Change, 24(2), 19-31.

West, C.K. & Rhee, Y. (1995). Ranking departments of sites within colleges of education using multiple standards: Departmental and individual productivity. Contemporary Educational Psychology, 20, 151-171.

HEALTH EDUCATION RESPONSIBILITY AND COMPETENCY ADDRESSED

Responsibility IX: Administering Health Education Programs

Competency D: Obtain acceptance and support for programs.

Sub-competency 4: Use needs assessment information to advocate for health education programs.

Stephen J. Notaro, Ph.D. and Thomas O'Rourke, Ph.D., MPH, CHES are affiliated with the Department of Community Health at the University of Illinois at Urbana-Champaign. James M Eddy, D.Ed., CHES is a professor in the Department of Health and Kinesiology at Texas A&M University. Address all correspondence to Thomas O'Rourke, Ph.D., MPH, CHES, University of Illinois at Urbana-Champaign, Department of Community Health, 1206 S. Fourth Street, Champaign, IL 61820, PHONE: [phone], FAX: [phone], E-MAIL: [e-mail].