Use of demographic and quantitative admissions data to predict performance on the national physical therapy examination.

To become a physical therapist in the United States, a student must first graduate from an accredited professional physical therapist education program and then pass the National Physical Therapy Examination (NPTE). Students who fail the NPTE face restrictions on their ability to practice while they prepare to retake the examination. For physical therapist education programs, low NPTE pass rates may negatively affect accreditation decisions and reflect poorly on the program. Therefore, prediction of performance on the NPTE could be useful to both students and faculty in physical therapist education programs.

The NPTE is a 200-question, standardized examination that is administered by computer. (1) The examination questions are in multiple-choice format and require the student to respond to clinical scenarios that are commonly encountered by new physical therapist graduates. The NPTE is scored on a scale of 200 to 800, and the minimum passing scaled score is 600 in all 50 states. (1) The examination is based on an analysis of professional (entry-level) physical therapist practice that is updated every 5 years by the Federation of State Boards of Physical Therapy (FSBPT). A new practice analysis was completed and a new passing standard was established as of November 15, 2002. (1) As a result of the practice analysis, the content and overall difficulty of the examination changed. (2) Mean first-time pass rates on the NFFE were lower for students who graduated from all accredited programs in 2003 and 2004 (77% and 71%, respectively) than for 2000, 2001, and 2002 graduates (90%, 91%, and 85%, respectively). (2) The updated test contents and standard are the likely reasons why the pass rates for 2003 and 2004 were lower than the pass rates for 2000 to 2002.

The Commission on Accreditation in Physical Therapy Education (CAPTE) requires all programs to report pass rates as part of ongoing curriculum review. (3) Students considering a career in physical therapy also may review programs' NPTE pass rates when selecting a physical therapist education program. The financial stakes for a student who fails the NPTE are high. The FSBPT charges $350 to take the test, in addition to a $50 fee charged by the testing center and application fees paid to the state where the student seeks a license. In many jurisdictions, the student cannot practice until the examination is passed, meaning that a student who fails the examination loses potential wages in addition to the testing and application fees.

Despite the importance of the NPTE as a gateway to clinical practice, few researchers have investigated whether data obtained prior to admission to a professional physical therapist program can be used to predict performance on the licensing examination. Two early studies (4,5) examined students enrolled in baccalaureate programs. Because all programs are currently educating students at the post-baccalaureate level, these data likely are no longer generalizable. More recently, Thieman et al (6) used data from 121 students in a single master's-level program to determine whether grade point average (GPA) in prerequisite courses and Graduate Record Examination (GRE) scores could be used to predict raw scores on the NPTE. They found that prerequisite course GPA and GRE scores, along with age, accounted for 11% of the variance in raw NPTE score. (6) In a similar study using data from 107 students in a single program, Dockter (7) found that prerequisite course grades were weakly correlated (r=.34) with raw NPTE score.

Although these studies (6,7) indicate that past academic performance may be associated with performance on the NPTE, neither study provides information that could be directly applied to current physical therapist education programs or students. The purpose of our study was to investigate the utility of demographic and quantitative admissions data in predicting whether students failed the NPTE at least one time. We examined only admissions data and not post-matriculation variables such as professional program GPA. We were interested only in the variables that academic program admissions committees use in making admissions decisions and the extent to which these types of variables predict NPTE performance. By matching data from a nationally representative sample of professional Master of Physical Therapy degree (MPT) and Doctor of Physical Therapy degree (DPT) programs with NPTE data provided by the FSBPT, we examined the following questions: (1) can preadmission demographic data, GPA, and verbal and quantitative Graduate Record Examination scores (vGRE, qGRE), either in isolation or in combination, be used to predict physical therapist student risk for failing the NPTE, and (2) does the extent of prediction of NPTE failure vary by program?

Method

Program Recruitment

Physical therapist education programs were considered eligible for the study based on 2 criteria. First, they must have used GRE scores in their admissions processes for students who were admitted to the classes graduating in 2000 through 2004. Second, because we were interested in assessing predictors both between and within programs, we considered only those programs with planned class sizes of 30 or more students to be eligible. We estimated that at least 150 subjects (30 students per year for 5 years) would allow a minimum of 5 subjects per cell given the number of independent variables in the analysis. We chose the years 2000 to 2004 because data for those years were the most recent data available at the time of the study. Overall, 95 programs met the eligibility criteria. Quota sampling was used to recruit 23 of these programs to participate in the study. Details on specific recruitment procedures are available in our companion articles in this issue.

Subjects

Participating programs provided data regarding all students admitted to the classes scheduled to graduate in the years 2000 through 2004. During the data collection period, 3 of the 23 participating programs withdrew from the study. These 3 programs cited lack of time and difficulty compiling the required data as reasons for not participating. The final sample included data on 3,585 students admitted to 20 physical therapist education programs.

Data Collection

For each student in the sample, the FSBPT indicated student performance on the examination using the coding outlined in Table 1. This variable was recoded into a dichotomous pass (category 1) or fail (categories 2, 3, and 4) score, which was used as the dependent variable in the study. Physical therapist education programs provided data on the independent variables, which included undergraduate GPA (uGPA), vGRE, and qGRE. Undergraduate GPA was collected as a continuous variable, ranging from 0 to 4.0. The vGRE and qGRE are each scored on a scale of 200 to 800 in 10-point increments. (9) In cases where students attended more than one undergraduate institution or took the GRE multiple times, programs were asked to record uGPA and vGRE and qGRE scores that were used during the admissions process. Analytic GRE (aGRE) scores were not included in the study because the aGRE was changed to an essay-based assessment in 2002. (10) Most students enrolled in physical therapist education programs during the years 2000 through 2004 would have applied for admission prior to 2002 and would have taken the older, multiple-choice version of the aGRE. Data regarding student demographics and program characteristics also were collected. These independent variables are described in Table 3 in our companion articles in this issue.

Procedure

For each partipating program, a data collection spreadsheet and detailed instructions were sent to the program director. The program director or the director's designee recorded student identifiers, demographic information, uGPA, vGRE, and qGRE on each student admitted to the program during the study period. The program sent the spreadsheet, excluding the student identifiers, to us as a password-protected electronic mail attachment. The program also sent a spreadsheet, which included only student identifiers, to the FSBPT. The FSBPT recorded NPTE status information on each student, deleted the student identifiers, and then returned the remainder of the data to us. At no time did we have access to individual students' names, social security numbers, or dates of birth. To protect student anonymity, we used random matching codes to match data provided by the programs (student demographic data, uGPA, vGRE, and qGRE) to the data provided by the FSBPT (NPTE status). The Institutional Review Board of Virginia Commonwealth University and the institutional review boards of select participating academic programs approved the methods used in this study.

Data Analysis

Prior to analyzing the data, the uGPA was transformed by multiplying by 10. By doing so, we were able to describe how the odds of NPTE failure change when the uGPA changes by 0.10 (eg, from 3.1 to 3.2). Similarly, vGRE and qGRE were transformed by dividing by 10. This transformation allowed discussion of how odds of NPTE failure change for every 10-point change (eg, from 400 to 410) in GRE score.

Between-program analysis. Because 2 different versions of the NPTE were administered during the study period, data were analyzed separately for each version of the examination. Throughout the remainder of this article, the examination that was used prior to November 2002 (and completed by the majority of the graduates from 2000 to 2002) will be referred to as version 1, and the version used after November 2002 (completed by graduates from years 2003 and 2004) will be referred to as version 2.

Two separate hierarchical logistic regression models were generated for the entire sample of students, one for those taking version 1 and one for those taking version 2 of the NPTE. The dependent variable was dichotomized into students who passed the NPTE on their first attempt and those who did not. Race or ethnicity was not included in the models generated for each version of the NPTE examination because cell sizes were too small for some racial or ethnic categories. Tabachnick and Fidell (11) noted that small cell sizes can cause errors in logistic regression analyses and recommended expected cell frequencies of at least 5.0. Cross-tabulation analyses of all combinations of the nominal independent variables (program, school ownership, degree offered, school classification, school region, sex, and cohort) and NPTE failure revealed 3.5% of cells with expected frequencies less than 5.0. This is well within Tabachnick and Fidell's recommendation that fewer than 5% of cells should have frequencies less than 5.0. After dichotomizing NPTE score to pass/fail and eliminating the race or ethnicity variables, our data set exceeded these recommendations.

We used hierarchical regression models because we wanted to enter variables into the model in a pre-specified order based on evidence or theoretical arguments. (12) First, we entered the program-level variables of year graduated, academic program, and degree level. We wanted to adjust for these variables to account for differences among programs and among pools of applicants to physical therapist education programs over different years. We suspected that these differences would affect the prediction of NPTE failure and, therefore, wanted to control for this variation in the analysis. Second, we entered the individual level variables of age and sex. We included individual-level variables in our analysis because research has suggested that performance on standardized tests may vary with respect to demographic characteristics. (13-17) For the final step in model, we entered uGPA, qGRE, and vGRE.

We were unable to enter geographic region and Carnegie classification into the models because the academic program variable led to cell sizes that were too small. We conducted 2 simple logistic regressions with NPTE performance as the dependent variable and Carnegie classification (as listed in the online directory of accredited programs (18)) or geographic region as the independent variable. Doctoral institution category and the Northeast region, respectively, served as the referent categories. Carnegie classification was not predictive of NPTE failure ([chi square]=5.097, P=.078). Geographic region was predictive of NPTE failure ([chi square]=33.716, P>.001), with students at institutions in the south having 1.8 times the odds of failure compared with students from northeastern institutions (95% confidence interval [CI]=1.370-2.464).

We used the -2 Log-Likelihood test (P<.05) to evaluate whether the final hierarchical models for the 2 NPTE test versions represented improvement over the constant-only model. (11) We used the Hosmer and Lemeshow test (P>.05) to evaluate how well the model fit the data. (11,19,20) To determine which independent variables contributed to the prediction models, Wald statistics and adjusted odds ratios were examined. (20)

We found that the hierarchical logistic regression analysis for version 1 of the NPTE indicated that uGPA, vGRE, qGRE, and age contributed to prediction of NPTE failure after controlling for other potential confounding variables in the model. For version 2 of the examination, the analysis indicated that uGPA, vGRE, and qGRE contributed to the prediction of NPTE failure after controlling for confounding variables. Age did not contribute to prediction in the model using data from version 2 of the NPTE. The logistic regression models for version 1 and version 2 were similar in that the [beta] coefficients and odds ratios for uGPA, vGRE, and qGRE were nearly identical; the odds ratios varied by 0.01 to 0.02. This indicated to us that uGPA and GRE scores were equally good at predicting NPTE failure for both versions of the NPTE. Therefore, a logistic regression model was developed using data from all students in the study that included test version as a control variable. All of the variables used in the version-specific models were included in the combined model. Because our sample size was effectively doubled over that for either version 1 or version 2, we also were able to include the race or ethnicity variable with 5 levels (African American, Asian/Pacific Islander, Hispanic, white/non-Hispanic, and "other").

For continuous independent variables (ie, uGPA, vGRE, and qGRE) identified as significant predictors of NPTE failure in the hierarchical model for the entire sample, we plotted receiver operating characteristic (ROC) curves to determine how well a variable discriminated between students who passed the NPTE on the first attempt and students who did not. (21) The curves also were used to identify threshold scores for the independent variables for predicting NPTE failure. (22) To identify the best cut-point thresholds for uGPA, vGRE, and qGRE, the upper left point on the ROC curve was selected. We encourage readers to examine the article by Deyo and Centor (23) for a thorough description of ROC curve analysis. These thresholds were used to recode uGPA, vGRE, and qGRE into discrete variables. A final hierarchical logistic regression model was constructed using these newly coded independent variables, along with the other significant variables from the hierarchical logistic models described earlier.

First, the uGPA and GRE variables were divided into tertiles, and a logistic regression model was fit using the recoded variables. The model was evaluated for goodness of fit and contribution of the recoded variables. If dividing a variable into tertiles did not result in significant differences among the 3 score ranges, the variable was dichotomized using the best cut-point from the ROC curve analysis, and the logistic regression model was tested again. We began with the tertile method of recoding because we wanted to examine the predictive ability for reasonably small ranges of scores for each variable. We also believed that presenting the data in this manner would allow easier interpretation of the results.

Using this process, uGPA was recoded as a dichotomous variable: 3.48 or lower versus 3.49 or higher. The vGRE was recoded as a discrete variable with 3 levels: 400 or lower, 410 to 480, and 490 or higher. The qGRE score was recoded as a dichotomous variable: 530 or lower versus 540 or higher. Race or ethnicity was recoded with 2 levels: students who were white or Hispanic were categorized into one group, and all other students were placed in the second group. Test version, program, and cohort were used as control variables. After examining this final model for goodness of fit, variables contributing to the model were used to develop a prediction rule for NPTE failure.

Clinicians use clinical prediction rules to assist in making diagnoses or predicting disease or treatment outcomes. (24,25) Clinical prediction rule development involves systematic collection of data from a large sample. These data are analyzed to identify the best combination of variables to predict the outcome of interest. In our study, we used the term "prediction rule" because the rule we developed is intended for use by academicians rather than clinicians. We used logistic regression to identify variables that predict NPTE failure, then used the [beta] coefficients to weight the importance of those variables. This approach is commonly used to develop clinical prediction rules. (26-28) Finally, we tested the prediction rule using the data from all students in the data set. Data were analyzed using the Statistical Package for the Social Sciences (SPSS), version 12.0. *

Within-program analysis. Physical therapist education programs differ with respect to curricular design, applicant pools, grading scales, and academic standards. We therefore hypothesized that the prediction of NPTE failure using uGPA and GRE scores also may vary by program. As a result, we conducted individual hierarchical regression analyses for each program. After controlling for NPTE version, individual-level variables (age, sex, and ethnicity, when permitted by sample size) were entered, followed by the key independent variables of interest (ie, uGPA, vGRE, and qGRE). Between-program variables were not applicable to individual programs.

Results

Population Comparisons

The characteristics of the program and student sample were compared with those of the 95 programs eligible for the study whenever possible. However, aggregate data regarding the 95 eligible programs were not available regarding all of the variables included in this study. Therefore, we relied on population estimates from all accredited programs (29) for the remaining population comparisons. Results of these comparisons are available in our companion articles in this issue. In brief, the program sample was generally representative of the group of 95 eligible programs with respect to geography, degree level, institutional ownership, and Carnegie classification. The student sample was representative of the population of students enrolled in all accredited physical therapist programs with respect to age and uGPA. The student sample included slightly smaller proportions of female, African-American, and Hispanic students than the population of students enrolled in all accredited programs.

Of the 3,585 students in the original sample, data regarding NPTE failure were available on 3,365 students (Tab. 2). Of these, 1,965 students took version 1 and 1,389 students took version 2. Only 7 students took both versions of the NPTE. Because these 7 students first took version 1 and encountered difficulty, they were counted as taking version 1 for the between-program analysis. Ninety-three percent of the students who took version 1 of the NPTE passed on the first attempt, compared with 79% of those who took version 2.

Between-Program Analysis

The hierarchical logistic regression model developed using data from all subjects is presented in Table 3. Program, cohort, and NPTE version were each predictive of NPTE failure. Odds ratios indicated that, when controlling for other variables, the odds of failing the NPTE were increased 12% for each 0.10 decrease in uGPA. As vGRE and qGRE scores decreased by 10, odds of NPTE failure were increased by 6.6% and 3.5%, respectively. Race or ethnicity also contributed significantly to the model. Odds of failing the NPTE were more than 200% higher for students identified as African American, Asian/Pacific Islander, or "other" as compared with white/non-Hispanic and Hispanic students. Based on the Nagelkerke [r.sup.2] test, (11) the final model accounted for 28% of the variance in odds of NPTE failure.

Receiver operating characteristic curves were plotted for uGPA, vGRE, and qGRE. The area under each curve (Tab. 4) was significantly greater than 0.5 (P<.001). The final hierarchical logistic regression model using the recoded variables from the ROC curve analysis is shown in Table 5.

Table 6 presents the individual items in the prediction rule created by rounding the [beta] coefficients to the nearest 0.5 from the final logistic regression model presented in Table 5. For example, students with a high uGPA (3.49 or higher) had reduced odds of NPTE failure, with a [beta] coefficient of -0.740. We rounded this coefficient to -0.5 for the prediction rule. Therefore, if a student earned a uGPA of 3.49 or higher, 0.5 was subtracted from that student's prediction rule score. Each student's prediction rule score was calculated, and then the scores were cross tabulated with the student's actual NPTE performance (Tab. 7). A total of 7% (95% CI = 5.9 - 8.1) of students with a prediction rule score of 0 or less failed the NPTE at least once, and 39.3% (95% CI=33.5-45.2) of the students failed the NPTE when they had a prediction rule score of 1.5 or higher.

Within-Program Analysis

Logistic regression models for predicting NPTE failure varied by program. Verbal GRE was the most consistent predictor, contributing to prediction of NPTE failure in 11 programs (alone in 9 programs, in combination with uGPA in 1 program, and in combination with qGRE in 1 program). Quantitative GRE score alone contributed to prediction of NPTE failure in 2 programs and in combination with uGPA in 1 program. Undergraduate GPA alone contributed to prediction of NPTE failure in 1 program. For 5 other programs, the logistic regression model included only control variables, such as age and cohort.

Using ROC curves, threshold cut-points were developed for each variable contributing to prediction of NPTE failure in each academic program. The results of the logistic regression and the corresponding prediction rules for 3 selected programs are presented in Supplemental Tables 1, 2, and 3 (available online only at www.ptjournal.org). These 3

programs were chosen to illustrate the variation in prediction across the entire sample of participating programs.

Discussion

Between-Program Analysis

The logistic regression analyses indicate that uGPA, vGRE, and qGRE are independently predictive of failing the NPTE. These results are consistent with those of other researchers (6,7) who have found relationships among previous course grades, GRE scores, and scores on the NPTE. However, these earlier studies relied on small samples and used raw NPTE score as the dependent variable. Our study is the first, to our knowledge, to provide estimates of risk of failing the NPTE based on data from a large, nationally representative, multiple-program sample.

The prediction rule we developed may be used to estimate students' risk of failing the NPTE. A logistic regression model that accounted for differences among programs, cohorts, and test versions was used to develop the prediction rule. Thirty-nine percent of students in the sample with a prediction rule score of 1.5 or higher failed the examination. The NPTE version was the strongest predictor of NPTE failure. When data from the 2 versions were examined separately, the contributions of uGPA, vGRE, and qGRE to the prediction of NPTE failure were almost identical. This finding suggests that the newer version of the examination is more difficult, but uGPA, vGRE, and qGRE are predictive of difficulty regardless of NPTE version. "Program" was an important predictor in the model; therefore, programs should consider developing their own prediction rules using data from their own students.

Influence of Covariates on Prediction of NPTE Failure

After controlling for program, geographic region and Carnegie classification could not be included in the hierarchical logistic regression model. Results of a simple logistic regression analysis using region as the sole independent variable suggest that NPTE failure may vary by region. The results of a study by Mohr and colleagues (30) suggest that other program characteristics may be predictive of student NPTE performance. They found that accreditation status, number of faculty with PhD or EdD degrees, and total program length (years of preprofessional and professional course work) were related to program pass rates on the NPTE. (30) Future research should consider these and other program-level variables.

Race or ethnicity was the only student demographic characteristic that contributed to the final logistic regression model. Previous studies in physical therapist education (4-7) did not include this variable in their analyses. Koenig et a1 (16) studied records of 11,279 medical students to investigate differences in prediction of United States Medical Licensing Examination (USMLE) performance based on race or ethnicity. They developed a model using Medical College Admission Test (MCAT) scores and GPA, which over-predicted UMSLE performance for most students. However, the model under-predicted success (ie, the model predicted that the students would not pass when they actually did) for 30 African-American students. (16) These findings suggest that interaction exists among MCAT scores, GPA, and race or ethnicity in predicting USMLE performance in medical students. In our study, no evidence of interaction between race or ethnicity and uGPA, vGRE, or qGRE was found. However, the proportions of students in our sample from some ethnic groups (African American = 2.5%, Hispanic = 2.5%) were very small, which may have reduced the statistical power to detect interactions.

Further study regarding the relationships among race or ethnicity, quantitative admissions data, and NPTE performance is recommended. From the data collected in this study, it is not possible to infer why students in some ethnic categories had increased odds of difficulty passing the NPTE. The increased odds of NPTE failure for some students in these groups may be related to socioeconomic or educational factors that we were unable to measure.

Within-Program Analysis

A variety of models emerged for predicting NPTE failure within individual programs. Some programs had not filled their classes to capacity for some cohorts, and not all programs admitted students took the NPTE. Samples from those programs included data on fewer than 150 students. Thus, the variation in program-specific models may be due to small sample sizes. Swanson et al (31) reported a similar phenomenon in their study on predicting medical student performance on the USMLE. The programs also likely differ with respect to student applicant pools, admissions policies, curriculum design, methods for student testing and evaluation, and academic rigor. Further research is needed to identify factors that may contribute to the variation in prediction within schools.

Score on the vGRE was predictive of NPTE failure in 11 of the 20 programs. Scores on the qGRE were predictive in 4 programs, and uGPA was predictive in 3 programs. For each program, odds ratios, ROC curve areas, and prediction rules varied widely. This finding suggests that discrimination between students who had difficulty passing the NPTE and students who did not varied widely within programs. Program directors should analyze data from their own students whenever possible to generate program-specific thresholds and rules for prediction of NPTE failure.

Implications for Academic Programs and Students

This study provides strong evidence that uGPA, vGRE, and qGRE are predictive of failing the NPTE at least once. Undergraduate GPA reflects the student's previous academic performance, and in our other study, (8) uGPA was the most consistent predictor of student academic performance in professional physical therapist programs. In this study, the most consistent predictor of NPTE failure was vGRE score. According to the Educational Testing Service, the vGRE tests the "ability to analyze and evaluate written material and synthesize information obtained from it, to analyze relationships among component parts of sentences, and to recognize relationships between words and concepts." (9(p4)) The questions on the NPTE require students to analyze and evaluate written material regarding clinical cases, so the skills tested by the vGRE would theoretically be helpful for a student taking the NPTE. Our study appears to support this hypothesis.

Although the GRE was not designed specifically for physical therapist students, many programs use GRE scores as part of the admissions process. An admissions test with content more relevant to physical therapy may improve the ability of the test to predict which students might have difficulty passing the NPTE. Until a better alternative is identified, the results of this study and those of the study described in our companion articles in this issue suggest that vGRE and qGRE scores should be used by academic programs to assist in making admissions and academic training decisions. Because there was so much unexplained variance in the final logistic regression model, we are not suggesting that the prediction rule developed in this study be the sole factor in deciding which students should or should not be offered admission to an academic program. We recommend that physical therapist education programs use the prediction rule to identify students who are at risk for failing the NPTE. At-risk students then can be encouraged to utilize additional resources while completing their academic training and while preparing to take the NPTE. This additional effort may increase at-risk students' likelihood of success, although additional research in this area is clearly needed.

Other authors have suggested that performance during the first year of the professional physical therapist program (7) and professional program GPA (6) may be useful predictors of NPTE performance. We were primarily interested in the utility of admissions data, rather than data collected during the professional physical therapist education program, for predicting NPTE performance. Our results indicate that quantitative admissions data alone account for a small amount of variance in odds of failing the NPTE. Future research that includes data about student performance in the professional program is needed to provide a more complete picture of contributors to NPTE performance.

Limitations

The between-program logistic regression model explained 28% of the variance in odds of NPTE failure. A variety of factors may account for the unexplained variance. First, we were unable to control for differences in the rigor of students' undergraduate education. Studies of medical students (16,31) have included variables to control for rigor of undergraduate education. Second, we did not control for how programs recorded uGPA for students who attended multiple undergraduate institutions or who took the GRE multiple times. Third, we did not include any measures of students' academic performance during the professional physical therapist program. We were primarily interested in identifying predictors of NPTE performance that were measured prior to a student's training in a professional program. Dockter (7) found that GPA during the first year of professional education was the strongest predictor of raw NPTE scores ([r.sup.2] = .420) in her sample of 107 students. Future work with larger sample sizes should examine the extent to which academic performance during professional physical therapist education predicts NPTE performance.

Research in other fields suggests that socioeconomic, educational, and psychosocial factors are likely to contribute to performance on standardized tests such as the NPTE. For example, Fadem et al (32) found that parental income was related to performance on a medical student licensing examination. Gallagher et al (13) found that performance on computerized versions of the GRE varies based on ethnicity, sex, and age. Sedlacek (17,33) developed a noncognitive questionnaire to assess student psychosocial traits such as self-appraisal, positive self-concept, and leadership. Work by Sedlacek and colleagues (17,33,34) suggests that students from different ethnic groups perform differently on standardized tests and that questionnaires such as the Noncognitive Questionnaire (NCQ) may be helpful in predicting student performance. Because students at different institutions often differ in socioeconomic and cultural backgrounds, Webb et al (34) suggested that programs should develop admissions policies that take these factors into account. Because our study showed that NPTE failure varies with race or ethnicity, we concur with this recommendation.

Other study limitations are related to potential sampling and historical biases. Our study included only larger programs that use the GRE in their admissions processes. Therefore, comparisons with smaller programs may not be valid. Because the 95 eligible programs represent only half of the accredited physical therapist education programs in the United States, future research should investigate whether preadmission grades or other measures of intellectual ability predict NPTE failure for students in programs with smaller class sizes or that use preadmission tests other than the GRE.

With respect to historical bias, the NPTE was changed during the study period. We first analyzed data from the 2 versions separately, and we used test version as a control variable in all combined analyses and in the development of the prediction rule. It is important to note that academic program was predictive of NPTE failure and that test version itself was the strongest predictor. We adjusted for these variables in our prediction rule so that academic programs could use the prediction rule shown in Table 7 in lieu of calculating a program-specific prediction rule.

It is possible that the results were influenced by rapid transition of professional physical therapist programs to the DPT degree. However, degree level (MPT versus DPT) did not contribute to the prediction of NPTE failure in our analysis. Changes in the pool of students applying to physical therapist education programs also may have introduced bias. The number of students applying to physical therapist education programs declined during the years of study, while the mean uGPA declined slightly. (29) During the same years, the number of students graduating from accredited programs also declined. (30) This was addressed statistically by controlling for cohort in the between-program analyses and, when possible, in the within-program analyses.

The results of our analyses explained a small proportion of the variance in odds of NPTE failure. Future research should investigate other program and student characteristics that may contribute to NPTE performance. A valid method for adjusting for academic rigor would be an important contribution to the program-level analysis. In addition, qualitative approaches may be useful in identifying variables for further research and clarifying reasons why students from some demographic groups are at higher risk for failing the NPTE than others.

Conclusion

Undergraduate GPA, vGRE, and qGRE, along with race or ethnicity, are independent predictors of NPTE failure. Race or ethnicity may serve as a proxy for other student-level factors, such as socioeconomic status, and requires further exploration in future studies. We concur with other authors who suggested that other sources of data be used in conjunction with GPA and GRE scores when evaluating students for admission. Because of the variation in prediction of NPTE failure within programs, individual institutions should use data from their own programs to develop measurable guidelines for identifying students who may be at risk for failing the NPTE. If this is not possible, using the prediction rule shown in Table 6 is a reasonable, but less accurate, alternative.

All authors provided concept/idea/project design, writing, and data analysis. Dr Utzman provided data collection. Dr Utzman and Dr Riddle provided project management and fund procurement. Dr Riddle and Dr Jewell provided consultation and manuscript review.

This work was completed in partial fulfillment of requirements for Dr Utzman's PhD degree in Health Related Sciences, Medical College of Virginia Campus, Virginia Commonwealth University.

This research was presented at the annual meeting of the Federation of State Boards of Physical Therapy; September 10, 2006; Portland, Ore.

This work was funded, in part, by the Federation of State Boards of Physical Therapy.

This article was received August 8, 2006, and 1 was accepted April 18, 2007.

DOI:10.2522/ptj.20060222

References

(1) 2005 NPTE Candidate Handbook for the National Physical Therapy Examinations: PT, PTA. Alexandria, Va: Federation of State Boards of Physical Therapy; 2005.

(2) Federation of State Boards of Physical Therapy. NPTE Pass Rate Reports. Available at: http://www.fsbpt.org/exams/ PassRates/index.asp. Accessed February 12, 2006.

(3) Commission on Accreditation in Physical Therapy Education. Evaluative Criteria for Accreditation of Education Programs for the Preparation of Physical Therapists. Alexandria, Va: American Physical Therapy Association; 2006.

(4) Gross MT. Relative value of multiple physical therapy admissions criteria in predicting didactic, clinical, and licensure performance. Journal of Physical Therapy Education. 1989; 3(1): 7-14.

(5) Roehrig SM. Prediction of licensing examination scores in physical therapy graduates. Phys Ther. 1988; 68: 694-698.

(6) Thieman TJ, Weddle ML, Moore MA. Predicting academic, clinical, and licensure examination performance in a professional (entry-level) master's degree program in physical therapy. Journal of Physical Therapy Education. 2003; 17(2): 32-37.

(7) Dockter M. An analysis of physical therapy preadmission factors on academic success and success on the National Licensing Examination. Journal of Physical Therapy Education. 2001; 15(1): 60-64.

(8) Utzman RR, Riddle DL, Jewell DV. Use of demographic and quantitative admissions data to predict academic difficulty among professional physical therapist students. Phys Ther. 2007; 87: 1164-1180.

(9) 2005-2006 Guide to the Use of Scores. Princeton, NJ: Educational Testing Service; 2005.

(10) GRE Analytical Writing Measure: An Asset in Admissions Decisions. Princeton, NJ: Educational Testing Service; 2004.

(11) Tabachnick BG, Fidell LS. Logistic regression. In: Tabachnick BG, Fidell LS, eds. Using Multivariate Statistics. 4th ed. Boston, Mass: Allyn & Bacon; 2001: 517-581.

(12) Cohen J, Cohen P. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates; 1983.

(13) Gallagher A, Bridgeman B, Cahalan C. Effect of Computer-Based Tests on Racial/ Ethnic, Gender, and Language Groups. Princeton, NJ: Educational Testing Service; 2000. Report no. 96-21P.

(14) Gallagher A, Levin J, Cahalan C. Cognitive Patterns of Gender Differences on Mathematics Admissions Tests. Princeton, NJ: Educational Testing Service; 2002. Report no. 96-17P.

(15) Grandy J. Trends and Profiles(Second Edition): Statistics About General Test Examinees by Gender, Age, and Ethnicity. Princeton, NJ: Educational Testing Service; 1999. Report no. RR-99-16.

(16) Koenig JA, Sireci SG, Wiley A. Evaluating the predictive validity of MCAT scores across diverse applicant groups. Acad Med. 1998; 73: 1095-1106.

(17) Sedlacek WE, Prieto DO. Predicting minority students' success in medical school. Acad Med. 1990; 65: 161-166.

(18) Commission on Accreditation in Physical Therapy Education. Accredited Physical Therapy Programs, 2004. Available at: http://www.apta.org/Education/accredita tion/dir_acc_PT_ed_prog. Accessed May 2, 2004.

(19) Munro BH. Statistical Methods for Health Care Research. 4th ed. Philadelphia, Pa: Lippincott Williams & Wilkins; 2000: 283-302.

(20) Peng CJ, Lee KL, Ingersoll GM. Introduction to logistic regression analysis and reporting. Journal of Educational Research. 2002; 96(1): 3-14.

(21) Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd ed. New York, NY: Wiley; 2000.

(22) Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology: A Basic Science for Clinical Medicine. 2nd ed. Boston, Mass: Little, Brown & Co Inc; 1991: 69-152.

(23) Deyo RA, Centor RM. Assessing the responsiveness of functional scales to clinical change: an analogy to diagnostic test performance. J Chronic Dis. 1986; 39: 897-906.

(24) Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006; 144: 201-209.

(25) Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules: applications and methodological standards. N Engl J Med. 1985; 313: 793-799.

(26) Leung GM, Rainer TH, Lau FL, et al. A clinical prediction rule for diagnosing severe acute respiratory syndrome in the emergency department. Ann Intern Med. 2004; 141: 333-342.

(27) Solomon DH, Avorn J, Warsi A, et al. Which patients with knee problems are likely to benefit from nonarthroplasty surgery? Development of a clinical prediction rule. Arch Intern Med. 2004; 164: 509-513.

(28) Wells PS, Anderson DR, Bormanis J, et al. Value of assessment of pretest probability of deep-vein thrombosis in clinical management. Lancet. 1997; 350: 1795-1798.

(29) American Physical Therapy Association. Fact Sheet: Physical Therapist Education Programs, 2005. Available at: http://www. apta.org/AM/TemplateRedirect.cfm?template=/CM/ContentDisplay. cfm&Content ID=23836. Accessed February 12, 2006.

(30) Mohr T, Ingram D, Hayes SH, Du Z. Educational program characteristics and pass rates on the National Physical Therapy Examination. Journal of Physical Therapy Education. 2005; 19(1): 60-66.

(31) Swanson DB, Case SM, Koenig J, Killian CD. Preliminary study of the accuracies of the old and new medical college admission tests for predicting performance on USMLE Step 1. Acad Med. 1996; 71 (1 suppl): S25-S27.

(32) Fadem B, Schuchman M, Simring SS. The relationship between parental income and academic performance of medical students. Acad Med. 1995; 70: 1142-1144.

(33) Sedlacek WE. Beyond the Big Test: Noncognitive Assessment in Higher Education. San Francisco, Calif: Jossey-Bass; 2004.

(34) Webb CT, Sedlacek W, Cohen D, et al. Impact of nonacademic variables on performance at two medical schools. J Natl Med Assoc. 1997; 89: 173-180.

* SPSS Inc, 233 S Wacker Dr, Chicago, IL 60606.

RR Utzman, PT, MPH, PhD, is Associate Professor and Academic Coordinator of Clinical Education, Division of Physical Therapy, West Virginia University School of Medicine, PO Box 9226, Morgantown, WV 26506-9226 (USA). Address all correspondence to Dr Utzman at: [e-mail].

DL Riddle, PT, PhD, FAPTA, is Otto D Payton Professor, Department of Physical Therapy, Medical College of Virginia Campus, Virginia Commonwealth University, Richmond, Va.

DV Jewell, PT, PhD, CCS, FAACVPR, is Assistant Professor, Department of Physical Therapy, Medical College of Virginia Campus, Virginia Commonwealth University.

[Utzman RR, Riddle DL, Jewell DV. Use of demographic and quantitative admissions data to predict performance on the National Physical Therapy Examination. Phys Ther. 2007;87:1181-1193.]

Table 1.
Definitions and Coding for National Physical Therapy
Examination Performance

Original Coding of Data              Dependent
Provided by the Federation of        Variable
State Boards of Physical Therapy     Coding

1=passed on first attempt            Pass

2=failed on first attempt, passed    Fail
on second attempt within 15
months of first attempt

3=failed multiple attempts, but      Fail
eventually passed within 15 months
of first attempt

4=took examination one or more       Fail
times, but did not pass within 15
months of first attempt

Table 2.
National Physical Therapy Examination Performance by Cohort

Cohort   Passed on   Passed on   Failed 2 or   Never    Total
         First       Second      More Times,   Passed
         Attempt     Attempt     but Passed

2000       744        39           9            6         798
2001       713        35          12            7         767
2002       659        74          32           15         780
2003       456        63          31           18         568
2004       369        60          23            0         452
Total    2,941       271         107           46       3,365

Table 3.
Logistic Regression for Prediction of National Physical Therapy
Examination (NPTE) Failure Using Continuous Data for Undergraduate
Grade Point Average (uGPA) and Graduate Record Examination Scores
and Data From All Programs and Both NPTE Test Versions

                                           Wald
Predictor             [beta]   SE [beta]   [chi square]   df   P

Program
  1                    0.723   0.612         1.395         1    .238
  2                    1.955   0.457        18.300         1   <.001
  3                    1.885   0.514        13.455         1   <.001
  4                    0.199   0.641         0.097         1    .756
  5                    1.917   0.527        13.227         1   <.001
  6                    0.882   0.456         3.750         1    .053
  7                    2.277   0.428        28.292         1   <.001
  8                    1.494   0.512         8.533         1    .003
                       1.158   0.470         6.081         1    .014
  10                   1.685   0.552         9.325         1    .002
  11                   1.891   0.397        22.732         1   <.001
  12                   2.201   0.402        29.935         1   <.001
  13                   2.141   0.440        23.661         1   <.001
  14                   1.309   0.424         9.551         1    .002
  15                   1.897   0.403        22.119         1   <.001
  16                   1.349   0.494         7.450         1    .006
  17                   2.120   0.528        16.109         1   <.001
  18                   0.115   0.450         0.066         1    .798
  19                   1.308   0.470        73.763         1    .005
  20                                        90.953        19   <.001
Cohort
  2000                 1.011   0.337         9.005         1    .003
  2001                 0.994   0.323         9.441         1    .002
  2002                 0.786   0.214        13.539         1   <.001
  2003                -0.040   0.180         0.048         1    .826
  2004                                                     4    .001
NPTE version 2         2.084   0.252        68.232         1   <.001
DPT                   -0.127   0.322         0.155         1    .694
Age                    0.031   0.017         3.426         1    .064
Race or ethnicity
  Other                0.767   0.328         5.464         1    .019
  African              0.882   0.280         9.940         1    .002
    American
  Asian/Pacific        0.872   0.227        14.829         1   <.001
    Islander
  Hispanic            -0.138   0.412         0.111         1    .739
  White/non-                                               4   <.001
    Hispanic
Male sex              -0.108   0.129         0.705         1    .401

uGPA                  -0.126   0.020        39.902         1   <.001
vGRE                  -0.068   0.010        47.167         1   <.001
gGRE                  -0.036   0.008        18.817         1   <.001
Constant               5.397   1.028        27.537         1   <.001

Test                                       [chi square]   df   P
  Overall model fit
    Likelihood                             522.599        34   <.001
      ratio test
  Goodness-of-
      fit tests
    Hosmer and                               9.074         8    .336
      Lemeshow
  Nagelkerke           0.278
      [r.sup.2]

Predictor             Odds Ratio (95% CI)

Program
  1                    2.060 (0.621-6.838)
  2                    7.061 (2.884-17.290)
  3                    6.589 (2.408-18.045)
  4                    1.221 (0.347-4.291)
  5                    6.802 (2.421-19.116)
  6                    2.417 (0.989-5.904)
  7                    9.744 (4.211-22.547)
  8                    4.456 (1.635-12.144)
                       3.185 (1.268-7.998)
  10                   5.394 (1.829-15.911)
  11                   6.625 (3.045-14.413)
  12                   9.034 (4.106-19.876)
  13                   8.507 (3.590-20.157)
  14                   3.704 (1.614-8.499)
  15                   6.665 (3.023-14.692)
  16                   3.854 (1.463-10.153)
  17                   8.328 (2.958-23.445)
  18                   1.122 (0.464-2.711)
  19                   3.700 (1.474-9.289)
  20                   1.000 (referent)
Cohort
  2000                 2.749 (1.420-5.320)
  2001                 2.701 (1.433-5.092)
  2002                 2.194 (1.444-3.335)
  2003                 0.961 (0.676-1.368)
  2004                 1.000 (referent)
NPTE version 2         8.035 (4.901-13.17)
DPT                    0.881 (0.468-1.657)
Age                    1.032 (0.998-1.067)
Race or ethnicity
  Other                2.152 (1.132-4.093)
  African              2.415 (1.396-4.177)
    American
  Asian/Pacific        2.393 (1.535-3.731)
    Islander
  Hispanic             0.871 (0.388-1.956)
  White/non-           1.000 (referent)
    Hispanic
Male sex               0.897 (0.388-1.956)

uGPA                   0.882 (0.848-0.917)
vGRE                   0.934 (0.917-0.953)
gGRE                   0.965 (0.950-0.965)
Constant              20.687
Test
  Overall model fit
    Likelihood
      ratio test
  Goodness-of-
      fit tests
    Hosmer and
      Lemeshow
  Nagelkerke
      [r.sup.2]

(a) Data from all programs (N=3,186). SE=standard error,
CI=confidence interval, DPT=Doctor of Physical Therapy degree,
vGRE=verbal Graduate Record Examination score, gGRE=quantitative
Graduate Record Examination score.

Table 4.
Receiver Operating Characteristic Curve Areas, National Physical
Therapy Examination Test Versions 1 and 2 (a)

       Area Under the Curve   SE
       (95% CI)

uGPA   0.674 (0.621-0.673)    0.013
vGRE   0.687 (0.660-0.715)    0.014
gGRE   0.663 (0.634-0.692)    0.015

(a) CI=confidence interval, SE=standard error, uGPA=undergraduate
grade point average, vGRE=verbal Graduate Record Examination score,
gGRE=quantitative Graduate Record Examination score.

Table 5.
Logistic Regression for Prediction of National Physical Therapy
Examination (NPTE) Failure Using Receiver Operating Characteristic
Curve Cut-points and Data From All Programs and Both NPTE Test
Versions (a)

                                               Wald
Predictor                 [beta]   SE [beta]   [chi square]   df

Program
  1                        0.353   0.536         0.432         1
  2                        1.783   0.425        17.594         1
  3                        1.335   0.441         9.167         1
  4                        0.026   0.619         0.002         1
  5                        1.599   0.418        14.636         1
  6                        0.459   0.432         1.129         1
  7                        2.120   0.420        25.527         1
  8                        1.270   0.499         6.479         1
  9                        0.913   0.453         4.049         1
  10                       1.410   0.447         9.942         1
  11                       1.724   0.381        20.431         1
  12                       2.152   0.391        30.279         1
  13                       1.968   0.428        21.179         1
  14                       1.049   0.410         6.534         1
  15                       1.746   0.389        20.098         1
  16                       1.323   0.485         7.456         1
  17                       1.844   0.405        20.728         1
  18                       0.125   0.443         0.080         1
  19                       1.078   0.470         5.255         1
  20                                            97.167        19
Cohort
  2000                     1.149   0.331        12.048         1
  2001                     1.099   0.319        11.847         1
  2002                     0.890   0.208        18.235         1
  2003                    -0.001   0.183        -0.001         1
  2004                                          25.144         4
NPTE version 2             2.074   0.254        66.611         1
Race or ethnicity          0.934   0.165        32.030         1
  other than white/
  non-Hispanic or
  Hispanic
uGPA of 3.49 or higher    -0.740   0.132        31.593         1
vGRE
  400 or lower             0.684   0.140        23.707         1
  490 or higher           -0.404   0.154         6.836         1
  410-480                                       44.446         1
gGRE of 530 or lower       0.825   0.129        40.644         1
  Constant                -5.169   0.488       112.035         1

Test                                           [chi square]   df
  Overall model fit
    Likelihood                                 482.127        29
      ratio test
  Goodness-of-fit tests
    Hosmer and                                   5.394         8
      Lemeshow
  Nagelkerke [r.sup.2]     0.264

Predictor                 P       Odds Ratio (95% CI)

Program
  1                        .511   1.423 (0.497-4.069)
  2                       <.001   5.948 (2.586-13.685)
  3                        .002   3.801 (1.601-9.023)
  4                        .966   1.027 (0.305-3.455)
  5                       <.001   4.946 (2.181-11.220)
  6                        .288   1.583 (0.678-3.693)
  7                       <.001   8.328 (3.660-18.951)
  8                        .011   3.562 (1.339-9.474)
  9                        .044   2.491 (1.024-6.058)
  10                       .002   4.095 (1.705-9.836)
  11                      <.001   5.609 (2.656-11.847)
  12                      <.001   8.604 (3.997-18.518)
  13                      <.001   7.158 (3.096-16.552)
  14                       .011   2.856 (1.277-6.384)
  15                      <.001   5.732 (2.672-12.298)
  16                       .006   3.756 (1.453-9.711)
  17                      <.001   6.323 (2.858-13.987)
  18                       .777   1.134 (0.476-2.700)
  19                       .022   2.938 (1.169-7.382)
  20                      <.001   1.000 (referent)
Cohort
  2000                     .001   3.154 (1.694-6.035)
  2001                     .001   3.003 (1.605-5.616)
  2002                    <.001   2.432 (1.619-3.664)
  2003                     .998   1.000 (0.699-1.432)
  2004                    <.001   1.000 (referent)
NPTE version 2            <.001   7.959 (4837-13.099)
Race or ethnicity         <.001   2.554 (1.841-3.515)
  other than white/
  non-Hispanic or
  Hispanic
uGPA of 3.49 or higher    <.001   0.477 (0.369-0.617)
vGRE
  400 or lower            <.001   1.981 (1.505-2.609)
  490 or higher            .009   0.668 (0.493-0.904)
  410-480                 <.001   1.000 (referent)
gGRE of 530 or lower      <.001   2.283 (1.771-2.942)
  Constant                <.001   0.006

Test                      P
  Overall model fit
    Likelihood            <.011
      ratio test
  Goodness-of-fit tests
    Hosmer and             .715
      Lemeshow
  Nagelkerke [r.sup.2]

(a) Data from all programs (N=3,186). SE=standard error, CI= confidence
interval, uGPA=undergraduate grade point average, vGRE=verbal Graduate
Record Examination score, gGRE=quantitative Graduate Record Examination
score.

Table 6.
Prediction Rule for National Physical Therapy Examination (NPTE)
Performance (a)

Predictor                Prediction   Proportion of Students
                         Rule         Who Failed the NPTE at
                         Score (b)    Least Once(95% CI)

Race or ethnicity other  +1.0          90/388=0.232 (0.190-0.274)
  than white/non-
  Hispanic or Hispanic
uGPA of 3.49 or higher   -0.5         123/1579=0.078 (0.065-0.091)
vGRE
  400 or lower           +0.5         167/629=0.266 (0.231-0.300)
  490 or higher          -0.5          81/1,321=0.061 (0.048-0.074)
gGRE of 530 or lower     +1.0         178/716=0.249 (0.217-0.280)

(a) CI=confidence interval, uGPA=undergraduate grade point average,
vGRE=verbal Graduate Record Examination score, gGRE=quantitative
Graduate Record Examination score.

(b) Based on [beta] coefficients from logistic regression model in
Table 5; the model adjusts for test version, program, and cohort.

Table 7.
Prediction Rule Score for Predicting National Physical Therapy
Examination Performance

Score           Passed First Time          Failed at Least Once

                No.     Percentage         No.   Percentage
                        (95% CI (a))             (95% CI)

0 or less       1,998   93.0 (91.9-94.1)   151    7.0 (5.9-8.1)
0.5-1.0           623   80.8 (78.0-83.6)   148   19.2 (16.4-22.0)
1.5 or higher     162   60.7 (54.8-66.5)   105   39.3 (33.5-45.2)

(a) CI=confidence interval.