Printer Friendly

Do early career indicators of clinical skill predict subsequent career outcomes and practice characteristics for general internists?

Our study addresses a fundamental question, "Are early career clinical skill indicators among general internists associated with different career paths, leading them to treat different types of patients, practice in different locations, earn different incomes, and have different levels of career satisfaction?" To address this question, we merged measures of clinical skill recorded at the beginning of a general internist's career with measures of career outcomes and practice characteristics recorded throughout their career. Our study fills an important gap in the literature as there is little prior research on how physician capabilities and skill levels are associated with professional outcomes, practice characteristics, and patient panel attributes.


The Market for New General Internists

In the final year of medical school, medical students, including international medical school graduates (IMGs), apply and interview for U.S. residency program positions. Internal medicine residency programs differ in reputation, with those affiliated with university and medical school hospitals generally regarded as being more selective and of better quality than those run by community hospitals. (1)

Some medical students go directly into specialty residencies (e.g., surgical subspecialties), whereas many select internal medicine, which provides an opportunity to further subspecialize during fellowship programs that follow the 3-year internal medicine residency. Internal medicine residents wishing to become general internists (or who are unable to gain entry into a fellowship program) typically obtain employment at the end of their residency. In addition to interviews, hiring decisions are based on residency director and faculty recommendations, as well as the stature of the residency program. Residency program directors are required to rate all residents on a standardized set of competencies and overall performance in each year of training and prior to graduation from the program. Each resident is scored on a one to nine scale: 7-9 (superior), 4-6 (satisfactory), and less than 4 (unsatisfactory). A score of 4 or higher qualifies a resident for the board certification examination. Unsatisfactory evaluations are uncommon (Papadakis et al. 2008).

Upon acquiring a position, new general internists will typically be expected to pass the internal medicine board certification examination. For most physician practices, certification is a requirement of continued employment because many health plans and hospitals use board certification as part of their credentialing for plan networks and admitting privileges, respectively (Cassel and Holmbe 2006; Freed et al. 2006a, b; Freed, Dunham, and Singer 2009). Military physicians receive a bonus for certification (Gray and Grefer 2011). Board certification examinations can be taken multiple times if not initially passed. The examination focuses on clinical knowledge and judgment, and passage indicates that minimal competency has been achieved. American Board of Internal Medicine (ABIM) shares board certification status with employers, payers, hospitals, and other organizations. Furthermore, websites that rate physician quality (e.g., Health Grades) typically include information on board certification status. However, the examination score and number of times it took to pass the examination are not divulged by ABIM to the public. Although an internist may share his or her own board scores with potential employers, this is not routinely done.

Theoretical Literature

The theoretical literature describing physician labor markets considers how physician quality of care affects patients' perceived reservation prices (willingness to pay) and physicians' willingness to accept patients. The physician services market is characterized by asymmetric information between patients and physicians and by insurance, which alters the prices faced by patients (Gaynor 1994). This literature speaks to effects that these factors have on patient search costs and benefits, and their impact on equilibrium prices, quality of care, and quantity of services provided. The theory implies that the demand elasticity with respect to price is negatively associated with physician quality, a relationship that is strengthened in markets where consumer prices are subsidized by insurance (Gaynor 1994). This implies that higher quality physicians command higher prices than do lower quality physicians, all else held equal. Although higher prices imply higher physician incomes (Laugesen and Glied 2011), the ability of physicians to induce demand for their services, including altering the mix of services they provide toward those with higher margins, weakens the relationship between physician quality and compensation. Insurance companies provide a check in two ways (Gaynor 1994). Due to returns to scale in gathering physician quality information, they help ameliorate information asymmetry effects by exercising some control over the appropriateness of services for the treatment of specific conditions as well as gauging overall quality of physicians (Dranove and Satterthwaite 1992). In terms of our study, these factors imply that higher quality will be related to higher reimbursements for physicians and insurance providers have an incentive to set minimum quality standards for physicians in their networks (Roemer 1961; Evans 1974; Pauly 1980). As fees for treating disadvantaged patients are often lower than for treating other populations, this theory also implies that disadvantaged patients will receive lower quality of care (Gray 2001).

More broadly, the labor economics literature argues that higher skills resulting from greater experience, training, or innate ability are rewarded with higher pecuniary and nonpecuniary compensation (Ehrenberg and Smith 2003). Thus, the compensation package and other nonpecuniary aspects of the practice position will affect the degree of competitiveness for that position. This implies that physician skills may be low in practices dominated by Medicaid insured patients as Medicaid fees tend to be lower than private insurance or Medicare fees. However, some highly skilled physicians may have a preference for treating disadvantaged patients (and other practice characteristics) that would act to reverse this association (Gray 2001).


Our study uses a unique dataset that combines physician information from the 2000-2001 and 2004-2005 CTS Physician Surveys with information on residency program evaluations and board certification scores from the ABIM. The CTS survey sampled nonfederal, patient-care physicians providing at least 90 hours per week in direct patient care. The sample was clustered in 60 nationally representative sites. A total of 12,408 and 6,627 physicians were interviewed in the two surveys, with response rates of 58.6 and 52.4 percent, respectively. The survey was conducted for the Center for Studying Health System Change using computer-assisted telephone interviews and was funded by the Robert Wood Johnson Foundation. (2)

The two data sources were linked using personal information from the survey's sampling frame (the American Medical Association and American Osteopathic Association Masterfiles) and administrative information from ABIM. Because the survey reinterviews a portion of each round's sample from physicians sampled in the previous round, we have observations at two points in time for some physicians. The matched sample had 2,331 observations, representing 1,810 unique general internists. The survey provided outcome and control variables, whereas ABIM provided physician skill measures.


Outcome Measures. Physician Income: Survey respondents reported net annual income from medical practice in the prior year. Results are expressed in 2005 dollars.

Physician Career Satisfaction: Career satisfaction was asked on the CTS survey as a five-level Likert scale ranging from very satisfied to very dissatisfied. We dichotomized the variable to indicate those very or somewhat satisfied with their career versus all others.

Practice Setting: Most general internists practice in group practices. We categorized practice settings by indicating whether the physician was in a solo, group, or hospital/academic health center practice. These practice types vary in terms of the patient populations, the degree of physician autonomy, control over work hours, compensation methods, and prestige.

Patient Panel: We measured the race/ethnicity and insurance characteristics of patients treated by sample physicians. Our measures focus on the degree physicians treat underserved populations, represented by percent of practice revenue from Medicaid and percent of the physician's patients who are minorities (black, Hispanic, and Native American). The latter measure is only available from the 2004 2005 CTS survey, so sample sizes and statistical power to detect differences will be lower for this outcome.

Practice Location: Our measure of location was whether a physician practiced in a nonmetropolitan area. We choose this measure because many rural areas often face physician shortages, presumably because most physicians view rural location as disadvantageous relative to urban location (Reschovsky and Staiti 2005).

Early Career Clinical Skill Measures. We used two sets of dichotomous clinical skill measures, both measured at the start of a general internist's career. The first was a five-level variable that represents an interaction between the score on the physician's first attempt at the internal medicine board certification examination and whether the physician eventually achieved board certification. Physicians were divided into quartile groups, based on the score on their initial examination attempt relative to all physicians taking the examination for the first time that year. Because higher scoring physicians were more likely to enter fellowship programs, greater proportions of general internists are found in lower scoring quartiles than higher scoring ones. Some of those scoring in the lowest quartile of their initial examination either barely passed the examination or passed subsequent examinations. Because of this, general internists scoring in the lowest quartile were further categorized as to whether they were board certified at the time of the CTS interview. The omitted reference group is internists scoring in the lowest quartile who achieved board certification either through the initial examination or subsequent tries.

The second measure indicates whether the residency program was academic health center-based (hereafter "academic"), or community hospital-based, and the residency director's evaluation of the physician, which is categorized between those evaluated as superior versus satisfactory. Academic programs are generally regarded as being more prestigious, although many factors such as location and specific attributes of a program might make a community-based residency program more attractive to a specific applicant.

We formed an interaction term for these two dichotomous variables to create a four-level categorical variable. The reference group was physicians who received satisfactory evaluations from community hospital-based residency programs.

Because many patients value physician interpersonal skills and are less able to assess clinical skills, we also tested a measure constructed from one component of the residency evaluation that assesses physicians' interpersonal skills and humanistic values, on the assumption that this might be most reflective of patient satisfaction (Fung et al. 2005). The measure was consistently insignificant, its inclusion did not materially affect other model coefficients, and it was ultimately dropped from our models.

Model Specifications. Control variables in all regressions included indicators for physician gender, race/ethnicity (white, black, Hispanic, other), international medical graduates, year of observation, and years in practice (<4, 4-10,>10). Cutoff values for years of experience were based on the fact that it may take several years for physicians to achieve board certification or conclude that further attempts are likely futile. The other two categories were formed to divide the remaining observations into groups of roughly equal size. For the income regression, we additionally controlled for physicians' labor supply (i.e., hours worked per week and number of weeks worked in the previous year). (34)

We tested two alternative sets of control variables, to understand both direct and indirect relationships between our dependent measures and our skill measures. We included all practice characteristic dependent variables as control variables in the income and career satisfaction regressions, and the subset unrelated to the dependent variable in the other practice characteristic equations (e.g., practice type, share of revenue from Medicaid). Compared with our core model, coefficients on physician skill measures in these models reflect direct associations with our dependent measures and omit indirect influences through the effect on other practice characteristics. We also tested the inclusion of site fixed effects (except for the nonmetropolitan location equation) to capture unmeasured local market factors. Coefficients on physician skill variables in these equations reflect associations within the context of the local market chosen by the physician, but nets out the effect of skill indicators on the choice of local market.

Results of these two alternative regression specifications were generally robust and did not change conclusions. They are shown in Appendix Table A2. (5)

Apart from including years of experience in our core model, we also tested the persistence of physician early career skills on outcomes by estimating models that included interactions between the years of experience dummy variables with clinical skill variables.

Estimation. We estimated regressions for each career outcome and practice characteristic measure. The nature of the dependent measure dictated functional form: ordinary least squares for the income regression, logistic for dichotomous outcomes (nonmetropolitan location and career satisfaction), multinomial logit for practice type (solo, hospital/medical school, with group practice as reference group), generalized linear model with a logit link, and the binomial family for outcomes expressed as proportions (percent of practice revenue from Medicaid, percent of patients who are minorities).

For all models, we expressed associations between our clinical skill measures and our outcome measures in terms of marginal effects across our sample population. To do this, we simulated the change in the dependent variable (and standard error) when moving our clinical skill measures from zero to one across each physician in our sample (Karaca-Mandic, Norton, and Dowd 2012). For the income regression, marginal effects are in dollars and come directly from OLS coefficients. For all other regressions, marginal effects are expressed as percentage point changes. In models in which interactions between years of experience and clinical skill measures were included, marginal effects and their standard errors were calculated using the technique described by Karaca-Mandic, Norton, and Dowd (2012).

For all regressions, we used survey weights and statistical software that provided variance estimates that accounted for the complex CTS sample and multiple observations of some physicians (Center for Studying Health System Change 2006).


Sample Characteristics

Table 1 shows sample characteristics, including those of our dependent variable measures. Our general internal medicine physician sample was 68 percent male, 65 percent white, with an average of 11 years in practice. The average income of our sample was $153,299 ($2005). About one in 10 were solo practitioners (10.8 percent) and a fifth worked in an academic health center or hospital (21.5 percent). One third of general internists' patients were minorities and 12.4 percent practiced in nonmetropolitan areas. Average practice revenue from Medicaid was 13.0 percent; 78 percent reported that they were somewhat or very satisfied with their career.

Clinical Skill Measure Distribution

Table 2 shows the distribution of our skill measures as well as how they correspond with one another. Only nine percent of our general internists sample reported not being board certified, although another 20.4 percent scored in the bottom quartile on their initial board certification examination attempt but achieved certification. About one in five scored in the highest quartile. With respect to the residency program/evaluation measure, about four in ten physicians went through community residency programs. Roughly three in ten received superior evaluations, regardless of residency type. Judged against general internists who completed community hospital-based residencies, those completing academic residency programs were more likely to be board certified and somewhat more likely to score in the highest board examination score quartile. However, superior residency evaluations were more strongly associated with certification and initial examination scores than type of residency program. Although initial board examination scores and board certification were correlated with type of residency program and residency program director evaluation, there remains a considerable variation in scores within each residency/evaluation group and coefficient estimates for our variables of interest were similar when only one of the sets of skill measures were included in our regressions instead of both.

Regression Model Results

Table 3 shows regression results for our early career skill measures. In addition to showing marginal effect estimates and statistical significance relative to reference groups, we also included indicators of statistical significance for examination performance indicators relative to non-board-certified physicians rather than board-certified physicians who tested in the bottom quartile. Full regression results are available in Appendix Table A1.

Income. Compared with board-certified physicians who scored in the lowest quartile on the initial board examination, there was no significant difference in income for those with higher examination scores. However, lacking board certification was associated with over $27,000 (18 percent) less income than physicians who also scored in the bottom quartile on their first board examination attempt but who obtained board certification (p < .05). Overall, higher scoring general internists had significantly higher incomes than uncertified general internists did. Residency program type or evaluation did not have a statistically significant independent association with incomes.

Career Satisfaction. Being a general internist who scored in the highest quartile of the board examination was associated with a nearly 10 percentage point increased likelihood of reporting being satisfied with their medical career than those who scored in the lowest quartile. This association was also significant when the reference group was physicians who were not board certified (p < .05). Other clinical skill measures did not have statistically significant effects.

Practice Setting. Physicians scoring in the top three quartiles of their initial board certification examination were between 8.5 and 9.6 percentage points less likely to be solo practitioners than certified physicians who scored in the lowest quartile (p < .05). Consistent with the inverse relationship between board examination scores and likelihood of being a solo practitioner, being a physician trained in an academic residency program was significantly associated with a reduced likelihood of being a solo practitioner compared with physicians who trained in community hospital residency programs and received only a satisfactory evaluation (marginal probability = 10.4 and 9.0 percent for those evaluated as superior and satisfactory, respectively [p < .05]). Conversely, physicians with the highest early career clinical skills, as indicated by scoring in the top quartile on their board examination as well as receiving a superior evaluation from an academic residency program, were more likely to end up practicing in hospitals or academic health centers than those with scores in the bottom quartile or who received a satisfactory evaluation from a community hospital-based residency program, respectively.

Treating Disadvantaged Patients. Being a general internist from a community hospital residency program and evaluated as satisfactory was associated with a 2-4 percentage point increased share of revenue from Medicaid than other physicians (p < .05). These physicians were estimated to have patient panels consisting of a 14.5 percent greater share of minority patients than those from similar programs who achieved superior residency director evaluations (p < .05). Moreover, being a physician who scored in the lowest quartile during their initial board examination attempt and failing to achieve board certification was associated with nearly a 14.9 percent larger share of minority patients than those who become board certified (p < .01).

Practice Location. No early career skill indicators were significantly associated with nonmetropolitan location.

Persistence of Relationships over a Physician's Career

Our data included observations on physicians at various time points during their careers. As a control variable, years of experience was associated with greater income, negatively associated with the treatment of Medicaid patients, and had inconsistent effects on the likelihood of practicing in a solo practice. When models were estimated in which we included interactions between years of experience and each of our early career skill measures, nearly all interaction terms failed to reach statistical significance (Appendix Table A3). There were some notable exceptions. As expected, the income penalty associated with lack of board certification appears to be realized beyond the initial 0- to 3-year period when physicians are typically taking board examinations. The estimated marginal effect of not being board certified between the 4- to 10-year period and this initial period was -$28,524 (p < .05). Although the income penalty appears to continue to grow with years of experience, the change in the marginal effect of being board certified between the 4- to 10-year and greater than 10-year period was not statistically significant.

Physicians with community hospital residencies who received satisfactory evaluations were significantly less likely to express satisfaction with their career than other physicians in the greater than 10-year period relative to the 4- to 10-year period.


This study addressed whether early career indicators of clinical skill among general internists were associated with different career paths, leading them to treat different types of patients, operate in different practice settings, earn different incomes, and report different levels of career satisfaction. We related cognitive- and clinical-based measures of clinical skill among general internists at the beginning of their career with measures of practice characteristics and career outcomes gathered throughout their careers. In general, we found that being a bottom performer on our early career clinical skills measures was related to physicians' practice characteristics and that not being board certified was associated with lower incomes. Moreover, we found that these relationships were generally persistent over the course of a physician's career. (6) Finally, with the exception of career satisfaction, top performers on our skill measures generally did not differentiate themselves from middle-range performers. One possible explanation for why we did not find a relationship between being a top performer on our skill measures and income but did find such a relationship for career satisfaction is that we cannot account for non-pecuniary benefits received from different work circumstances. Top performers on board examinations likely had the option to subspecialize in higher paying procedural subspecialties of internal medicine (Gray, Park, and Lipner 2010). Presumably, these physicians not only had greater range of primary care job opportunities but chose primary care over high-paying subspecialties for the nonpecuniary benefits they ascribed to primary care.

Because our analyses are cross-sectional, it should be noted that the associations we present do not imply causation. For instance, the strong association between top board certification examination performance and reported career satisfaction could reflect unmeasured behavioral characteristics, such as motivation, that are correlated with both examination performance and career satisfaction. If so, an overall improvement in initial skill levels resulting from an improvement in medical education may not result in greater career satisfaction. From an assessment perspective, this distinction is not important.

Another issue to consider is the validity of our clinical skill measures as predictors of care quality among practicing internists. Our measures of physician skill are heavily focused on clinical knowledge and judgment (Lipner and Lucey 2010) at the beginning of an internist's career. Although clearly not a direct measure of care quality, several studies validate our two skill measures, both as predictors of quality at the start and over the course of a physician's career. Haber and Avins (1994), Papadakis et al. (2008), and Durning, Cation, and Jackson (2007) illustrate that quality of residency program (academic vs. community) and residency evaluations (better vs. worse) correspond with independent assessments of clinical quality and even disciplinary action by state licensing boards later in a physician's career. Across multiple specialties, a number of studies indicate that board certification is associated with subsequent measures of physician quality, including patient outcomes (Norcini, Lipner, and Kimball 2001, 2002; Sharp et al. 2002; Bach et al. 2004; Chen et al. 2006; Holmboe et al. 2008; Curtis et al. 2009). For instance, certification has been found to be associated with lower mortality following heart attacks (Kelly and Hellinger 1987; Norcini and Lipner 2000; Norcini et al. 2001), whereas other work has found differences in the use of appropriate medications (Chen et al. 2006). Scores on initial board examinations have been associated with physician clinical outcomes (Brennan et al. 2004) as have scores from maintenance of certification examinations (Holmboe, Lipner, and Greiner 2008). Among internal medicine physicians who passed the board certification examination, the initial board certification examination score is highly correlated with the maintenance of certification examination score taken 10 years later (r = .65). (7) Board examination scores have been found to be related to greater preventive care and more appropriate prescribing patterns (Tamblyn et al. 2002, 2008).

A drawback of our measures of clinical skill is that we lack discrete information on physicians' interpersonal and communication skills, both important contributors to patient satisfaction (Gray, Holmboe, and Weng 2012). Although these dimensions enter into residency director evaluations, the component of residency director evaluations specifically addressing these skills was not found to be independently related to career outcomes.

The fact that we measure skill at the beginning of a physician's career is both an advantage and a disadvantage. More contemporaneous measures of physician quality could show stronger associations with career outcomes and practice characteristics. However, we found that the relationship between these outcomes and initial career skill signals was, for the most part, very persistent over a physician's career. There are several potential explanations for this persistence: (1) physicians effectively sort themselves into practice settings they prefer early in their careers; (2) there are large costs associated with changing the nature of one's job among physicians in clinical practice; (3) early career signals (especially type of residency program) figure heavily in hiring decisions later on in a physician's career; or (4) coupled with past research which reports that the early signals of physician skill we apply in this study are correlated with physician quality more generally, our findings suggest that lower skilled physicians may be persistently constrained in their job opportunities as a result of their lower care quality.

Board certification was found to be associated with income (18 percent less among noncertified physicians compared with similar physicians who were certified). This effect could be in part attributable to the fact that among our skill measures, board certification is most visible to patients and other market actors. Identifying the precise mechanism by which board certification affects income is beyond this study. However, board certification is often used for health plan network selection and hospital privilege credentialing, as well as in hiring/retention decisions (Cassel and Holmbe 2006). Health plans, hospitals, and medical groups, at least in part, act as agents for patients by restricting access to low-skilled physicians.

We found that noncertified physicians and those trained in community residency programs who only achieved satisfactory evaluations treated a larger share of minority patients than physicians who also had low scores on their initial board examination but achieved certification or were given superior evaluations from a community residency program, respectively. This suggests a potential source of persistent racial, ethnic, and socioeconomic disparities in health care whereby lower skilled physicians tend to treat greater numbers of disadvantaged patients. These findings are consistent with other research that has shown minority patients are heavily concentrated into a relatively small number of high-minority practices, in which physicians are less likely to be board certified and face greater challenges providing high-quality care (Bach et al. 2004). However, other results found no relationship with skill indicators.

In summary, results regarding our most visible signal of quality, board certification, provide some support of the theoretical predictions concerning the relationship between physician clinical skill and income. However, we found little relationship between income (presumably related to prices for physician services) and early career indicators of physician skill above these minimum standards. This finding, combined with the long-term relationship reported in the literature between our skill measures and quality indicators among practicing physicians, suggests that either patients/insurers value other physician attributes that are not correlated with our early career skill measures, or that these quality differences are unseen. A related explanation for this finding is the predominance of fee-for-service reimbursement that rewards productivity as measured by the quantity of services provided, rather than a system that rewards physicians for clinical performance (Reschovsky, Hadley, and Landon 2006). From a wider perspective, beyond meeting minimum standards, the weak relationship between our measures of physician skills and compensation for certified general internists suggests that the current primary care physician labor market offers few financial rewards for being a more highly skilled physician.

DOI: 10.1111/1475-6773.12011


Joint Acknowledgment/Disclosure Statement: Financial support for the analysis was provided by the American Board of Internal Medicine (ABIM). The authors did not seek or receive approval by ABIM for the content of the paper. All computations on microdata were prepared by the first author, and the responsibility for the use and interpretation of these data is entirely that of the authors. The CTS Physician Survey was funded by the Robert Wood Johnson Foundation. The authors wish to thank Beny Wu, formerly of Social and Scientific Systems, Inc., for creating the linked dataset.

Disclosures: None.

Disclaimers: None.


Bach, P. B., H. H. Pham, D. Schrag, R. C. Tate, and J. L. Hargraves. 2004. "Primary Care Doctors Who Treat Blacks and Whites." New England Journal of Medicine 351 (6): 575-84.

Brennan, T. A., R. I. Horwitz, F. D. Duffy, C. K. Cassel, L. D. Goode, and R. S. Lipner. 2004. "The Role of Physician Specialty Board Certification Status in the Quality Movement." Journal of the American Medical Association 292 (9): 1038-43.

Cassel, C., and E. Holmbe. 2006. "Professional Standards in the USA: Overview and New Developments." Clinical Medicine 6 (4): 363-7.

Center for Studying Health System Change. 2006. Physician Survey Restricted Use File: User's Guide, 2004-05 (Round 4) Technical Publication No. 66. Washington, DC: Center for Studying Health System Change [accessed on October 11, 2012]. Available at

Chen, J., S.S. Rathore, Y. Wang, M.J. Radford, and H. M. Krumholz. 2006. "Physician Board Certification and the Care and Outcomes of Elderly Patients with Acute Myocardial Infarction." Journal of General Internal Medicine 21 (3): 238-44.

Curtis, J. P., J. J. Luebbert, Y. Wang, S. S. Rathore, J. Chen, P. A. Heidenreich, S. C. Hammill, R. I. Lampert, and H. M. Krumholz. 2009. "Association of Physician Certification and Outcomes among Patients Receiving an Implantable Cardioverter-Defibrillator." Journal of the American Medical Association 301 (16): 1661-70.

Dorner, F. H., R. M. Burr, and S. L. Tucker. 1991. "The Geographic Relationships between Physicians' Residency Sites and the Locations of Their First Practices. "Academic Medicine 66: 540-44.

Dranove, D., and M. A. Satterthwaite. 1992. "Monopolistic Competition When Price and Quality Are Not Perfectly Observable." Rand Journal of Economics 1: 8-534.

Durning, S.J., L.J. Cation, and J. L. Jackson. 2007. "Are Commonly Used Resident Measurements Associated with Procedural Skills in Internal Medicine Residency Training?" Journal of General Internal Medicine 22 (3): 357-61.

Ehrenberg, R. G., and R. S. Smith. 2003. Modern Labor Economics, Theory and Public Policy, Chapter 9, 8th Edition. Boston: Addison Wesley.

Evans, R. A. 1974. "Supplier-Induced Demand: Some Empirical Evidence and Implications." In The Economics of Health and Medical Care, edited by M. Perlman, pp. 162-73. London: MacMillan.

Freed, G. L., K. M. Dunham, and D. Singer. 2009. "Health Plan Use of Board Certification and Recertification of Surgeons and Nonsurgical Subspecialists in Contracting Policies." Archives of Surgery 144 (8): 753-8.

Freed, G. L., D. Singer, I. Lakhani, J. R. C. Wheeler, and J. A. Stockman. 2006a. "Use of Board Certification and Recertification of Pediatricians in Health Plan Credentialing Policies." Journal of the American Medical Association 295 (8): 913-8.

Freed, G. L., R. L. Uren, E.J. Hudson, I. Lakhani, J. R. C. Wheeler, and J. A. Stockman III. 2006b. "Policies and Practices Related to the Role of Board Certification and Recertification of Pediatricians in Hospital Privileging." Journal of the American Medical Association 295 (8): 905-12.

Fung, C.J., M. N. Elliott, R. D. Hays, K. L. Kahn, D. E. Kanouse, E. A. McGlynn, M. D. Spranca, and P. G. Shekelle. 2005. "Patients' Preferences for Technical versus Interpersonal Quality When Selecting a Primary Care Physician." Health Services Research 40: 957-77.

Gaynor, M. 1994. Issues in the Industrial Organization of the Market for Physician Services. National Bureau of Economic Research Working Paper Series. Cambridge: National Bureau of Economic Research.

Gray, B. 2001. "Do Medicaid Physician Fees for Prenatal Services Affect Birth Outcomes?" Journal of Health Economics 20 (4): 571-90.

Gray, B., and J. Grefer. 2012. "Career Earnings and Retention of U.S. Military Physicians." Journal of Defense and Peace Economics 23: 51-76.

Gray, B., E. Holmboe, and W. Weng. 2012. "An Assessment of Patient Based and Practice Infrastructure Based Measures of the Patient Centered Medical Home. Do We Need to Ask the Patient?" Health Services Research 47(1): 4-21.

Gray, B., J. Park, and R. Lipner. 2010. "What Factors Predict Primary Care Choice among Top Performing Physicians in Internal Medicine?" Working Paper Series. Philadelphia, PA: American Board of Internal Medicine.

Haber, R. J., and A. L. Avins. 1994. "Do Ratings on the American Board of Internal Medicine Resident Evaluation Form Detect Differences in Clinical Competence?" Journal of General Internal Medicine 9 (3): 140-5. doi: 10.1007/ BF02600028

Holmboe, E. S., R. Lipner, and A. Greiner. 2008. "Commentary: Assessing Quality of Care: Knowledge Matters." Journal of the American Medical Association 299: 338-40.

Holmboe, E. S., Y. Wang, T. P. Meehan, J. P. Tate, S. Y. Ho, K. S. Starkey, and R. S. Lipner. 2008. "Association between Maintenance of Certification Examination Scores and Quality of Care for Medicare Beneficiaries." Archives Internal Medicine 168 (13): 1396-403.

Karaca-Mandic, P., E. C. Norton, and B. Dowd. 2012. "Interaction Terms in Nonlinear Models." Health Services Research 47 (10 Pt I): 255-73.

Kelly, J. V., and F. J. Hellinger. 1987. "Heart Disease and Hospital Deaths: An Empirical Study." Health Services Research 22 (3): 369-95.

Laugesen, M. J., and S. A. Glied. 2011. "Compared to Other Countries Higher Fees Paid to US Physicians Drive Higher Spending For Physician Services." Health Affairs 30 (9): 1647-56.

Lipner, R. S., and C. R. Lucey. 2010. "Putting the Secure Examination to the Test." Journal of the American Medical Association 304 (12): 1379-80.

Norcini, J. J., H. R. Kimball, and R. S. Lipner. 2000. "Certification and Specialization: Do They Matter in the Outcome of Acute Myocardial Infarction?" Academic Medicine 75 (12): 1193-8.

Norcini, J. J., and R. S. Lipner. 2000. "The Relationship between the Nature of Practice and Performance on a Cognitive Examination." Academic Medicine 75: 868-70.

Norcini, J.J., R. Lipner, and H. R. Kimball. 2001. "The Certification Status of Generalist Physicians and the Mortality of their Patients after Acute Myocardial Infarction." Academic Medicine 76: S21-3.

Norcini, J.J., R. S. Lipner, and H. R. Kimball. 2002. "Certifying Examination Performance and Patient Outcomes Following Acute Myocardial Infarction." Medical Education 36(9): 853-9.

Papadakis, M. A., G. K. Arnold, L. L. Blank, E. S. Holmboe, and R. S. Lipner. 2008. "Performance during Internal Medicine Residency Training and Subsequent Disciplinary Action by State Licensing Boards." Annals of Internal Medicine 148 (11): 869-76.

Pauly, M. V. 1980. Doctors and Their Workshops. Chicago: University of Chicago Press.

Reschovsky, J., J. Hadley, and B. Landon. 2006. "The Effects of Compensation Methods and Physician Group Structure on Physicians' Perceived Incentives to Alter Services to Patients." Health Services Research 41 (4 Pt 1): 1200-20.

Reschovsky, J. D., and A. Staiti. 2005. "Physician Incomes in Rural and Urban America?" Issue Brief No. 92. Washington, DC: Center for Studying Health System Change [accessed on November 7, 2011]. Available at content/725/

Roemer, M. I. 1961. "Bed Supply and Hospital Utilization: A National Experiment." Hospitals 35:36-42.

Sharp, L. K., P. G. Bashook, M. S. Lipsky, S. D. Horowitz, and S. H. Miller. 2002. "Specialty Board Certification and Clinical Outcomes: The Missing Link." Academic Medicine 77 (6):534-42.

Tamblyn, R., M. Abrahamowicz, W. D. Dauphinee, J. A. Hanley, J. Norcini, N. Girard, P. I. Grand'Maison, and C. Brailovsky. 2002. "Association between Licensure Examination Scores and Practice in Primary Care." Journal of the American Medical Association 288 (23): 3019-26.

Tamblyn, R., M. Abrahamowicz, C. Brailovsky, P. Grand'Maison, J. Lescop, J. Norcini, N. Girard, and J. Haggerty. 2008. "Association between Licensing Examination Scores and Resource Use and Quality of Care in Primary Care Practice." Journal of the American Medical Association 280 (11): 989-96.


Additional supporting information may be found in the online version of this article:

Appendix SA1: Author Matrix.

Appendix SA2: Full Model Results.

Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.


(1.) A computerized program that considers applicant and residency program preferences is used to match nearly all applicants with residency programs.

(2.) More information about the 2000/01 and 2004/05 CTS Physician Surveys can be found in technical documentation available at func=pubs&what=8

(3.) Results were not materially impacted by exclusion of these labor supply measures.

(4.) Missing value dummies were used for several variables (residency evaluation, IMG, physician race/ethnicity), where small amounts of missing data occurred.

(5.) The Appendix is available in the online version available on Wiley Online Library.

(6.) The relationship between our quality measures and employment in academic medical centers or hospitals we found is confirmed by prior research indicating graduates of academic residency programs are more likely to be hired by the hospital(s) associated with them (Dorner, Burr, and Tucker 1991).

(7.) Based on internal tabulations at ABIM.

Address correspondence to Bradley Michael Gray, Ph.D., American Board of Internal Medicine, 501 Walnut Street, Suit 1700, Philadelphia PA, 19130; e-mail: Eric Holmboe, M.D., and Rebecca Lipner, Ph.D., are also with the American Board of Internal Medicine, Philadelphia, PA. James Reschovsky, Ph.D., is with the Center for Studying Health System Change, Washington, DC.

Table 1: Physician Characteristics
                                                Mean/%    Std. Error

% International medical school graduate           30.2         2.2
% Male                                            68.4         1.8
Hours per week worked                             56.4         0.6
Weeks worked prior year                           47.1         0.2
Years of practice (%)
Less than 5 years                                 21.6         1.4
5-9 years                                         27.0         1.5
10+ years                                         51.4         2.0
Practice size (%)
  Solo                                            10.8         1.1
Two physician practice                            29.6         2.2
  Group 3-5                                        5.8         0.8
  Group 6-10                                       6.5         0.8
  Group 11-50                                     15.0         1.5
  Group 50 plus                                   11.7         1.1
  Hospital/academic health center                 21.5         1.7
Physician race/ethnicity (%)
  Hispanic                                         8.0         1.3
  White non-Hispanic                              65.2         2.2
  Black                                            6.3         1.0
  Other                                           20.5         2.0
Mean income (2005 dollars)                   153,299       3,331
  % practice revenue from Medicaid                13.0         0.5
  % patients who are minorities                   33.1         1.7
  % practicing in non-metropolitan areas          12.4         2.2
  % somewhat or very satisfied with career
    in medicine                                   78.0         1.7

Note. N = 2,331.

Source. ABIM-CTS Physician Survey linked dataset.

Table 2: Quality Measure Distribution and Cross-Tabulation

                                             Board Certification
                                            Interacted with Initial
                                              Board Certification
                                               Examination Score

                             Distribution    Bottom       Bottom
                             of Residency   Quartile,    Quartile,
                                Type/       Not Board      Board
                              Evaluation    Certified    Certified
                               (Col. %)      (Row %)      (Row %)

All physicians                                 9.1         20.4
Residency type
    and evaluation
  Academic and
    superior evaluation          18.2          5.6         12.6
  Academic and
    evaluation                   41.3          8.0         22.2
Community and
  superior evaluation            11.3          1.5         11.0
Community and
  satisfactory evaluation        29.2         14.2         26.2

                                  Board Certification
                                Interacted with Initial
                                  Board Certification
                                   Examination Score

                               2nd        3rd        Top
                             Quartile   Quartile   Quartile
                             (Row %)    (Row %)    (Row %)

All physicians                 22.7       26.4       21.3
Residency type
    and evaluation
  Academic and
    superior evaluation        18.0       29.1       34.6
  Academic and
    evaluation                 26.7       24.5       18.6
Community and
  superior evaluation          15.7       40.7       31.1
Community and
  satisfactory evaluation      24.1       22.1       13.4

Note. N = 2,331 Residency measures were missing for
observations and 2.26% of exam 1.26% of scores were

Source. ABIM-CTS Physician Survey linked dataset.

Table 3: Regression Result Simulations

                                                Satisfied with
Physician Early                  Income         Medical Career
Career Skill Measures         (2005 Dollars)     (Marg. Prob)

Initial certification examination score quartile
    interacted with board certification
  Exam 4th quartile (best)        4,078 (a)        9.8% ** (a)
  Exam 3rd quartile               7,292 (b)       -0.7%
  Exam 2nd quartile              -2,340 (a)        5.2% (b)
  Bottom quartile and
    board certified               Ref.             Ref.
  Bottom quartile and
    not board certified         -27,206 **        -6.1%
Quality of residency program and physician evaluation
  Academic program
    & superior evaluation         5,036            4.6%
  Academic program
    & satisfactory evaluation       543            3.0%
  Community program
    & superior evaluation         9,650            6.5%
  Community program
    & satisfactory evaluation     Ref.             Ref.
No. of observations               2,300        2,304

Physician Early               Solo Practice      Ctr./Hospital
Career Skill Measures         (Marg. Prob.)      (Marg. Prob)

Initial certification examination score quartile
    interacted with board certification
  Exam 4th quartile (best)         -9.0% ** (c)       10.0% ** (b)
  Exam 3rd quartile                -8.5% ** (c)       -0.8%
  Exam 2nd quartile                -9.6% *** (b)       8.1% (b)
  Bottom quartile and               Ref.               Ref.
  board certified
  Bottom quartile and
    not board certified            -0.07%             11.3%
Quality of residency program and physician evaluation
  Academic program
    & superior evaluation         -10.4% **           10.9% **
  Academic program
    & satisfactory evaluation      -9.0% ***           8.4% ***
  Community program
    & superior evaluation          -6.8%               6.2%
  Community program
    & satisfactory evaluation        Ref.               Ref.
No. of observations             2,304              2,304

                               % Practice                  Practice
                                 Revenue        Percent    Location
Physician Early               from Medicaid    Minority     (Marg.
Career Skill Measures           (Marg. %)      (Marg. %)    Prob.)

Initial certification examination score quartile
   interacted with board certification
  Exam 4th quartile (best)          1.1%          4.3% (b)       2.9%
  Exam 3rd quartile                 0.6%         -0.6 (a)        2.9%
  Exam 2nd quartile                 0.5%          5.4% (c)      -0.9%
  Bottom quartile and               Ref.          Ref.           Ref.
    board certified
  Bottom quartile and               1.9%         14.9% ***      -1.6%
   not board certified
Quality of residency program and physician evaluation
  Academic program                 -2.3% *       -3.6%          -3.3%
    & superior evaluation
  Academic program                 -2.3% **      -4.7%          -2.5%
    & satisfactory evaluation
  Community program                -3.6% **     -14.5% **        3.8% *
    & superior evaluation
  Community program                 Ref.          Ref.          Ref.
    & satisfactory evaluation
No. of observations             2,329           701          2,328

*** p < 0.01; ** p < 0.05; * p < 0.10.

Significance level relative to non-board-certified general
internists: (a) p < 0-01; (b) P < 0.0,5; (c) P < 0.10.

Source. ABIM-CTS Physician Survey linked dataset.
COPYRIGHT 2013 Health Research and Educational Trust
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2013 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Gray, Bradley; Reschovsky, James; Holmboe, Eric; Lipner, Rebecca
Publication:Health Services Research
Geographic Code:1USA
Date:Jun 1, 2013
Previous Article:Mental illness, access to hospitals with invasive cardiac services, and receipt of cardiac procedures by Medicare acute myocardial infarction...
Next Article:Physician styles of patient management as a potential source of disparities: cluster analysis from a factorial experiment.

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters