Challenging the 10-year rule: The accuracy of patient life expectancy predictions by physicians in relation to prostate cancer management.
Decisions on patient management are often predicated by the physician's own clinical judgement. These decisions become increasingly difficult when factors, such as age of the patient, quality of life, comorbidities, tolerance to the treatment and long-term side-effects, become significant. In some conditions, such as prostate cancer, understanding the value of clinical judgement becomes crucial when deciding whether or not to screen.
In 2006, it was reported that life expectancy for males at age 65 is on average 19.9 years in Canada (ranging from 16.8-20.5 among the provinces and territories) and 17.0 years in the United States.[sup.1,2] Prostate cancer is the most prevalent cancer among men and represents 38% of all cancers.[sup.3] The lifetime probability of developing prostate cancer is 13.6% (1 in 7.4 men); 25 500 new cases are diagnosed every year in Canada.[sup.3] Furthermore, the lifetime chance of dying from prostate cancer is 3.7%.[sup.3]
In terms of screening, though there is significant controversy surrounding its value and need (Appendix A), those that do describe it often cite that patients should have at least a 10-year life expectancy.[sup.4]-[sup.7] Accuracy and precision regarding life expectancy are therefore important. In this regard, we wanted to see which physician groups do better at predicting life expectancy and we wanted to evaluate how well physicians predict life expectancy.
Respondents were randomly selected within demographic groups of interest. Specifically, we restricted our respondents to attending physicians and residents in urology, internal medicine and family medicine, as well as medical students. We included respondents who would be dealing with prostate cancer screening. All three levels of training were included to be able to later analyze if the accuracy of life expectancy predictions improved with training and experience. To maintain a high response rate, surveys were distributed directly by the primary author. A goal of 100 respondents was set and met.
Demographic information was collected and included gender, level of training, years in practice (if any), subspecialty (if any), place of work (if any), medical school, residency school and program (if any), and fellowship school and program (if any). We also presented seven case summaries and asked respondents to provide their estimation of life expectancy based on the information provided. These cases were based on data of real deceased patients (Appendix B).
A priori sample size calculations were not undertaken, as this was an observational study that did not involve a specific intervention or effect size. Rather, a convenience sample consisting of the first 100 respondents from the attending physicians and residents in urology, internal medicine, and family medicine, as well as medical students, was obtained. A sample size of 50 is commonly considered the minimum number of patients required for testing associations;[sup.8] therefore the current sample size of twice that is believed to be sufficient to support the use of inferential statistics.
Data were collected in Microsoft Excel 2007 and statistical analyses were performed using SPSS version 17.0 (SPSS, Cary, NC). Predicted life expectancy was subtracted from the actual life expectancy for all respondents and all cases, so that underestimated life expectancies were represented by negative values. Following a descriptive analysis, independent samples t-tests were used to compare the actual scores for 2-level variables, such as sex, while one-way analysis of variance was used to compare multilevel variables, such as level of training, specialty and working environment. Chi-square tests were used to compare the aggregated values and the characteristics of the respondents. Responses for the 7 cases for all 100 respondents were also aggregated for a sample size of 700 to assess the differences between estimated and actual life expectancy, and the association of the difference scores to actual survival time (Spearman rank correlation). These data were also graphed to further illustrate the relationship between estimated life expectancy and survival time, to determine if physicians were more likely to underestimate life expectancy as survival time increased.
To calculate the dichotomous accuracy of life expectancy predictions (Fig. 1), the responses of the 7 cases of all 100 respondents were again aggregated to form a sample size of 700. Using Microsoft Excel 2007, each life expectancy prediction was simplified to either a true or false value to the question "Is this value greater or equal to 10 years." For each case, these predictions were then dichotomously separated into whether they were correct when compared to the actual survival of the case study's patient. The aggregate average of correct responses over the 700 responses was then calculated to give the dichotomous accuracy of 10 years. This same technique was then applied across all life expectancy cut-offs, simulating different life-expectancy cutoffs, from 0 years to 25 years at 0.5 year increments. A line of best fit was then graphed to better illustrate the trend of dichotomous accuracy across the spectrum of cut-off years.
This analysis was taken a step further to calculate the non-dichotomous accuracy of life expectancy predictions by calculating the deviance of each prediction in relation to the actual survival of each case. These data were calculated as a both a percentage and number value with analyses in terms of both actual and absolute values. The mean and standard deviation for all cases was then calculated from these results. For interest, the sample size of 700 was also stratified by stage of training, and the best and worst predictions were identified from within the pool.
Surveys were distributed and collected personally; of the 127 surveys distributed, 100 surveys were collected. This translates to a response rate of 78.7%. Of those who responded, most were male urologists (47%). Some statistics, such as medical school, residency school and program, fellowship school and program, were too stratified for analysis and therefore left out (Table 1).
Comparative differences between groups
Analysis was done to compare outcomes between groupings. The four factors that were analyzed based on adequate numbers were sex, level of training, location of work and specialty. In terms of sex, we found that there was no significant difference in predictive accuracy between men and women (p = 0.27). Similarly, level of training, when broken down into medical student, resident, and attending physician (including all physicians who have completed residency), yielded no significant difference in predictive accuracy (p = 0.46). Location of work did not affect the ability to predict life expectancy (p = 0.48), and no particular specialty did significantly better (p = 0.24) (Table 2). However, post-hoc power analyses indicated that the study was not adequately powered to detect differences between these groups, with power ranging from a low of 8% for the comparison of medical student, resident and attending, to 31% for the comparison of men and women. These findings therefore need to be interpreted within the context of the limited study power.
Accuracy of prediction overall
General trends in responses showed that medical professionals generally underestimate their patients' life expectancy. Across all 700 responses to the clinical scenarios, the difference between the actual and estimated life expectancy ranged from −19.3 to 13.6, with a mean of −2.0 years (standard deviation [SD] 6.1), The coefficient relating actual survival time to estimated survival time was −0.57, (p < 0.001), suggesting that physicians are more likely to underestimate life expectancy as actual survival time increases.
To understand how well we do overall at estimating life expectancy in the context of screening guidelines, a dichotomous accuracy analysis was done, plotting the percentage of correct responses against actual life expectancy cut-offs (Fig. 1). For example, if we categorized answers as being [greater than or equal to]10 years or <10 years and compared them to the patients' actual survival time, overall, medical professionals predicted correctly only 68.1% of the time; within the overall result, medical students scored 75.6%, residents 70.9% and attending physicians 64.8%. Interestingly, when graphed with a line of best fit, we found that the life expectancy cut-off at which medical professionals predict the least accurately was at exactly 10 years.
In investigating how accurate we are in our non-dichotomous life expectancy predictions versus actual survival, we considered each case response as an individual instance rather than as a case within a series (n=700). We found that, overall, if we considered only the absolute values of our error, we were 67.4% inaccurate on average compared to actual survival (SD=90.4%). Within that average, medical students were 77.5% inaccurate (SD=106.4%); residents were 68.6% inaccurate (SD=97.0%); attending physicians were 63.8% inaccurate (SD=81.9%). Overall, the range of inaccuracy per respondent ranged from 26.7% to 149.3% in the predictions of the actual survival over the 7 case scenarios, with both the best and worst predictors being attending physicians (Table 3). Another way of expressing this is in how many years off our predictions are from actual survival. We also stratified the predictions by group (Table 2). Overall, respondents underestimated life expectancy by an average of 2.0 years (SD=5.2).
Finally, though the analysis did not carry enough power of response, there was a positive trend between years in practice and non-dichotomous accuracy of life expectancy prediction (p = 0.122).
Use for screening
The use of screening tests for prostate cancer has become controversial and, in response, several official guidelines have been updated (Appendix A). While most agencies either cite an inadequacy of information to choose a side or provide limited partial guidelines, agencies that do describe the screening test recommend that patients have at least a 10-year life expectancy for them to benefit from the screening; 80% of Canadian specialists concur with this cut-off.[sup.9]
Similarly, there has also been controversy over whether patients over a certain age should be screened at all. This is because of their lesser likelihood to fulfill this "10-year rule" and benefit from treatment, even though it has been shown that survival after initial diagnosis is on average 15 to 20 years.[sup.4] Interestingly, Pearce and colleagues found that most urologists recommend screening patients up to 70 years old, while most radiation oncologists continue screening past 80.[sup.9] Contrarily, the U.S. Preventative Task Force currently advocates that patients over 75 years old should not be screened,[sup.10] though this recommendation might soon change to recommend against screening at all, regardless of age.[sup.11] The result of this type of cap is that potential patients who might benefit from screening and treatment are arbitrarily excluded, despite the fact that their life expectancies might exceed 10 years.
Recently, two large long-term studies have also offered guidance on the use of screening. It is interesting to note, however, that both study groups strictly limited their patient population to men who fell well within the traditional screening ages of 50 and 75 (ages that were chosen to reflect the 10-year rule): 55 to 74 in the PLCO (Prostate, Lung, Colorectal and Ovarian) trial and 55 to 69 in the ERSPC (European Randomized Study of. Screening for Prostate Cancer) trial.[sup.12,13] This is significant because it means that the way we measure the effectiveness of screening might be inherently faulty since we apply and study the effects of screening on a population that was chosen by fundamentally arbitrary mechanisms. Because of this general uncertainty, physicians are left with little guidance, save their personal opinions, as to how and if they should screen.
Life expectancy predictions
While the "10-year rule" is suspect, the question remains that if we were to include it in our screening protocol, how well can we apply it? Because this protocol can also affect treatment decisions, those who have to apply the "10-year Rule" include a wide range of physicians, such as primary care physicians, internists, urologists and radiation oncologists. Our results are based on a paper presentation of cases which have been shown by Kirwan and colleagues to correlate well with in-person assessments.[sup.14] They show that physicians are poor predictors of life expectancy, with no significant distinction between groups, such as sex, level of training, location of work or medical specialty. In addition, when combining our results with the literature on whether physicians overestimate or underestimate life expectancy, we find that the variability suggests no particular favour either way. The same applies to the accuracy relative to a physician's number of years in practice.[sup.15]-[sup.18] Wilson and colleagues summarize this well by arguing that clinicians are not only inaccurate, but also imprecise and inconsistent with their predictions.[sup.16]
Physicians in modern and historic times have had to estimate how long patients have left to live. "Yes" and "No" answers to such questions as "Do I have 10 more years to live?" demonstrate the concept of dichotomous accuracy. Even states such as death have historically been a challenge to confirm. As patients' actual survival moves away from 0 years, our analysis shows that physicians get progressively worse in predictive dichotomous accuracy. Interestingly, this decrease in the ability to estimate life expectancy only continues up to a point: 10 years. After this point, as actual survival continues to climb, the dichotomous accuracy slowly climbs back up to almost 100% accuracy. This might be due to the fact that life expectancies of patients at the ends of the spectrum, who are immediately sick or relatively healthy, are easier to assess as they are on more defined paths of well-being, a phenomenon that can be described as a "horizon effect;" however, moderately sick patients tend to occupy a grey zone of well-being and thus their life expectancy becomes a challenge to predict.
For this particular study, we questioned whether the 10-year mark is a valid standard for use as a screening criterion. In our study, physicians correctly estimated patients' 10-year life/death status 68.14% of the time. Interestingly, the study by Walz and colleagues yielded a similar result at 10 years of 68% mean accuracy; physicians' ability to predict patients' 10-year life expectancy was "moderate at best."[sup.18] While the judgement call of whether 68.1% accuracy is passable or not is subjective, we found that physicians' ability to predict patients' 10-year life expectancy was in fact the worst.
If clinical life expectancy prediction fails as a criterion, then what other options are we left with? One possible method is a mathematical model to predict life expectancy. More than 10 models have been proposed, including complex actuarial insurance models. Their precisions vary, from being highly synchronous with physician predicted life expectancy to yielding only modest success with none of them providing any additional benefit or significantly better predictions. Furthermore, in every case studied, these models were measured relative to hypothetical predictions rather than actual end points.[sup.18]-[sup.23]
Another method proposed is age cut-offs. However, as discussed above, this similarly arbitrary cut-off not only categorizes many potential patients to inappropriate management, but has not been fully studied aside from retrospective studies. Currently, there is little tolerance of "age-ism" and so this would likely not be a viable alternative.
One limitation of the study is the use of a convenience sample of the first 100 physicians and medical students who agreed to participate. While the study does have sufficient numbers to support the use of inferential statistics, it was under-powered to detect between-group differences such as the comparisons for men and women, level of training, specialty and working environment. Another limitation includes the case studies themselves; more information can influence a decision, such as if the patient looks ill or not. One must also balance this, however, with the reality that we never have perfect information.
Although physicians might be poor predictors of life expectancy, there may not be any significantly better alternatives at the moment. Our study implies that Canadian physicians are more likely to underestimate life expectancy. If this is true, then we are also missing a population of men who indeed have 10 years or more to live and who may benefit from screening. Further study in methods to improve our accuracy and determine the chances of death from competing causes is an understudied aspect of potential prostate cancer patients.
Julie Degroot and Patti Groome.
Competing interests: None declared.
This paper has been peer-reviewed.
1. Statistics Canada. Canadian Vital Statistics, Births and Deaths Databases, Demography Division, CANSIM Table 102-0511. http://www5.statcan.gc.ca/cansim (Accessed September 11, 2012).
2. Heron M, Hoyert DL, Murphy SL, et al. Deaths: final data for 2006. Natl Vital Stat Rep 2009;57:1-134.
3. Ellison LF, De P, Mery LS, et al. Canadian cancer statistics at a glance: cancer in children. CMAJ 2009;180:422-4http://dx.doi.org/10.1503/cmaj.081155.
4. American Urological Association: Prostate-Specific Antigen Best Practice Statement: 2009. http://www.auanet.org/content/media/psa09.pdf (Accessed September 11, 2012).
5. Greene KL, Albertsen PC, Babaian RJ, et al. Prostate specific antigen best practice statement: 2009 update. J Urol 2009;182:2232-41http://dx.doi.org/10.1016/j.juro.2009.07.093.
6. Wolf AM, Wender RC, Etzioni RB, et al. American Cancer Society guideline for the early detection of prostate cancer: update 2010. CA Cancer J Clin 2010;60:70-98http://dx.doi.org/10.3322/caac.20066.
7. Mistry S, Mayer W, Khavari R, et al. Who's Too Old To Screen? Prostate Cancer in Elderly Men. Can Urol Assoc J 2009;3:205-10.
8. Altman DG. Practical statistics for medical research. London: Chapman and Hall; 1991.
9. Pearce A, Newcomb C, Husain S. Recommendations by Canadian urologists and radiation oncologists for the treatment of clinically localized prostate cancer. Can Urol Assoc J 2008;2:197-203.
10. U.S. Preventive Services Task Force. Screening for prostate cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med 2008;149:185-91.
11. U.S. Preventive Services Task Force. Screening for Prostate Cancer: U.S. Preventive Services Task Force Recommendation Statement draft 2011. 2012.
12. Andriole GL, Crawford ED, Grubb RL 3rd, et al. Mortality results from a randomized prostate-cancer screening trial. N Engl J Med 2009;360:1310-9http://dx.doi.org/10.1056/NEJMoa0810696.
13. Schroder FH, Hugosson J, Roobol MJ, et al. Screening and prostate-cancer mortality in a randomized European study. N Engl J Med 2009;360:1320-8http://dx.doi.org/10.1056/NEJMoa0810084.
14. Kirwan JR, Chaput de Saintonge DM, Joyce CR, et al. Clinical judgment in rheumatoid arthritis. I. Rheumatologists' opinions and the development of ‘paper patients'. Ann Rheum Dis 1983;42:644-7http://dx.doi.org/10.1136/ard.42.6.644.
15. Christakis NA, Lamont EB. Extent and determinants of error in physicians' prognoses in terminally ill patients: prospective cohort study. West J Med 2000;172:310-3http://dx.doi.org/10.1136/ewjm.172.5.310.
16. Wilson JR, Clarke MG, Ewings P, et al. The assessment of patient life-expectancy: how accurate are urologists and oncologists? BJU Int 2005;95:794-8http://dx.doi.org/10.1111/j.1464-410X.2005.05403.x.
17. Chow E, Davis L, Panzarella T, et al. Accuracy of survival prediction by palliative radiation oncologists. Int J Radiat Oncol Biol Phys 2005;61:870-3http://dx.doi.org/10.1016/j.ijrobp.2004.07.697.
18. Walz J, Gallina A, Perrotte P, et al. Clinicians are poor raters of life-expectancy before radical prostatectomy or definitive radiotherapy for localized prostate cancer. BJU Int 2007;100:1254-8http://dx.doi.org/10.1111/j.1464-410X.2007.07130.x.
19. Albertsen PC, Fryback DG, Storer BE, et al. The impact of co-morbidity on life expectancy among men with localized prostate cancer. J Urol 1996;156:127-32http://dx.doi.org/10.1016/S0022-5347(01)65964-0.
20. Krahn MD, Bremner KE, Asaria J, et al. The ten-year rule revisited: accuracy of clinicians' estimates of life expectancy in patients with localized prostate cancer. Urology 2002;60:258-63http://dx.doi.org/10.1016/S0090-4295(02)01712-0.
21. Koch MO, Miller DA, Butler R, et al. Are we selecting the right patients for treatment of localized prostate cancer? Results of an actuarial analysis. Urology 1998;51:197-202http://dx.doi.org/10.1016/S0090-4295(97)00622-5.
22. Henderson R, Keiding N. Individual survival time prediction using statistical models. J Med Ethics 2005;31:703-6http://dx.doi.org/10.1136/jme.2005.012427.
23. Cowen ME, Halasyamani LK, Kattan MW. Predicting life expectancy in men with clinically localized prostate cancer. J Urol 2006;175:99-103http://dx.doi.org/10.1016/S0022-5347(05)00018-2.
24. The World Health Organization. Prostate Cancer Screening 2009 2009.
25. Izawa JI, Klotz L, Siemens DR, et al. Prostate Cancer Screening: Canadian Guidelines 2011. Can Urol Assoc J 2011;5:235-40http://dx.doi.org/10.5489/cuaj.11134.
26. Canadian Cancer Society. Understanding the Pros and Cons of Prostate Cancer Testing 2009 2009.
27. Feightner JW. The early detection and treatment of prostate cancer: the perspective of the Canadian Task Force on the Periodic Health Examination. J Urol 1994;152:1682-4.
28. National Cancer Institute. Prostate Cancer Screening 2009 2009.
Appendix A.: Current screening protocol for all men for prostate cancer, abridged collection [Table omitted]
Appendix B.: Patient life expectancy assessment survey questionnaire [Table omitted]
Figure and Tables
Fig. 1: The dichotomous accuracy of life expectancy predictions in relation to a spectrum of life expectancy cut-offs. As actual survival moves towards the 10 year mark, dichotomous accuracy decreases precipitously; however, as actual survival increases past this point, dichotomous accuracy slowly climbs back up to near 100%. [Figure omitted]
Table 1: Descriptive statistics [Table omitted]
Table 2: Predictive accuracy between groups [Table omitted]
Table 3: Accuracy of life expectancy predictions vs. actual survival (calculated using absolute predictive error) [Table omitted]
 Queen's University, Kingston, ON;
 Kingston General Hospital and the Department of Community Health and Epidemiology, Queen's University, Kingston, ON;
 Department of Surgery, Division of Urology, University of Calgary, Calgary, AB
Correspondence: Dr. Kevin Leung, Queen's University, School of Medicine, Undergraduate Medical Education 68 Barrie St., Kingston, ON K7L 3N6
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Original Research|
|Author:||Leung, Kevin M.Y.B.; Hopman, Wilma M; Kawakami, Jun|
|Publication:||Canadian Urological Association Journal (CUAJ)|
|Date:||Oct 1, 2012|
|Previous Article:||A unique case of an alpha-fetoprotein-producing lung cancer with testicular metastasis.|
|Next Article:||Life expectancy estimation in prostate cancer patients.|