The accuracy of clinicians’ predictions of survival in advanced cancer: a review

Background

Clinicians’ predictions of survival (CPS) involve a complex process that attempts to formulate an easily understood survival prediction for patients (1,2). Health care professionals (HCPs) benefit from survival prediction, as they are able to make treatment recommendations based on this prediction (1). CPS is also important to patients and their families because of its relevance to prognosis, setting appropriate goals for management, and preparing for the patient’s future (1). In fact, 61% of patients preferred to have their prognosis communicated by their doctor, and 98% of patients wanted this information communicated accurately and honestly (2).

Advanced cancer is typically characterized by an accelerated decline in health over the final weeks of a patient’s life (3). Due to the predictability of this decline, prognostication of advanced cancer patients is often easier compared to patients with early stage disease (3). However, literature has shown that physicians tend to be overly optimistic and overestimate their patients’ survival (2,4-14). Inaccurate predictions may hinder optimal management; for instance, overestimating survival may lead to over-treatment or late referral to palliative care, while underestimating survival may lead to under-treatment or premature referral to palliative care (1). Thus, the objective of this paper is to review the accuracy of clinicians’ ability to predict survival in advanced cancer patients.

Methods

Search strategy

A search in Cochrane CENTRAL, Ovid EMBASE, and Ovid MEDLINE was conducted for articles published in English between 2000 and May 2015. Keywords and subject headings used in the literature search included: neoplasms, terminally ill, carcinoma, disease, neoplasm metastasis, forecasting, prognosis, survival analysis, life expectancy, survival, mortality, physicians, hospices, and terminal care. The search strategy used is provided in Appendix 1.

Selection criteria

Studies were considered eligible during title and abstract screening if: (I) they involved a prospective cohort or retrospective cohort study design; (II) the primary subjects in the study were advanced cancer patients; (III) the study reported on the ability of clinicians to predict survival with a documented comparison of CPS versus observed survival (either the ratio of the estimated to observed survival was reported, or both the estimated and observed survival were reported independently to allow calculation of the estimated-to-observed survival ratio). All non-original articles, including literature reviews, editorials, and commentaries, were excluded. All studies concerning individual prognostic markers and tools, including biological and molecular markers and their ability to predict survival, were also excluded. Finally, studies including patients without a cancer diagnosis were excluded.

Data collection

Two authors (Cheon and Agarwal) screened titles and abstracts independently and established consensus through discussion where necessary to determine articles eligible for full-text screening. Articles considered eligible at the title and abstract review stage were evaluated at the full-text screening phase independently using the same reviewers and inclusion/exclusion criteria. Consensus was established by discussion between the two reviewers where necessary.

Data extraction

The following data was extracted from all included articles: year and country of publication, the number of centres included, patient characteristics, clinician characteristics, survival prediction data, level of evidence, identified patient-related and clinician-related prognostic factors, and main conclusions. Patient characteristics included the number of patients, primary tumour site, location of metastases, description of patient/cancer/metastasis, mean and standard deviation (SD) of patients’ age, male to female patient ratio, time since diagnosis, number of patient deaths, and Karnofsky Performance Status score. Clinician characteristics consisted of the type of clinicians, their specialities, mean and SD of clinicians’ age, male to female clinician ratio, and clinicians’ duration of clinical and specialty-related experience. Lastly, the survival prediction data included the criteria for accuracy, median estimated and observed survival, the inter-quartile range of the survival estimate, the ratio of the estimated to observed survival, and the optimistic error.

Median estimated and observed survival was standardized to days. To compare the accuracy of clinical prediction among studies, a ratio of the estimated to observed survival was noted if presented by the papers, or calculated by the authors if directly absent in the papers. Estimated-to-observed survival ratios were calculated for each included study (Table 1) (2,4-17). To calculate the aforementioned ratio, we used the following formula: median estimated survival (days) divided by the median observed survival (days). In studies in which both probabilistic CPS and temporal CPS were presented, temporal CPS was analyzed.

Table 1 Clinicians’ predictions of survival (CPS)
Full table

Results

A total of 1,481 abstracts were initially identified in the literature search, of which 32 came from Cochrane CENTRAL, 1,041 came from Ovid EMBASE, and 408 came from Ovid MEDLINE. Data extraction and analysis of clinicians’ ability to predict survival were carried out for 15 articles (Figure 1), as shown in Tables 1 and 2.

Figure 1 Flow diagram for articles included in the review.

Table 2 Patient and clinician characteristics
Full table

Patient characteristics

Primary tumour site was reported in 14 of the 15 included studies. The three most common primary tumour sites and location of metastases for included patients are reported in Table 2. The most common primary tumour site, as listed by 13 of 14 studies, was respiratory/lung (2,4-9,11,13-17), followed by breast in 8 of 14 studies (4,6,7,11-15), and gastrointestinal (4,8,11,14-17) and genitourinary/gynaecologic/prostate (2,4,7,8,12,13,17), each in seven of 14 studies. Other primary tumour sites were melanoma (6) and unknown primary (2), each in one of 14 studies. The location of metastases for patients was also similar among studies: six of seven studies involved patients with brain metastases (2,6,7,11,14,15), five with bone metastases (2,7,12,14,15), three with liver metastases (7,14,15), two with lung or pleural metastases (14,15), and one with soft tissue metastases (14).

Clinicians’ predictions of survival (CPS)

Data on prediction of survival was reported in the literature for various HCPs, ranging from physicians to nurses. The methods by which data was collected varied among the articles. The majority of the papers recorded the expected survival time once (2,5-17); however, the time at which survival predictions were made differed. For example, survival predictions were made after clinical assessment (2), before patient consultation (11), after enrolment into the palliative care service (16), at study entry (12), or after consultation (13). When survival time was estimated more than once, survival prediction was made daily until death or discharge (4).

In addition to different methods of data collection, the criteria for accuracy of employed predictions also differed by study. Perez-Cruz et al. [2014], Hui et al. [2011], Lam et al. [2008], and Faris et al. [2003], who reported on 311, 151, 167, and 162 patients respectively, considered CPS to be accurate if it was within approximately 33% of the observed survival (4,8,11,14). In contrast, Kiely et al. reported on 244 patients in 2013 and considered CPS to be precise if it was within 0.75−1.33 times the observed survival (5), while Fairchild et al. reported on 155 patients in 2014 and determined CPS within 30 days of observed survival as correct (2). Lastly, Higginson et al. [2002] considered CPS to be accurate if observed survival fell between the minimum and maximum CPS, in their study involving 275 patients (17).

Accuracy of clinical prediction of survival

An estimated-to-observed survival ratio less than 1 (i.e., underestimation of survival) was calculated for all HCPs noted in two studies (15,16), and for a portion of HCPs or for 1 or more time-points assessed in three studies (6,7,10). The ratio ranged from 0.5−0.92 across these studies (6,7,10,15,16).

In contrast, nine studies reported a ratio greater than 1 (i.e., overestimation of survival) for all HCPs (2,4,5,8,9,11-14), and three studies reported a ratio greater than 1 for a portion of HCPs or for 1 or more assessed time-points (6,7,10). The range of the ratio was 1.06−6 (2,4-14). Patients’ survival duration was estimated more than once by Perez-Cruz et al. (4). Interestingly, the authors found that the median estimated survival was consistently higher than observed survival, regardless of how near patients were to death (4).

Differences among clinicians

While several studies did not report significant differences in the accuracy of survival prediction between physicians and nurses (4,11), some discrepancies were evident, suggesting that profession type may have some influence on CPS. Some studies noted a higher accuracy of CPS by physicians compared to nurses, and by residents and registrars compared to consultants (8,9). In addition, Fairchild et al. [2014] found no significant differences in accuracy between the medical group (consisting of a radiation oncologist, residents, medical students, and fellows), radiation therapists, nurses (consisting of a nurse practitioner, registered nurse, and nursing students), and allied health professionals (consisting of a clinical nutritionist, occupational therapist, pharmacist, and respiratory therapist), with the exception of a difference found between radiation therapists and allied health professionals (P=0.04) (2).

Although the specialities of clinicians varied, most clinicians practiced either in palliative care and/or oncology. Three studies indicated that the accuracy of CPS was not dependent on clinicians’ experience or years of practice (8,11,13). Hui et al. [2011] also reported that differences in clinicians’ age and sex were not significant factors for the prediction of survival (8).

Discussion

Summary of evidence

This review aimed to analyze the accuracy of clinicians’ prediction of survival in advanced cancer patients. Building on the initial landmark study by Parkes et al. [1972] (18), which suggested little concordance between predicted and observed survival with 83% of prediction errors being overoptimistic, this review of recent literature suggests that clinicians tend to more often overestimate than underestimate patients’ survival, as 12 studies reported optimistic predictions (2,4-14), while five studies (6,7,10,15,16) included pessimistic predictions.

Limitations

Several inconsistencies are present in the conclusions presented by the individual papers. Comparisons across different studies are difficult, considering differences in methodology, such as the diverse criteria used by each study to determine significance, and method employed to measure the accuracy of survival predictions. For instance, Kiely et al. [2013] suggested that oncologists’ estimates of their patients’ survival were imprecise (5). In contrast, Faris et al. [2003] stated that the correlation (0.678) between clinician-predicted and observed survival was significant (P=0.01) (14). Furthermore, Perez Cruz et al. [2014] noted that how near patients were to death did not affect the direction of survival estimation (i.e., survival was always overestimated). On the contrary, Chow et al. [2010] reported an interesting finding: survival predictions for patients with a survival duration of ≤6 months had a tendency to be optimistic, with an estimated-to-observed survival ratio range of 1.67−6.0, while patients with a survival duration of ≥9 months had a tendency to be pessimistic, with a range of 0.5−1.0 (10). Ultimately, several studies draw attention to the fact that CPS are inaccurate (4,6,7,9,10,12,13,17). It is also important to recognize that other considerations beyond CPS and observed patient survival, including patient and clinician characteristics, models to guide clinician predictions and decision-making, and other variables influencing clinician predictions and subsequent decision-making, warrant further investigation. Analysis of these other variables may be useful for better survival predictions. Lastly, while the quality of the included studies was not critically appraised, our review includes studies with small samples, which may limit the generalizability of our findings.

A strength of our review is that we conducted a search of three databases to identify relevant literature in the area, and included studies assessing any subset(s) of HCPs at varying lengths of follow-up.

Conclusions

The ability of clinicians to predict survival accurately is difficult; however an estimate of patients’ survival is necessary for HCPs, patients, and their families to make the most appropriate decisions. Clinicians must be aware and cognisant of this inaccuracy in CPS, and in particular, the tendency of HCPs to overestimate survival. This study highlights the need to further investigate the formulation of better survival prediction tools or perhaps the use of CPS in combination with other tools to predict survival. To improve the accuracy of survival prediction in advanced cancer patients, the ability of prognostic models as tools to estimate survival more accurately should be investigated (19-21). Accurate prediction of survival, followed by honest communication of prognosis with patients and their families, are essential for the appropriate delivery of palliative care.

Acknowledgements

We thank the generous support of the Bratty Family Fund, Michael and Karyn Goldstein Cancer Research Fund, Joey and Mary Furfari Cancer Research Fund, Pulenzas Cancer Research Fund, Joseph and Silvana Melara Cancer Research Fund, and Ofelia Cancer Research Fund.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Supplementary

Appendix 1 Search strategy for Ovid MEDLINE.

Database: Ovid MEDLINE(R) and Ovid OLDMEDLINE(R) [1946-May Week 1, 2015] Search Strategy:

• exp Neoplasms/ and exp terminally ill/ [1371]
• exp Neoplasms/ and ((terminal* or advanced or “stage IV”) adj2 (cancer or neoplasm* or carcinoma or disease)).mp. [58790]
• exp Neoplasm Metastasis/ [162600]
• or/1-3 [214514]
• exp Forecasting/ [72074]
• exp Prognosis/ [1160020]
• exp Survival Analysis/ [198744]
• exp Life Expectancy/ [14362]
• (prognostic factor* or prognostic tool*).mp. [61574]
• ((predict* or forecast or estimat* or timing) adj2 (survival or life expectancy or life span)).mp. [21292]
• exp Mortality/ [293231]
• or/5-11 [1505536]
• ((physician* or clinician* or clinical or oncologist*) adj4 (predict* or forecast or estimat* or timing) adj4 (survival or surviv* or life span or life expectancy or hospice or palliative or outcome)).mp. [6176]
• 4 and 12 and 13 [490]
• limit 14 to (English language and humans and yr=“2000 -Current”) [408]

References

1. Thai V, Wolch G, Tarumi Y. Survival Prediction of End Stage Cancer Patients: A Quick Review. J Palliat Care Med 2013;3:164.
2. Fairchild A, Debenham B, Danielson B, et al. Comparative multidisciplinary prediction of survival in patients with advanced cancer. Support Care Cancer 2014;22:611-7. [PubMed]
3. Scarpi E, Maltoni M, Miceli R, et al. Survival prediction for terminally ill cancer patients: revision of the palliative prognostic score with incorporation of delirium. Oncologist 2011;16:1793-9. [PubMed]
4. Perez-Cruz PE, Dos Santos R, Silva TB, et al. Longitudinal temporal and probabilistic prediction of survival in a cohort of patients with advanced cancer. J Pain Symptom Manage 2014;48:875-82. [PubMed]
5. Kiely BE, Veillard AS, Davidson JA, et al. Prognostic Significance, Accuracy and Usefulness of Oncologists Estimates of Survival Time for Patients Starting First Line Chemotherapy for Advanced Non-Small Cell Lung Cancer (ANSCLC). Asia Pac J Clin Oncol 2013;9:75.
6. Kondziolka D, Parry PV, Lunsford LD, et al. The accuracy of predicting survival in individual patients with cancer. J Neurosurg 2014;120:24-30. [PubMed]
7. Tseng YD, Krishnan MS, Sullivan AJ, et al. How radiation oncologists evaluate and incorporate life expectancy estimates into the treatment of palliative cancer patients: a survey-based study. Int J Radiat Oncol Biol Phys 2013;87:471-8. [PubMed]
8. Hui D, Kilgore K, Nguyen L, et al. The accuracy of probabilistic versus temporal clinician prediction of survival for patients with advanced cancer: a preliminary report. Oncologist 2011;16:1642-8. [PubMed]
9. Clément-Duchêne C, Carnin C, Guillemin F, et al. How accurate are physicians in the prediction of patient survival in advanced lung cancer? Oncologist 2010;15:782-9. [PubMed]
10. Chow E, Hruby G, Harris K, et al. Determining the Accuracy of Health Care Professionals in Predicting the Survival of Patients with Advanced Metastatic Cancer. J Pain Manag 2010;3:73-9.
11. Barnes EA, Chow E, Tsao MN, et al. Physician expectations of treatment outcomes for patients with brain metastases referred for whole brain radiotherapy. Int J Radiat Oncol Biol Phys 2010;76:187-92. [PubMed]
12. Hartsell WF, Desilvio M, Bruner DW, et al. Can physicians accurately predict survival time in patients with metastatic cancer? Analysis of RTOG 97-14. J Palliat Med 2008;11:723-8. [PubMed]
13. Chow E, Davis L, Panzarella T, et al. Accuracy of survival prediction by palliative radiation oncologists. Int J Radiat Oncol Biol Phys 2005;61:870-3. [PubMed]
14. Faris M. Clinical estimation of survival and impact of other prognostic factors on terminally ill cancer patients in Oman. Support Care Cancer 2003;11:30-4. [PubMed]
15. Kiely BE, Martin AJ, Tattersall MH, et al. The median informs the message: accuracy of individualized scenarios for survival time based on oncologists’ estimates. J Clin Oncol 2013;31:3565-71. [PubMed]
16. Lam PT. Accuracy of clinical prediction of survival in a palliative care unit. Prog Palliat Care 2008;16:113-7.
17. Higginson IJ, Costantini M. Accuracy of prognosis estimates by four palliative care teams: a prospective cohort study. BMC Palliat Care 2002;1:1. [PubMed]
18. Parkes CM. Accuracy of predictions of survival in later stages of cancer. Br Med J 1972;2:29-31. [PubMed]
19. Jang RW, Caraiscos VB, Swami N, et al. Simple prognostic model for patients with advanced cancer based on performance status. J Oncol Pract 2014;10:e335-41. [PubMed]
20. van de Laar R. External validation of three prognostic models for overall survival in patients with advanced-stage epithelial ovarian cancer. Br J Cancer 2014;110:42-8. [PubMed]
21. Gwilliam B, Keeley V, Todd C, et al. Development of prognosis in palliative care study (PiPS) predictor models to improve prognostication in advanced cancer: prospective cohort study. BMJ 2011;343:d4920. [PubMed]