American College of Surgeons National Surgical Quality Improvement Program as a quality-measurement tool for advanced cancer patients

Introduction

Surgical palliation refers to the deliberate use of a procedure in a patient diagnosed with incurable malignancy with the intention of relieving symptoms, minimizing patient distress, and improving quality of life (1-3). These procedures play invaluable roles in patients with disseminated malignancy, and at high volume cancer centers, may account for 6-21% of all surgical interventions (4-8). With appropriate counseling and patient selection, symptom resolution can be achieved in as many as 80% of patients (5,9). The relief of intractable pain, bleeding, and intestinal obstructions, among other debilitating symptoms, allows patients to be comfortable and retain an acceptable level of functionality. Despite the success of most palliative operations, approximately 25% of patients will require further interventions for new or recurrent symptoms (5,9). Post-operative complications can present in as many as 40% of patients and overall mortality can reach 23%, mostly secondary to the advanced disease and associated comorbidities (5,6,8). It has been demonstrated that even when prolonged symptom relief is not obtained, these patients do not experience a reduction in quality of life (10). Given the potential risks, decisions regarding the use of surgical procedures for palliation in patients diagnosed with advanced and incurable cancer require a thorough understanding of surgical outcomes data and the highest level of surgical judgment (11,12).

With the onset of surgical quality improvement initiatives, much attention has been placed on data collection to objectively measure and compare surgical outcomes (13). These data are obtained across the United States as part of well-structured databases such as the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP). Data within ACS-NSQIP is collected and analyzed under rigorous guidelines, by trained personnel, at every participating institution. For each contributing hospital, approximately 20% of all surgical patients and 136 variables are captured and analyzed (14).

When applied to patients with advanced malignancy, the database allows for identification of cancer-related operations if they meet the “Disseminated Cancer” criterion. Per ACS-NSQIP guidelines, these include patients diagnosed with “cancer that has spread to one site or more sites in addition to the primary site”, or the “presence of multiple metastases which indicate the cancer is widespread, fulminant, or near terminal” (14). No further characterization is made regarding the intent of the operation. Documenting surgical intent for patients with advanced malignancy is of utmost importance when analyzing outcomes data. It has been demonstrated previously that failure to do so limits the appropriate analysis of postoperative morbidity and mortality (2,6,15,16).

The Surgical Risk Calculator was developed using data collected within ACS-NSQIP over a 3-year period, from 2009 to 2012 (17). This tool provides surgeons with information pertaining to the risk of post-operative morbidity and mortality for a particular operation. The calculator allows for easy access to patient-specific data that may aid in clinical decision-making, operative planning, or as an adjunct when counseling patients and family. Authors have proposed using the ACS-NSQIP database in the development of risk calculators, or as a tool for clinical decision-making in the advanced cancer patient (7,18,19). For various particular patient groups, outcomes have already been compared utilizing ACS-NSQIP data and specialty- and procedure-specific databases; yet, no validation has been performed for patients diagnosed with advanced cancer (20). In this study we seek to define the role of ACS-NSQIP data when used for risk-stratification in the advanced cancer patient, particularly those undergoing operations with palliative intent.

Methods

A retrospective review of all cases contained within Rhode Island’s Hospital ACS-NSQIP database between 2007 and 2013 was performed. Data was collected exclusively by ACS-NSQIP certified personnel, following established guidelines. All cases within the database labeled with the “Disseminated Cancer” identifier were included as our study population. These data encompassed patients from different surgical services, including general and orthopedic surgery and neurosurgery.

By querying a single institution’s ACS-NSQIP database we were able to cross-reference entries with hospital records. Explicit chart review was performed for all patients. Demographic, operative, and post-operative survival data, not contained within the ACS-NSQIP database, were obtained. Patients were followed for a minimum of 30 days, as per ACS-NSQIP protocol. Survival data was censored to that available through the month of February 2014 or at the time of death. Post-operative complications were analyzed and documented utilizing previously established methods (1,5,21). A grade 1 complication required local or bedside care; a grade 2 complication required invasive monitoring or intravenous medication; a grade 3 complication required an operation, interventional radiology procedure, intubation, or therapeutic endoscopy; a grade 4 complication resulted in a persistent disability or required major organ resection; and a grade 5 complication resulted in death. The highest severity level was recorded when a patient had more than one complication associated with a specific procedure. Post-operative mortality was utilized as a point of comparison when validating these data.

Patients were assigned to one of two groups based on the pre-operative intent of surgery. Those who underwent an operation for palliation were identified as previously described (2,3,5,15). Pre-operative clinic, counseling, and progress notes, as well as operative reports, were analyzed for terms indicative of palliative intent. An operation was considered palliative only when records clearly stated that it was performed to relieve specific symptoms or improve quality of life. When unclear statements regarding intent of surgery were documented, the operative surgeon was contacted and clarification was obtained when possible. All other patients were included in the “Non-Palliative” group. Patients within this group underwent operations that were unrelated to their primary diagnosis, or with intent to prolong survival, treat recurrences, or cure the underlying malignancy.

Means were compared using the Fisher’s exact test method, given the small sample size, and then expressed as percentages. This method was utilized to compare mortality at 30, 45, 60 and 90 days. Survival analysis was then performed using the Kaplan-Meier method; a Log Rank test was utilized to indicate differences between survival curves.

P values of less than 0.05 were considered to be significant. All calculations were performed using GraphPad Prism (version 6.00, GraphPad Software, La Jolla California, USA).

Results

Between June 2007 and June 2013, 7,763 operations were captured within Rhode Island’s Hospital ACS-NSQIP database. Of these, only 138 (1.8%) entries were identified as having “Disseminated Cancer” (Figure 1), and constituted our study population. Of the remaining 7,625 entries, only 4,486 entries contained data that allowed for survival analysis. These 4,486 entries constituted the “Non-disseminated Cancer” group—representing the general surgery patient population. The distribution of operations performed in the “Disseminated Cancer” group among surgical services was: general surgery 98 operations (71%), thoracic surgery 12 operations (8.7%), vascular surgery 7 operations (5.1%), and neurosurgery-orthopedic spine services 21 operations (15.2%). General surgery cases encompassed major abdominal, endocrine, skin and soft tissue, and breast operations. Spinal operations were performed for decompression, biopsy, and stabilization. Thirty-day mortality was significantly higher in patients within the “Disseminated Cancer” group when compared to all other surgical patients captured during this time period (7.9% vs. 0.9%, P<0.001).

Figure 1 Distribution of patients within Rhode Island Hospital’s ACS-NSQIP database from 2007-2013. ACS-NSQIP, American College of Surgeons National Surgical Quality Improvement Program.

Operations were performed with palliative intent on 32 of the 138 patients (23.2%). Within this group, general surgery procedures were performed on 13 patients (40.6%), thoracic procedures on 2 patients (6.3%), and spine-related operations on 17 patients (53.1%). No vascular operations were performed within the “Palliative” group. The “Non-Palliative” group constituted the remaining 106 patients (76.8%) with “Disseminated Cancer”. By specialty, general surgery procedures were performed on 85 patients (80.2%), thoracic procedures on 10 patients (9.4%), vascular procedures on 7 patients (6.6%), and spine-related procedures on 4 patients (3.8%).

Table 1 lists the maximum grade complication following initial palliative and non-palliative operations, as documented within ACS-NSQIP data. At 30 days, a total of 55 patients suffered a post-operative complication within the disseminated cancer group (39.9%). These included 15 patients within the palliative group and 40 patients within the non-palliative group (46.9% vs. 37.7%, P=0.41). The increased number of post-operative complications identified within the palliative group had no significant effects on survival. At the 30-day data collection limit for ACS-NSQIP, palliative operations had a post-operative mortality of 9.4%. When compared to “Non-Palliative” operations at 30 days, we were unable to detect a difference in postoperative mortality (9.4% vs. 7.5%, P=0.72).

Table 1 Maximum grade of complication at 30 days following initial procedure. Palliative and non-palliative groups were compared
Full table

Survival analysis was performed at numerous time points. The overall median survival for patients who underwent a palliative operation was 104 days. This was significantly reduced when compared to the 709 days for non-palliative procedures (P<0.001) (Figure 2). A significant difference in post-operative mortality between these two groups became evident at 45 days (31.3% vs. 10.4%, P=0.009), 60 days (37.5% vs. 13.2%, P=0.004), and 90 days (43.8% vs. 14.2%, P<0.001) after the initial intervention (Figure 3).

Figure 2 Using Log rank, overall survival between patients undergoing operations with palliative and non-palliative intent was compared (Kaplan-Meier estimation, P<0.001).

Figure 3 Bar graph illustrating mortality rates, in percentages, for palliative and non-palliative operations at 30, 45, 60 and 90 days.

Discussion

In this study, a critical shortfall of the commonly utilized ACS-NSQIP database and its attendant ACS Risk Surgical Calculator has been demonstrated when applied to the palliative surgical patient. Patients with advanced, incurable malignancy represent a small but unique subset within the surgical population. Previous work has shown that routine outcome measures such as post-operative 30-day morbidity and mortality are objective but rather incomplete outcome measures for palliative operations (2,5,6,16). Although procedure-related complications certainly negatively impact quality of life, palliative series have consistently shown excellent patient satisfaction despite relatively high morbidity and mortality rates (1,5,10). Application of clinical tools such as the palliative triangle, which involves the patient, patient’s family, and surgeon in the decision-making process, has been credited with improved patient counseling and therefore patient satisfaction following palliative operations (1,22).

The ACS-NSQIP database and the associated ACS Surgical Risk Calculator use 30-day morbidity and mortality as fundamental outcome measures in assessing overall hospital-specific outcomes and to aid in pre-operative counseling, respectively. Such measures are useful insofar as they provide objective information to patients and providers, helping to answer patients’ questions regarding their odds of developing post-operative complications or not surviving an operation. These measures may also guide surgeons in their assessment of patients’ operative risk and to balance that risk against non-operative management or other treatment modalities. The ACS-NSQIP database itself is built on rigorously collected data points, 136 in total, which attempt to accurately depict a hospital’s patient population, the surgical procedures performed, and their post-operative courses (14). For the general surgery patient population, it is the most robust and reliable database available. However, an inherent shortfall of the database is that it is arguably a population-based database that loses its power when analyzing more specific patient populations, such as the advanced cancer patient. Despite the “Disseminated Cancer” classification, the database does not specifically classify procedures performed in such patients as palliative or non-palliative in intent, which has been shown to be independently associated with patient outcomes following such procedures (2,6,15,16).

Analysis of our institution’s ACS-NSQIP database showed that 1.8% of the sampled surgical patient population was designated as having “Disseminated Cancer”. Case-specific reviews revealed that approximately one-quarter of these patients were operated on specifically with palliative intent, comprising less than 1% of the sampled population across surgical specialties. Comparison of palliative and non-palliative procedures could not establish that palliative patients have higher morbidity (46.9% vs. 37.7%, P=0.41) or mortality (9.4% vs. 7.5%, P=0.72) at 30 days. These differences did not reach statistical significance, which was most likely due to small patient numbers. Our institution is a large tertiary referral center, active in the management of cancer patients and palliation; despite this, only a limited number of “Advanced Cancer” patients, and palliative operations were captured over these 6 years in ACS-NSQIP. This further demonstrates that calculations utilizing these data may approximate outcomes at 30 days, but cannot be representative of actual outcomes for this particular patient population given the limited number of cases collected. Unfortunately, a broader evaluation of palliative and non-palliative outcomes across the entire ACS-NSQIP database is not possible given the inability to determine palliative intent on a case-by-case basis across multiple institutions and innumerable providers. Despite that, one can see that advanced cancer patients represent a small subset within the surgical population with specific considerations that a population-based database is unable to optimally characterize.

The standardized outcome measures of 30-day morbidity and mortality have become ubiquitous in assessing patient and institution outcomes. Such data is linked to hospital and provider reimbursements by the Centers for Medicare and Medicaid Services. Clinical medical literature routinely cites morbidity and mortality within 30 days when describing patient study populations, facilitating comparison of patient populations among institutions undergoing similar interventions. Its usefulness lies in objectively quantifying post-operative complication and death rates within an easily defined timeframe. Adverse events within the stated timeframe are likely to be directly or indirectly related to the intervention, as opposed to events that occur later. When applied to patients with advanced malignancy, such outcome measures, although objective, are often inaccurate in assessing the benefits of a palliative operation (23). Undoubtedly, the application of 30-day outcomes to a patient with an actuarial life expectancy of years may be very appropriate, as opposed to the cancer patient with a prognosis of mere months, regardless of planned surgical intervention. For this reason, some authors have utilized 90-day outcomes data as a more precise measure when analyzing patients with advanced malignancy (5). Our analysis demonstrated similar mortality at 30 days for palliative and non-palliative patients, but a significant difference in mortality when the comparison was carried out to 45 days (31.3% vs. 10.4%) and beyond. This correlated with the significant difference in median survival between the populations (104 vs. 709 days). The initially similar mortality rates were potentially a result of careful patient counseling and selection, which reduced treatment-related toxicity. Later mortality was more likely due to progression of disease. This demonstrates the incompleteness of 30-day morbidity and mortality in assessing outcomes of palliative operations as compared to the inclusion of other outcome measures such as symptom resolution, quality of life, and patient satisfaction.

Conclusions

When used for operative risk-assessment in patients with advanced cancer, the data contained within ACS-NSQIP may provide results that approximate actual morbidity and mortality outcomes at 30 days. However, due to the small number of operations captured, calculations utilizing these data do not reflect actual outcomes for patients with advanced cancer, or those undergoing palliative operations. More suitable outcomes measures such as symptom relief and patient satisfaction, which are essential to adequately evaluate the success of a palliative operation, are not included within the data. Therefore, thorough understanding of the information contained within the ACS-NSQIP database, as well as its limitations, is crucial if intended for the development of risk assessment tools or for operative planning in patients with advanced or incurable malignancy.

Acknowledgements

None.

Footnote

Conflicts of Interest: This work was presented as a poster presentation at the annual meeting of the New England Surgical Society’s 95^th Annual Meeting on September 12-14, 2014, Stowe, VT.

References

1. Miner TJ, Cohen J, Charpentier K, et al. The palliative triangle: improved patient selection and outcomes associated with palliative operations. Arch Surg 2011;146:517-22. [PubMed]
2. Miner TJ, Jaques DP, Shriver CD. A prospective evaluation of patients undergoing surgery for the palliation of an advanced malignancy. Ann Surg Oncol 2002;9:696-703. [PubMed]
3. Miner TJ, Jaques DP, Tavaf-Motamen H, et al. Decision making on surgical palliation based on patient outcome data. Am J Surg 1999;177:150-4. [PubMed]
4. Badgwell BD, Smith K, Liu P, et al. Indicators of surgery and survival in oncology inpatients requiring surgical evaluation for palliation. Support Care Cancer 2009;17:727-34. [PubMed]
5. Miner TJ, Brennan MF, Jaques DP. A prospective, symptom related, outcomes analysis of 1022 palliative procedures for advanced cancer. Ann Surg 2004;240:719-26; discussion 726-7.. [PubMed]
6. Krouse RS, Nelson RA, Farrell BR, et al. Surgical palliation at a cancer center: incidence and outcomes. Arch Surg 2001;136:773-8. [PubMed]
7. Tseng WH, Yang X, Wang H, et al. Nomogram to predict risk of 30-day morbidity and mortality for patients with disseminated malignancy undergoing surgical intervention. Ann Surg 2011;254:333-8. [PubMed]
8. McCahill LE, Smith DD, Borneman T, et al. A prospective evaluation of palliative outcomes for surgery of advanced malignancies. Ann Surg Oncol 2003;10:654-63. [PubMed]
9. Podnos YD, Wagman LD. The surgeon and palliative care. Ann Surg Oncol 2007;14:1257-63. [PubMed]
10. Miner TJ. Palliative surgery for advanced cancer: lessons learned in patient selection and outcome assessment. Am J Clin Oncol 2005;28:411-4. [PubMed]
11. McCahill LE, Krouse RS, Chu DZ, et al. Decision making in palliative surgery. J Am Coll Surg 2002;195:411-22; discussion 422-3. [PubMed]
12. McCahill LE, Krouse R, Chu D, et al. Indications and use of palliative surgery-results of Society of Surgical Oncology survey. Ann Surg Oncol 2002;9:104-12. [PubMed]
13. Porter GA, Skibber JM. Outcomes research in surgical oncology. Ann Surg Oncol 2000;7:367-75. [PubMed]
14. User Guide for the 2007 Participant Use Data File. American College of Surgeons National Surgical Quality Improvement Program. August 2008. Available online: https://www.facs.org/~/media/files/quality%20programs/nsqip/ug07.ashx
15. Miner TJ, Jaques DP, Karpeh MS, et al. Defining palliative surgery in patients receiving noncurative resections for gastric cancer. J Am Coll Surg 2004;198:1013-21. [PubMed]
16. Mosca PJ, Blazer DG 3rd, Wheeler JL, et al. When a chance to cut is not a chance to cure: a future for palliative surgery? Ann Surg Oncol 2011;18:3235-9. [PubMed]
17. ACS NSQIP®. Surgical Risk Calculator. Available online: http://riskcalculator.facs.org/
18. Thirunavukarasu P, Singla S, Nurkin S. Using Mobile App Technology for Point of Care Cancer Surgery Risk Assessment. In: Abstract Book. Society of Surgical Oncology 67th Annual Cancer Symposium, March 12-15, 2014, Phoenix, Arizona. Ann Surg Oncol 2014;21 Suppl 1:S137.
19. Paruch JL, Ko CY, Bilimoria KY. An opportunity to improve informed consent and shared decision making: the role of the ACS NSQIP Surgical Risk Calculator in oncology. Ann Surg Oncol 2014;21:5-7. [PubMed]
20. Bilimoria KY, Liu Y, Paruch JL, et al. Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg 2013;217:833-42.e1-3.
21. Martin RC 2nd, Jaques DP, Brennan MF, et al. Achieving RO resection for locally advanced gastric cancer: is it worth the risk of multiorgan resection? J Am Coll Surg 2002;194:568-77. [PubMed]
22. Thomay AA, Jaques DP, Miner TJ. Surgical palliation: getting back to our roots. Surg Clin North Am 2009;89:27-41. vii-viii. [PubMed]
23. Langenhoff BS, Krabbe PF, Wobbes T, et al. Quality of life as an outcome measure in surgical oncology. Br J Surg 2001;88:643-52. [PubMed]