Comparing the predictive values of five scales for 4-year all-cause mortality in critically ill elderly patients with sepsis

Introduction

More than half of the patients admitted to the intensive care unit (ICU) are older than 65 years (1,2). Sepsis is a medical emergency which may result in multiple organ dysfunction, accounting for more than 20% of ICU admissions with approximately 15% of all-cause in-hospital deaths (3,4). Among others, age is an important risk factor of sepsis (5). The incidence of sepsis increases with age, causing a sharp incidence particularly in people older than 80 years (5,6). Age is considered a risk factor for mortality in elderly patients admitted to ICU and is an independent predictor of mortality of patients with sepsis; the mortality of sepsis increases with age as well, especially for patients older than 65 years (7).

Aging is associated with various biological changes, such as multiple organ aging, metabolic dysfunction, and immunosenescence, etc. (8,9). Immunosenescence, which is defined as the degeneration and dysregulation of the immune function because of aging, has been recognized as an important predisposing factor of sepsis in the elderly (10,11). On one hand, the senescent immunity exposes the elderly to infections, autoimmune disorders and malignancies, which act solely or in combination to increase the risk of sepsis (10). On the other hand, immune responses may be reduced by the aged immunity, yielding atypical clinical manifestations of infection and sepsis in the elderly, which may impede prompt diagnosis and treatment. Hence, the elderly have poorer prognosis than young patients with sepsis (12,13). Elderly septic patients may present different disease progression from young patients, which may decrease the accuracy of prognostic prediction by commonly used clinical tools. Therefore, early and accurate prognostic assessments of sepsis in elderly patients remain an unmet need.

Several clinical biomarkers are potentially valuable to predict the short-term prognosis of elderly sepsis patients, including albumin, C-reactive protein, and procalcitonin (14,15). However, accuracy of single biomarker-based prognostication is limited due to the clinical heterogeneity of disease (16). In light of this limitation, severity scales combining several biomarkers and vital signs may provide an overall assessment of patients (17). Several severity scales are commonly used to evaluate the severity of disease in the ICU, including the Simplified Acute Physiology Score II (SAPS II) (18), the Oxford Acute Severity of Illness Score (OASIS) (19), the Modified Logistic Organ Dysfunction System (MLODS) (20), the Systemic Inflammatory Response Syndrome (SIRS) (21), and the Sequential Organ Failure Assessment (SOFA) (22). Although new severity scales including artificial intelligence models (23) have been frequently proposed (24,25), the accuracy is far from satisfactory (26,27). Researchers attempted to compare different clinical severity systems in elderly patients, while the results are inconsistent. A common drawback of such studies is the small sample size which renders stratified analysis impossible (13).

Long-term prognosis of sepsis is scarcely evaluated due to the difficulty of long-term follows-up and high research cost. After a septic episode, however, the risk of death driven by inflammation, age-related chronic diseases and sepsis related disabilities remains high for the elderly during a long period (28). In this regard, the prediction of long-term outcome for elderly septic patients in the ICU is exclusively important for the medical decision-making pending discharge, e.g., discharged to community health care center for prolonged treatment or directly discharged home.

In this study, we used a large cohort of patients with sepsis to compare the predictive efficacies of different severity scales for a follow-up up to four years. Following the World Health Organization (WHO) and the Organisation for Economic Co-operation and Development (OECD), “elderly” refers to an age of 65 years or more (1,29). We aim to validate which severity scale can better predict the long-term prognosis of elderly patients with sepsis in ICU. We present the following article in accordance with the TRIPOD reporting checklist (available at http://dx.doi.org/10.21037/apm-20-1355).

Methods

Data source

The Medical Information Mart for Intensive Care III (MIMIC-III) database is an open-access, single-center critical care database integrating the clinical data of patients admitted to the Beth Israel Deaconess Medical Center in Boston, United States from 2001 to 2012 (30). The database was approved by the Massachusetts Institute of Technology and the Institutional Review Boards. One researcher (LJ) has passed the Protecting Human Research Participants exam of National Institutes of Health (Record ID: 27638410) and gained permissible access to the MIMIC-III database. Because eighteen identifying data elements were removed from the MIMIC-III database (30) according to the Health Insurance Portability and Accountability Act (HIPAA) standards (www.hhs.gov), the approval of ethics committee and individual patient consents were waived. The demographic data, vital signs, laboratory data and other indices of 57,787 ICU patients were extracted from the database. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

Inclusion and exclusion criteria

Sepsis was diagnosed according to the Surviving Sepsis Campaign 2016 (31). Included subjects were those: (I) with first admission to the ICU during hospital stays; (II) aged ≥65 years; (III) were followed for 4 years by the CareVue system (19). Patients missing >5% indices were not included in our study. From 57,787 ICU patients in the MIMIC-III database, we excluded non-septic patients, non-elderly patients, subjects with incomplete data and those followed not by the CareVue system. Finally, 4,370 subjects were enrolled for analysis. The sample size was much larger than those of previous studies (32,33), which may ensure the effectiveness of our study. Age is an independent determining factor for long-term mortality, especially for the elderly. To exclude the interference of age-related natural death, we performed subgroup analysis according to the age. Since the ages of patients over 85 years old were anonymized (19), enrolled subjects were divided into three groups according to the age, namely the younger-old group (65 years ≤ age <75 years), the older-old group (75 years ≤ age <85 years) and the oldest-old group (age ≥85 years).

Data extraction

The Structured Query Language was used to extract data of each subject from the MIMIC-III database. The demographical data of gender, admission type, ethnicity, marriage, days of hospital stay and days of ICU stay were collected. We extracted important vital signs as well, including systolic blood pressure (SBP), diastolic blood pressure (DBP), heart rate (HR), respiratory rate (RR), body temperature and oxygen saturation. Comorbidities were defined as per the International Statistical Classification of Disease and Related Health Problems 10th Revision (ICD-10). Severity scale scores were calculated by SAPS II, OASIS, MLODS, SIRS and SOFA, respectively according to data of the first 24 hours after admission.

Outcomes

Four-year follow-up data were recorded by the CareVue system. The 4-year all-cause mortality was used as the long-term outcome to evaluate the predictive effect of each index. The endpoint was defined as all-cause death.

Statistical analysis

Categorical variables were expressed as percentages and analyzed with the Chi-squared test. Continuous variables were expressed as medians and quartiles and compared using the Student’s t-test. The linear regression model, the Kaplan-Meier (K-M) curve and the receiver operating characteristic (ROC) curve were used to assess the prognostic value of the long-term mortality of patients. We further calculated the area under the ROC curve (AUC) value of each severity scale to investigate their respective prognostic efficiency. The cut-off value of the severity scale with the highest AUC value would be calculated in the ROC curve to identify the risk group. The SPSS 22.0 software (SPSS, IBM, NY, US) and the Medcalc 18.5.0 software (MedCalc Software, Ostend, Belgium) were used to perform the statistical analyses. Statistical tests were two-tailed, and significance was set at P<0.05. Graphs were generated by Medcalc 18.5.0 and GraphPad Prism 7.0 (GraphPad Software, California, US).

Results

Four thousand three hundred and seventy patients are finally enrolled

The flow chart of participant enrollment was illustrated in Figure 1. A total of 4,370 elderly septic patients were finally enrolled for analysis (Table 1). Males accounted for 50.3% of all subjects. A majority of patients (91.1%) were admitted from the emergency room. About 74.7% of subjects were Caucasians, and 7.1% of subjects were Black. Vital signs and severity scale scores of patients during first 24 hours after ICU admission were collected (Table 1). The leading comorbidities were congestive heart failure, chronic pulmonary disease and renal failure, which accounted for 53.6%, 26.8 and 22.2%, respectively. For all patients, the median hospital stay and ICU stay were 11.8 (range, 0.1–191.4) days and 4.40 (range, 0.2–100.1) days, respectively.

Figure 1 Flowchart of subjects screening. Initially, 57,787 critically ill patients from the Medical Information Mart for Intensive Care III (MIMIC-III) database were selected. According to the Implementation of the International Statistical Classification of Disease and Related Health Problems 10th Revision, 42,581 non-septic patients were excluded. Then, we screened out 6,439 non-elderly patients and 92 patients without age information. Among the remaining 8,674 elderly patients, 92 subjects with wrong survival time and 4,102 patients followed not by the CareVue system were excluded. Finally, after screening out 110 patients lacking one or more clinical parameters, 4,370 subjects were enrolled for analysis.

Table 1 Characteristics of study subjects
Full table

Four-year all-cause mortality of elderly septic patients in the ICU is increased with age

We compared the demographic data among different groups to identify the biomarkers related to age. Five severity scales were calculated as described above. Most of the parameters, except oxygen saturation, MLODS, SOFA, SIRS and the comorbidity of renal failure, were significantly different among groups (P<0.01, Table 1). Compared with patients in the oldest-old group, those in the younger-old and the older-old groups had higher scores of SAPS II and OASIS (Table 1), indicating more serious illness and worse prognosis. The survival time of patients was inversely related to age; the mean survival time was the longest in the younger-old group, followed by the older-old group and the oldest-old group (P<0.01, Figure 2A). Likewise, the death rate was increased with age as per the K-M analysis (P<0.01, Figure 2B). Collectively, age appears to be an independent risk factor of mortality in elderly patients with sepsis.

Figure 2 Relationship between survival and age. Elderly septic patients were divided into the younger-old (65 years ≤ age <75 years), older-old (75 years ≤ age <85 years) and oldest-old (age ≥85 years). Survival time was compared among the three groups. The mean survival time of the younger-old patients was significantly longer than that of the oldest-old patients (P<0.01, A). The survival time was 431.2±559.0 days for the oldest-old group, 536.6±616.7 days for the older-old group, and 658.4±638.7 days for the younger-old group. In the K-M analysis, the survival rate with time was the highest in the younger-old group and the lowest in the oldest-old (P<0.01, B).

SAPS II has the highest discriminatory power in predicting the long-term prognosis of elderly septic patients

The ROC analysis was performed to evaluate the 4-year prognostic value by the five commonly used severity scales (Figure 3). The AUC values of SAPS II for all patients was 0.648 (95% CI, 0.634–0.667), which was significantly higher than other severity scales (all P<0.01, Figure 3A and Table 2). The highest discriminatory power of SAPS II was also shown in the younger-old and the oldest-old groups with AUC values of 0.642 (95% CI, 0.617–0.666, Figure 3B and Table 2) and 0.649 (95% CI, 0.618–0.679, Figure 3D and Table 2) separately. For patients in the older-old group, the AUC value was higher than those in other groups, for SAPS II and OASIS, respectively (Figure 3C and Table 2). Taken together, SAPS II better predicts long-term prognosis of elderly septic patients.

Figure 3 Receiver operating characteristic (ROC) curve analyses of predictors of elderly septic patients. The area under ROC curve (AUC) value of the Simplified Acute Physiology Score II (SAPS II), the Oxford Acute Severity of Illness Score (OASIS), the Modified Logistic Organ Dysfunction System (MLODS), the Systemic Inflammatory Response Syndrome (SIRS) and Sequential Organ Failure Assessment (SOFA) were compared. The highest discriminatory power of SAPS II was also shown in all patients (A), the younger-old (B) and the oldest-old groups (D). And the second higher AUC value was OASIS. However, for the older-old patients, no significant difference of AUC values was found among SAPS II and OASIS (C).

Table 2 Area under receiver operating characteristic curve (AUC) values of mortality and different severity scales
Full table

Older age and SAPS II >43 predict a poor prognosis of elderly septic patients

As per the linear regression model, SAPS II was negatively correlated with survival time of elderly septic patients (r=−0.29, P<0.01, Figure 4A). Higher SAPS II scores denote shorter life expectancy. The cut-off value of SAPS II in the prediction of mortality was 43 with the sensitivity of 57.6 and specificity of 65.3 as calculated by the ROC curve analysis. The four-year all-cause mortality of patients with an SAPS II score ≤43 was significantly lower than patients with an SAPS II score >43 (64.8% vs. 82.5%, P<0.01). We then compared the survival time of patients with an SAPS II score ≤43 and an SAPS II score >43 in each group. The result showed that elderly septic patients with an SAPS II score ≤43 had a longer survival time irrespective of age (P<0.01, Figure 4B). As per the K-M analysis, patients in the oldest-old group with an SAPS II score >43 had the highest risk of mortality (all P<0.01, Figure 4C).

Figure 4 Relationship between the cut-off value of SAPS II and survival for elderly septic patients. In the linear regression model, the survival time was negatively correlated to the Simplified Acute Physiology Score II (SAPS II) for the elderly septic patients (r=−0.29, P<0.01, A). By the receiver operating characteristic curve, 43 was defined as the cut-off value of SAPS II. For all patients and each group, the survival time of patients with an SAPS II score ≤43 was much higher than that of patients with an SAPS II score >43 (B). Regardless of ages, septic patients with an SAPS II score ≤43 had the higher survival rate (P<0.01, C).

Discussion

Age has long been identified as a risk factor of poor prognosis in septic patients. In this study, we used the data of a large cohort of elderly patients with sepsis admitted to the ICU to predict the long-term prognosis of these patients. We found that long-term mortality was increased with age. Among the five severity scales, i.e., SAPS II, SOFA, SIRS, OASIS and MLODS, SAPS II had the best prognostic value for the 4-year all-cause mortality. A cut-off value of 43 appears able to better predict the long-term prognosis of patients.

The prognosis of the elderly with sepsis admitted to the ICU is usually poor (34). Older ages were closely related to poor outcomes leading to high mortality at all stages of sepsis (35). Age was also considered as the risk factor for hospital death (36). In our study, mortality of sepsis in the elderly was positively associated with age. The influence of age on mortality in elderly septic patients might be due to frailty, multiple chronic disorders, organ dysfunctions, polypharmacy and so forth (37). As for gender, no difference as to the incidence of sepsis was seen between males and females. In both the Norwegian and the Spanish cohorts, severe sepsis was observed to be more common in men (36,37), which might be related to the gender differences in levels of care, or responses to the sepsis (38).

We compared five commonly used scales for long-term prognosis of elderly septic patients. Indeed, various factors may influence the selection of severity scales in the clinical practice, such as time consumed, cost spent, inter-rater reliability and so forth. Notwithstanding, in this study, we only focus on the effectiveness of prognostic prediction. Prognosis solely being taken into account, SAPS II appears superior to the other four scales. Although SAPS III has been developed for clinical research, its superiority has yet to be corroborated. Most researchers deem SAPS II efficient in the differentiation of survivors and non-survivors in septic patients (32,33,39). For example, SAPS III is less efficient than SAPS II in predicting the mortality of ICU patients (32); SAPS II and SOFA performed comparably in the prognostication of 30-day mortality in septic patients (39). Indeed, SAPS II has been proved efficient to predict in-hospital mortality of critically ill patients but not long-term mortality (33). However, the above-mentioned study was based on a cohort of patients at all ages, instead of the elderly only. In our study, SAPS II shows superior prognostic efficiency to other scales. Comparison studies on commonly used severity scales in the elderly with sepsis are scarce. Literature searching on the PubMed yielded only one study in which Tiruvoipati et al. confirmed superior prognostic value of SAPS II to the Acute Physiology Age and Chronic Health Evaluation III score for in-hospital mortality in critically ill elderly patients with sepsis (40). Of note is that only those two severity scales were included for comparison in Tiruvoipati et al.’s study.

Usually patients with an SAPS II score over 52 have an in-hospital mortality of over 50% (41,42). In the present study, we found that 43 was the cut-off value of SAPS II to predict the survivorship of elderly septic patients, since patients with a SAPS II over 52 might die in short-term, which contributed less to the long-term prognosis. The K-M analysis showed that patients with an SAPS II score >43 had worse prognosis regardless of age, which means that SAPS II could be a more potent predictor than age. The cut-off value identified in our study was similar to that reported in hospital mortality of critically ill patients age over 90 (43). For patients of all ages, the cut-off value was 49 with the sensitivity of 0.5 and specificity of 0.95 (44), which was higher than that in our study, probably owing to dysfunction and aging of multiple organs of aged patients. The specific cut-off value of the elderly may help physicians to identify patients at high risk so as to administer intensive care and treatment. Likewise, our results also provide evidence for physicians to provide appropriate post-hospital guidance to elderly septic patients. Raising awareness of early and effective rehabilitation for elderly patients with an SAPS score >43 may improve the life expectancy after discharge (45,46).

The large sample size with long-term follow-up data is the strength of our study. However, our study bears several limitations. First, direct cause of ICU admission and etiological data of sepsis in this cohort were unavailable, and hence stratified analysis was impossible. Second, all AUC values are lower than 0.8, indicating that common severity scales are inaccurate for elderly patients. Although the AUC values are not satisfactory, the results of comparisons among the commonly used scales urge further prospective studies and the development of novel scales in this population. Third, as with numerous assessments in the clinical practice, severity scales were used at single time-points in our study, which might be confounded by various clinical interventions, such as levels of organ support and medications (26). Thus, standard criteria for the assessment and management of patients are necessary to minimize inter-patient variability (47,48). Fourth, the cut-off value of SAPS II was not validated in another cohort because of the retrospective nature of our study. Taken together, further prospective multi-center cohort studies are needed to verify the usefulness of SAPS II in the prognostication of elderly septic patients.

Conclusions

According to the clinical data of 4,370 elderly septic patients in the MIMIC-III database, SAPS II proved to be better as compared with MLODS, OASIS, SIRS and SOFA in the prediction the long-term mortality of patients. The value of 43 was the cut-off value to discriminate the survivorship in the elderly with sepsis.

Acknowledgments

We appreciate the efforts the staff has made to build and maintain the Medical Information Mart for Intensive Care III (MIMIC-III) database. We also thank all researchers and patients involved who are willing to share their data for research purposes.

Funding: The study was supported by grants from Wu Jieping Medical Foundation Clinical Research Funding (No.320.6750.16050), Beijing Excellent Talents Foundation (No. 2018000020124G144), National Natural Science Foundation of China Youth Program (No. 81900655) and General Projects of Science and Technology Plan of Beijing Municipal Commission of Education (No. KM202010025022).

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at http://dx.doi.org/10.21037/apm-20-1355

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/apm-20-1355). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The Medical Information Mart for Intensive Care III (MIMIC-III) database was approved by the Massachusetts Institute of Technology and the Institutional Review Boards. One researcher (LJ) has passed the Protecting Human Research Participants exam of National Institutes of Health (Record ID: 27638410) and gained permissible access to the MIMIC-III database. Because eighteen identifying data elements were removed from the MIMIC-III database according to the Health Insurance Portability and Accountability Act (HIPAA) standards (www.hhs.gov), the approval of ethics committee and individual patient consents were waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Vosylius S, Sipylaite J, Ivaskevicius J. Determinants of outcome in elderly patients admitted to the intensive care unit. Age Ageing 2005;34:157-62. [Crossref] [PubMed]
2. Nielsson MS, Christiansen CF, Johansen MB, et al. Mortality in elderly ICU patients: a cohort study. Acta Anaesthesiol Scand 2014;58:19-26. [Crossref] [PubMed]
3. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315:762-74. [Crossref] [PubMed]
4. Rhee C, Dantes R, Epstein L, et al. Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009-2014. JAMA 2017;318:1241-9. [Crossref] [PubMed]
5. Haas LE, van Dillen LS, de Lange DW, et al. Outcome of very old patients admitted to the ICU for sepsis: a systematic review. Eur Geriatr Med 2017;8:446-53. [Crossref]
6. Martin GS, Mannino DM, Eaton S, et al. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med 2003;348:1546-54. [Crossref] [PubMed]
7. Martin-Loeches I, Guia MC, Vallecoccia MS, et al. Risk factors for mortality in elderly and very elderly critically ill patients with sepsis: a prospective, observational, multicenter cohort study. Ann Intensive Care 2019;9:26. [Crossref] [PubMed]
8. Franceschi C, Garagnani P, Parini P, et al. Inflammaging: a new immune-metabolic viewpoint for age-related diseases. Nat Rev Endocrinol 2018;14:576-90. [Crossref] [PubMed]
9. Li X, Wang J, Wang L, et al. Impaired lipid metabolism by age-dependent DNA methylation alterations accelerates aging. Proc Natl Acad Sci U S A 2020;117:4328-36. [Crossref] [PubMed]
10. Umberger R, Callen B, Brown ML. Severe sepsis in older adults. Crit Care Nurs Q 2015;38:259-70. [Crossref] [PubMed]
11. Ferrucci L, Fabbri E. Inflammageing: chronic inflammation in ageing, cardiovascular disease, and frailty. Nat Rev Cardiol 2018;15:505-22. [Crossref] [PubMed]
12. Xu D, Liao S, Li P, et al. Metabolomics Coupled with Transcriptomics Approach Deciphering Age Relevance in Sepsis. Aging Dis 2019;10:854-70. [Crossref] [PubMed]
13. de Groot B, Stolwijk F, Warmerdam M, et al. The most commonly used disease severity scores are inappropriate for risk stratification of older emergency department sepsis patients: an observational multi-centre study. Scand J Trauma Resusc Emerg Med 2017;25:91. [Crossref] [PubMed]
14. Liang W, Chen C, Li L, et al. Effect of immune function on prognosis of patients with sepsis. Zhonghua Wei Zhong Bing Ji Jiu Yi Xue 2018;30:1128-31. [PubMed]
15. Arnau-Barrés I, Güerri-Fernández R, Luque S, et al. Serum albumin is a strong predictor of sepsis outcome in elderly patients. Eur J Clin Microbiol Infect Dis 2019;38:743-6. [Crossref] [PubMed]
16. Malhotra R, Kashani KB, Macedo E, et al. A risk prediction score for acute kidney injury in the intensive care unit. Nephrol Dial Transplant 2017;32:814-22. [Crossref] [PubMed]
17. Vincent JL. Organ dysfunction in patients with severe sepsis. Surg Infect (Larchmt) 2006;7:S69-72. [Crossref] [PubMed]
18. Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991;100:1619-36. [Crossref] [PubMed]
19. Johnson AE, Kramer AA, Clifford GD. A new severity of illness scale using a subset of Acute Physiology And Chronic Health Evaluation data elements shows comparable predictive accuracy. Crit Care Med 2013;41:1711-8. [Crossref] [PubMed]
20. Le Gall JR, Klar J, Lemeshow S, et al. The Logistic Organ Dysfunction system. A new way to assess organ dysfunction in the intensive care unit. ICU Scoring Group. JAMA 1996;276:802-10. [Crossref] [PubMed]
21. Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest 1992;101:1644-55. [Crossref] [PubMed]
22. Vincent JL, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 1996;22:707-10. [Crossref] [PubMed]
23. Schinkel M, Paranjape K, Nannan Panday RS, et al. Clinical applications of artificial intelligence in sepsis: A narrative review. Comput Biol Med 2019;115:103488 [Crossref] [PubMed]
24. Rudd KE, Seymour CW, Aluisio AR, et al. Association of the Quick Sequential (Sepsis-Related) Organ Failure Assessment (qSOFA) Score With Excess Hospital Mortality in Adults With Suspected Infection in Low- and Middle-Income Countries. JAMA 2018;319:2202-11. [Crossref] [PubMed]
25. Inada-Kim M, Nsutebu E. NEWS 2: an opportunity to standardise the management of deterioration and sepsis. BMJ 2018;360:k1260. [Crossref] [PubMed]
26. Lambden S, Laterre PF, Levy MM, et al. The SOFA score-development, utility and challenges of accurate assessment in clinical trials. Crit Care 2019;23:374. [Crossref] [PubMed]
27. Burns A. NEWS 2 sepsis score is not validated in primary care. BMJ 2018;361:k1743. [Crossref] [PubMed]
28. Abe T, Ogura H, Shiraishi A, et al. Characteristics, management, and in-hospital mortality among patients with severe sepsis in intensive care units in Japan: the FORECAST study. Crit Care 2018;22:322. [Crossref] [PubMed]
29. Metnitz PG, Moreno RP, Almeida E, et al. SAPS 3--From evaluation of the patient to evaluation of the intensive care unit. Part 1: Objectives, methods and cohort description. Intensive Care Med 2005;31:1336-44. [Crossref] [PubMed]
30. Johnson AE, Pollard TJ, Shen L, et al. MIMIC-III, a freely accessible critical care database. Sci Data 2016;3:160035 [Crossref] [PubMed]
31. Rhodes A, Evans LE, Alhazzani W, et al. Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock: 2016. Intensive Care Med 2017;43:304-77. [Crossref] [PubMed]
32. Bisbal M, Jouve E, Papazian L, et al. Effectiveness of SAPS III to predict hospital mortality for post-cardiac arrest patients. Resuscitation 2014;85:939-44. [Crossref] [PubMed]
33. Honselmann KC, Buthut F, Heuwer B, et al. Long-term mortality and quality of life in intensive care patients treated for pneumonia and/or sepsis: Predictors of mortality and quality of life in patients with sepsis/pneumonia. J Crit Care 2015;30:721-6. [Crossref] [PubMed]
34. Stoller J, Halpin L, Weis M, et al. Epidemiology of severe sepsis: 2008-2012. J Crit Care 2016;31:58-62. [Crossref] [PubMed]
35. Cag Y, Karabay O, Sipahi OR, et al. Development and validation of a modified quick SOFA scale for risk assessment in sepsis syndrome. PLoS One 2018;13:e0204608 [Crossref] [PubMed]
36. Bouza C, Lopez-Cuadrado T, Saz-Parkinson Z, et al. Epidemiology and recent trends of severe sepsis in Spain: a nationwide population-based analysis (2006-2011). BMC Infect Dis 2014;14:3863. [Crossref] [PubMed]
37. Knoop ST, Skrede S, Langeland N, et al. Epidemiology and impact on all-cause mortality of sepsis in Norwegian hospitals: A national retrospective study. PLoS One 2017;12:e0187990 [Crossref] [PubMed]
38. Adrie C, Azoulay E, Francais A, et al. Influence of gender on the outcome of severe sepsis: a reappraisal. Chest 2007;132:1786-93. [Crossref] [PubMed]
39. Goswami J, Balwani MR, Kute V, et al. Scoring systems and outcome of chronic kidney disease patients admitted in intensive care units. Saudi J Kidney Dis Transpl 2018;29:310-7. [Crossref] [PubMed]
40. Tiruvoipati R, Ong K, Gangopadhyay H, et al. Hypothermia predicts mortality in critically ill elderly patients with sepsis. BMC Geriatr 2010;10:70. [Crossref] [PubMed]
41. Godinjak A, Iglica A, Rama A, et al. Predictive value of SAPS II and APACHE II scoring systems for patient outcome in a medical intensive care unit. Acta Med Acad 2016;45:97-103. [Crossref] [PubMed]
42. Le Gall JR, Lemeshow S, Saulnier F. A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 1993;270:2957-63. [Crossref] [PubMed]
43. Haq A, Patil S, Parcells AL, et al. The Simplified Acute Physiology Score III Is Superior to the Simplified Acute Physiology Score II and Acute Physiology and Chronic Health Evaluation II in Predicting Surgical and ICU Mortality in the "Oldest Old". Curr Gerontol Geriatr Res 2014;2014:934852 [Crossref] [PubMed]
44. Cosentini R, Folli C, Cazzaniga M, et al. Usefulness of simplified acute physiology score II in predicting mortality in patients admitted to an emergency medicine ward. Intern Emerg Med 2009;4:241-7. [Crossref] [PubMed]
45. Mehlhorn J, Freytag A, Schmidt K, et al. Rehabilitation interventions for postintensive care syndrome: a systematic review. Crit Care Med 2014;42:1263-71. [Crossref] [PubMed]
46. Schmidt K, Worrack S, Von Korff M, et al. Effect of a Primary Care Management Intervention on Mental Health-Related Quality of Life Among Survivors of Sepsis: A Randomized Clinical Trial. JAMA 2016;315:2703-11. [Crossref] [PubMed]
47. Vincent JL, Sakr Y. Clinical trial design for unmet clinical needs: a spotlight on sepsis. Expert Rev Clin Pharmacol 2019;12:893-900. [Crossref] [PubMed]
48. Singer M. Sepsis: personalization v protocolization? Crit Care 2019;23:127. [Crossref] [PubMed]