Development of long-term cardiovascular disease risk prediction model for hemodialysis patients with end-stage renal disease based on nomogram

Introduction

Chronic kidney disease (CKD) has been recognized as a leading public health problem worldwide, with high morbidity and mortality (1-3). Kidney failure, especially end-stage kidney disease (ESKD), is a leading cause of mortality. Patients with ESKD require kidney replacement therapy, including hemodialysis, abdominal dialysis, and renal transplant. In hemodialysis patients, cardiovascular events are the primary cause of death, and more than half of known deaths are related to cardiovascular causes (4). Therefore, risk assessment of cardiovascular events and individualized management and treatment for hemodialysis patients are necessary. The development of a simple and visual clinical assessment tool has the potential to achieve these goals.

The most commonly used model to predict the rate of cardiovascular events is the Framingham risk score (FRS) (5). The FRS was developed to predict the cardiovascular risk for the general population in 10 years by considering traditional risk factors, including cigarette smoking, elevated blood pressure, elevated serum total cholesterol (TC), low serum high-density lipoprotein (HDL) cholesterol, diabetes mellitus (DM), and advanced age. However, several researchers have noted that some risk factors predicting high cardiovascular mortality (CVM) in hemodialysis patients may differ from those in the general population (6,7). Therefore, using the FRS to assess cardiovascular risk in hemodialysis patients may be inadequate, for some factors, such as low blood pressure and low TC (rather than high blood pressure and high TC), are associated with higher cardiovascular mortality and total mortality in ESKD in the reverse epidemiology (8,9). Anker et al. developed 2 separate risk scores, CVM and CVM and mortality (CVMM), for hemodialysis patients (10). Both risk scores were found to be more predictive than FRS, and additionally, the CVM score was more predictive than the CVMM score. We speculated that patients with non-fatal and fatal cardiovascular events might have diverse risk factors. Therefore, it is important to use different evaluation indicators to evaluate the risk of non-fatal or fatal cardiovascular events.

In the present study, we used the time of the first cardiovascular event, routine blood test indicators, and ultrasonic cardiogram (UCG) parameters to build a Cox proportional hazards model and to further to construct a nomogram to predict the risk of cardiovascular event occurrence in 3, 5, and 10 years among long-term hemodialysis patients. Our nomogram-based risk prediction model is a simple and visual assessment tool, and may be valuable to identify high-risk patients for earlier intervention.

We present the following article in accordance with the TRIPOD reporting checklist (available at http://dx.doi.org/10.21037/apm-21-286).

Methods

Study population and ethics

A cohort of 370 patients with hemodialysis were enrolled in the present study. The patients were recruited between January 2015 and September 2019 at the hemodialysis centers of the Third Affiliated Hospital of Southern Medical University and the Third Affiliated Hospital of Sun Yat-sen University. Inclusion criteria were hemodialysis >3 months due to renal failure. Exclusion criteria included non-renal failure, dialysis duration <3 months, and irregular dialysis. The end time of follow-up was May 31, 2020. The study was approved by the institutional review board of the Third Affiliated Hospital of Southern Medical University and the Third Affiliated Hospital of Sun Yat-sen University. All participants provided informed written consent before participating in the study. All procedures performed in this study involving human participants were in accordance with the Declaration of Helsinki (as revised in 2013).

Serum sample collection and test

Overnight fasting (12 h) blood samples were collected from all patients before first hemodialysis between January 2015 and September 2019. Biochemical measurements of procalcitonin, intact parathyroid hormone, serum ferritin (FER), serum phosphate, TC, triglyceride (TG), HDL, low-density lipoprotein cholesterol (LDL), apolipoprotein A, apolipoprotein B, C-reactive protein (CRP), white blood cells (WBC), hemoglobin (HB), platelet count, neutrophil absolute value (NEUT), lymphocyte absolute value, creatinine (CRE), blood urea nitrogen, serum calcium, uric acid, blood glucose (GLU), unsaturated iron-binding capacity, total iron-binding capacity, creatine kinase MB (CK-MB), and estimated glomerular filtration rate were performed using standard laboratory procedures.

UCG

UCG was performed by trained operators before first hemodialysis at the radiology centers in the Third Affiliated Hospital of Southern Medical University and the Third Affiliated Hospital of Sun Yat-sen University between January 2015 and September 2019. The test included aortic root diameter, left atrium diameter (LAD), right ventricle diameter (RVD), left ventricular end-diastolic diameter, main pulmonary artery diameter, interventricular septum thickness (IVST), left ventricular posterior wall thickness (LVPWT), ejection fraction (EF), left ventricular fraction shortening, mitral valve peak E velocity (E), mitral valve peak A velocity (A), and the ratio of E/A.

Outcome

We defined the development of the first non-fatal and fatal cardiovascular event after hemodialysis as the outcome. Cardiovascular endpoint events in CKD patients included coronary artery disease, chronic coronary artery disease, valvular heart disease, myocardial disease, arrhythmia, extracorporeal defibrillation, cerebrovascular disease, and peripheral vascular disease according to the Kidney Disease: Improving Global Outcomes 2012 clinical practice guideline (11). The time of the events was from the beginning of hemodialysis treatment to the first onset of cardiovascular events. Data were collected from the hemodialysis centers until May 31, 2020.

Development and validation of the prediction model

Marker selection

Variables with a missing amount of more than 50% were not included in this study and the remaining variables were filled with the mean value within the group. Two steps were involved for marker selection. First, 3 methods (stepwise logistic regression, hypothesis testing (Student’s t-test for normal parameters, Wilcoxon-Mann-Whitney test for non-normal parameters, and χ²-test for categorical variables), and the nomogram) were applied separately for all parameters. Parameters that met 1 condition (stepwise logistic regression P<0.05, hypothesis testing, P<0.05, or hazard ratio >50% in the nomogram) were screened out for further selection. Second, all possible combinations among the selected parameters were used to construct numerous Cox proportional hazards models. The final optimal parameter set was determined by balancing the number of parameters and the model’s performance [primarily based on the value of the Harrell’s concordance index (C-index)] (Figure 1).

Figure 1 Flowchart of the study design. In step 1 of parameter set selection, stepwise logistic regression, hypothesis testing, and nomogram were used jointly for preselection, and the final set was determined from all possible combinations. In step 2 of model construction and validation, a Cox proportional hazards model coupled with a nomogram were constructed using the optimal parameter set and multivariate regression analysis. Then, Self-validation, 10-fold cross-validation, and hold-out validation (70% training and 30% validation, repeated 100 times) were used to verify the reliability and accuracy of the model with C-index and AUROC.

Model construction and validation

On the basis of the optimized parameters and using multivariate regression analysis, the Cox proportional hazards model was built. Kaplan-Meier estimates were used to generate survival curves. The regression coefficient for each covariate could be converted to a hazard ratio, which was used to construct the nomogram.

The area under the receiver operating characteristic curve (AUROC) was calculated to evaluate the performance of the risk-prediction models. The predictive accuracy and discriminative ability of the nomogram were determined by C-index and the calibration curve. The C-index indicates the consistency between the predicted results and the actual results. Calibration was performed by comparing the means of predicted survival with those of actual survival.

Furthermore, considering no independent validation sets in the present study, 10-fold cross-validation and hold-out validation (70% training and 30% validation, repeated 100 times) were used to validate the reliability of our model (Figure 1).

Statistical analysis

Student’s t-test (for normal parameters), Wilcoxon-Mann-Whitney test (for non-normal parameters), and χ²-test (for categorical variables) were used for comparisons between 2 groups. Statistical significance was set at P<0.05. Survival curves were generated according to the Kaplan-Meier method. Multivariate analyses were performed using the Cox proportional hazards model. Cox regression coefficients were used to generate a nomogram.

R (version 4.0.3, https://www.r-project.org/) was used for data analysis and figure plotting in the study. R is a language and environment for statistical computing and graphics. It was initially written by Robert Gentleman and Ross Ihaka of the Statistics Department of the University of Auckland. The home page address is https://www.r-project.org/. Nomogram and calibration plots were generated using the rms (version 6.0-1, written by Frank Harrell, https://CRAN.R-project.org/package=rms) package. Cox proportional hazards model and C-index were generated using the survival (version 3.2-7, written by Terry M. Therneau, et al., https://CRAN.R-project.org/package=survival) package. AUROCs were calculated using the survivalROC (version 1.0.3, written by Patrick J. Heagerty, https://CRAN.R-project.org/package=survivalROC) and timeROC (version 0.4, written by Paul Blanche, https://CRAN.R-project.org/package=timeROC) package.

Results

Basic characteristics of the participants

A total of 370 patients with hemodialysis were involved in the present study. A total of 121 patients (32.7%) had a first cardiovascular event and 249 (67.3%) did not have any cardiovascular events. Patients with cardiovascular events were older and were more likely to have hypertension and DM. Their FER, CRP, WBC, GLU, and CK-MB levels were higher and their CRE levels were lower than those of patients with no cardiovascular events. UCG inspection results indicated that the values of LAD, RVD, IVST, LVPWT, A, and E/A were higher in patients with cardiovascular events and the values of EF and E were lower (Table 1).

Table 1 Clinical and demographical characteristics of hemodialysis patients with chronic kidney disease
Full table

Optimal parameter set selection

Two steps were conducted for optimal parameter set selection. After the first step, 20 of the 42 parameters were preselected by stepwise logistic regression, hypothesis testing, and nomogram as follows: hypertension, DM, age, FER, phosphate, TG, CRP, WBC, NEUT, CRE, GLU, CK-MB, LAD, RVD, IVST, LVPWT, EF, E, A, and E/A. In the second step, all possible parameter combinations among them were used to construct numerous Cox proportional hazards models. Eight parameters were finally selected, balancing the number of parameters used and the values of the C-index. The optimal parameter set consisted of hypertension, DM, age, phosphate, TG, CRP, WBC, and IVST.

Model construction

A Cox proportional hazards model was constructed using the optimal parameter set. The nomogram was constructed from the hazard ratio converted by the coefficients for each covariate to predict the probability of cardiovascular events among long-term hemodialysis patients (Figure 2). Each variable was assigned a score (ranging from 0 to 100) and a total point (ranging from 0 to 300) that was calculated for each patient. Furthermore, the sum risk scores were converted to predict 3-year, 5-year, and 10-year risk of a cardiovascular events.

Figure 2 Nomogram for the risk prediction of cardiovascular events in hemodialysis patients. Length of each variable axis in the nomogram varies. First, draw a line upward to determine the score of each variable in the points axis. Second, find the location of the sum scores of these variables in the total points axis. Finally, draw a line down to the morbidity axes to determine the possibility of 3-, 5- or 10-year cardiovascular event occurrence risk.

The calibration curve showed good agreement between predicted and observed outcomes for the cardiovascular event prediction model (Figure 3A,B,C). The C-index of the nomogram was 0.808 [95% confidence interval (CI): 0.773–0.844]. The AUROCs for predicting 3-, 5- and 10-year cardiovascular event risk were 0.836, 0.845, and 0.869, respectively (Figure 3D). The time-dependent AUROCs and the 95% CI of the nomogram at various time points are shown in Figure 3E. As expected, our model performed well in predicting cardiovascular event occurrence.

Figure 3 Performance of the nomogram in the risk assessment of cardiovascular events in hemodialysis patients. (A-C) Calibration curve of the nomogram. X-axis represents the nomogram-predicted morbidity and the y-axis represents actual morbidity; 95% confidence intervals (CIs) were measured by Kaplan-Meier analysis. (D) Time-dependent receiver-operating characteristic (ROC) curves of cardiovascular event risk prediction in 3 (red), 5 (green), and 10 (blue) years. (E) Line graph of the time-dependent area under the receiver-operating characteristic curve (AUROC; solid line) and its 95% CI (dashed lines) for cardiovascular event risk prediction.

Model validation

Two methods, 10-fold cross-validation and hold-out validation (70% training and 30% validation, repeated 100 times), were used to validate the reliability of our model. For the 10-fold cross-validation, the mean of the C-indexes was 0.794, and the time-dependent AUROCs for predicting 3-, 5-, and 10-year cardiovascular events were 0.832, 0.832, and 0.845, respectively (Figure 4A). For the hold-out validation, the mean of the C-indexes was 0.798, and the time-dependent AUROCs for predicting 3-, 5-, and 10-year cardiovascular events were 0.822, 0.832, and 0.860, respectively (Figure 4B). Both validation methods demonstrated satisfactory predictive ability of our cardiovascular event prediction model.

Figure 4 Validation of the nomogram in the risk assessment of cardiovascular events in hemodialysis patients. (A) Time-dependent ROC curves in 3 (red), 5 (green), and 10 (blue) years using 10-fold cross-validation. (B) Time-dependent ROC curves in 3 (red), 5 (green), and 10 (blue) years using hold-out validation (70% training and 30% validation, repeated 100 times).

Discussion

In the present study, we developed a Cox proportional hazards model and a nomogram for long-term hemodialysis patients to predict the risk of cardiovascular event occurrence. A total of 8 indicators (hypertension, DM, age, phosphate, TG, CRP, WBC, and IVST) were included in the model. Self-validation, 10-fold cross-validation, and hold-out validation all verified the reliability and accuracy of our model with high C-indexes and AUROCs. Therefore, our model is particularly valuable for clinically identifying high-risk hemodialysis patients and facilitating the development of personalized patient care. It can also be used as a basis for further research on cardioprotective hemodialysis strategies.

It has been confirmed that cardiovascular events are the leading cause of death in long-term hemodialysis patients (4). In 1999, the FRS was developed to predict the rate of cardiovascular events in the general population in 10 years (5). It reported that smoking, hypertension, DM, advanced age, elevated TC, elevated LDL, and reduced HDL were the major independent risk factors. long-term hemodialysis may lead to changes in pathophysiology, including anemia, microinflammatory state, water and sodium retention, calcium and phosphorus metabolism disorders, malnutrition, etc. these factors may interact with each other, and may affect the cardiovascular system, leading to hypertension, coronary heart disease, arrhythmia, cerebrovascular accident, congestive heart failure, and other cardiovascular diseases (12). Various studies on cardiovascular risk factors in dialysis patients have since been published, and several risk-prediction methods constructed (10,13-17). Serum phosphorus (13,18-20), calcium (10,19,20), WBC (21,22), CRP (10,16,23,24), and albumin (7,10,13,17) levels were also considered common risk predictors of cardiovascular events. In our study, we used 8 indicators (hypertension, DM, age, phosphate, TG, CRP, WBC, and IVST) to establish our risk-prediction model. Most of our parameters were strongly associated cardiovascular event risk, supporting the findings of previously published studies and indicating the reliability of our model. We also used UCG relative indicators in the present study and found that IVST was a good predictor. This is one of the strengths of our study. Many studies have indicated that combining serological indicators with imaging indicators can improve diagnostic performance (25-28). Other advantages of this study are as follows. First, all the parameters involved in this research can obtained from routine medical examinations and are not restricted by medical conditions. Even with poor clinical practice and medical resources, the indicators we chose are affordable. Second, our nomogram model not only had a satisfactory performance but also allowed the visual tracking of the risk assessment. It does not require complex calculations and is fairly straightforward for clinicians and patients to understand. Finally, the 2-step parameter selection strategy was used to ensure that indicators reached their full prediction potential. First carried on the parameter pre-selection using multiple methods (stepwise logistic regression, hypothesis testing, and nomogram) and then compared the results of all possible parameter combinations, which was time consuming but necessary.

However, our study also has some limitations. First, the prediction model was designed for renal failure patients on hemodialysis. Whether non-renal failure and irregular dialysis patients were suitable for this model was not clear and requires further validation. Second, the present study was based on Chinese patients. Further validation with diverse populations and races is essential. Finally, the present study was a cross-sectional study and longitudinal studies are needed to further validate the reproducibility of the current findings and the application value of our model in cardiovascular event risk prediction.

In conclusion, our nomogram model constructed from non-invasive conventional serological and UCG parameters was found to accurately predict individual cardiovascular event risk in hemodialysis patients. In clinical practice, this model can be used to identify high-risk individuals for more precise and efficient personalized management and treatment. In view of the poor prognosis of long-term hemodialysis in patients with end-stage renal disease, it is expected to carry out further research on renal prognosis and death in the future.

Acknowledgments

Funding: This study was supported by grants from the Key Project of Shenzhen Science and Technology Innovation Committee (No. JSGG20200225152709802), Major Science and Technology Projects in 13th Five-Year Plan (No. 2018ZX10715004-001-009), Natural Science Foundation of Guangdong (No. 2018A030310272), Guangdong Medical Science and Technology Research Fund (No. A2018366), the Third Affiliated Hospital of Sun Yat-Sen University Clinical Research Program (No. P000-277), and the Meizhou Science and Technology Project (No. 2019B0203003).

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at http://dx.doi.org/10.21037/apm-21-286

Data Sharing Statement: Available at http://dx.doi.org/10.21037/apm-21-286

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/apm-21-286). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was approved by the institutional review board of the Third Affiliated Hospital of Southern Medical University and the Third Affiliated Hospital of Sun Yat-sen University. All participants provided informed written consent before participating in the study. All procedures performed in this study involving human participants were in accordance with the Declaration of Helsinki (as revised in 2013).

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Coresh J, Selvin E, Stevens LA, et al. Prevalence of chronic kidney disease in the United States. Jama 2007;298:2038-47. [Crossref] [PubMed]
2. Lozano R, Naghavi M, Foreman K, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012;380:2095-128. [Crossref] [PubMed]
3. Zhang L, Wang F, Wang L, et al. Prevalence of chronic kidney disease in China: a cross-sectional survey. Lancet 2012;379:815-22. [Crossref] [PubMed]
4. Saran R, Robinson B, Abbott KC, et al. US Renal Data System 2019 Annual Data Report: Epidemiology of Kidney Disease in the United States. Am J Kidney Dis 2020;75:A6-7. [Crossref] [PubMed]
5. Grundy SM, Pasternak R, Greenland P, et al. Assessment of cardiovascular risk by use of multiple-risk-factor assessment equations: a statement for healthcare professionals from the American Heart Association and the American College of Cardiology. Circulation 1999;100:1481-92. [Crossref] [PubMed]
6. Shoji T, Masakane I, Watanabe Y, et al. Elevated non-high-density lipoprotein cholesterol (non-HDL-C) predicts atherosclerotic cardiovascular events in hemodialysis patients. Clin J Am Soc Nephrol 2011;6:1112-20. [Crossref] [PubMed]
7. Ma L, Zhao S. Risk factors for mortality in patients undergoing hemodialysis: A systematic review and meta-analysis. Int J Cardiol 2017;238:151-8. [Crossref] [PubMed]
8. Huang JC, Chen SC, Su HM, et al. Performance of the Framingham risk score in patients receiving hemodialysis. Nephrology (Carlton) 2013;18:510-5. [Crossref] [PubMed]
9. Weiner DE, Tighiouart H, Elsayed EF, et al. The Framingham predictive instrument in chronic kidney disease. J Am Coll Cardiol 2007;50:217-24. [Crossref] [PubMed]
10. Anker SD, Gillespie IA, Eckardt KU, et al. Development and validation of cardiovascular risk scores for haemodialysis patients. Int J Cardiol 2016;216:68-77. [Crossref] [PubMed]
11. . Kidney Disease: Improving Global Outcomes (KDIGO) CKD Work Group. KDIGO clinical practice guideline for the evaluation and management of chronic kidney disease. Kidney Int Suppl 2013;3:1-150.
12. David P, Navino C, Capurro F, et al. Native vascular access for hemodialysis in patients with diabetes: a single-center experience. G Ital Nefrol 2010;27:522-6. [PubMed]
13. Matsubara Y, Kimachi M, Fukuma S, et al. Development of a new risk model for predicting cardiovascular events among hemodialysis patients: Population-based hemodialysis patients from the Japan Dialysis Outcome and Practice Patterns Study (J-DOPPS). PLoS One 2017;12:e0173468 [Crossref] [PubMed]
14. House AA, Ronco C. Cardiovascular risk in hemodialysis patients: a mechanistic approach. Int J Artif Organs 2007;30:1020-7. [Crossref] [PubMed]
15. Poleszczuk J, Debowska M, Dabrowski W, et al. Patient-specific pulse wave propagation model identifies cardiovascular risk characteristics in hemodialysis patients. PLoS Comput Biol 2018;14:e1006417 [Crossref] [PubMed]
16. Russa D, Pellegrino D, Montesanto A, et al. Oxidative Balance and Inflammation in Hemodialysis Patients: Biomarkers of Cardiovascular Risk? Oxid Med Cell Longev 2019;2019:8567275 [Crossref] [PubMed]
17. Sun J, Axelsson J, Machowska A, et al. Biomarkers of Cardiovascular Disease and Mortality Risk in Patients with Advanced CKD. Clin J Am Soc Nephrol 2016;11:1163-72. [Crossref] [PubMed]
18. Jamal SA, Vandermeer B, Raggi P, et al. Effect of calcium-based versus non-calcium-based phosphate binders on mortality in patients with chronic kidney disease: an updated systematic review and meta-analysis. Lancet 2013;382:1268-77. [Crossref] [PubMed]
19. Slinin Y, Foley RN, Collins AJ. Calcium, phosphorus, parathyroid hormone, and cardiovascular disease in hemodialysis patients: the USRDS waves 1, 3, and 4 study. J Am Soc Nephrol 2005;16:1788-93. [Crossref] [PubMed]
20. Stevens LA, Djurdjev O, Cardew S, et al. Calcium, phosphate, and parathyroid hormone levels in combination and as a function of dialysis duration predict mortality: evidence for the complexity of the association between mineral metabolism and outcomes. J Am Soc Nephrol 2004;15:770-9. [Crossref] [PubMed]
21. Reddan DN, Klassen PS, Szczech LA, et al. White blood cells as a novel mortality predictor in haemodialysis patients. Nephrol Dial Transplant 2003;18:1167-73. [Crossref] [PubMed]
22. Ekdahl KN, Soveri I, Hilborn J, et al. Cardiovascular disease in haemodialysis: role of the intravascular innate immune system. Nat Rev Nephrol 2017;13:285-96. [Crossref] [PubMed]
23. Böger CA, Götz A, Stubanus M, et al. C-reactive protein as predictor of death in end-stage diabetic nephropathy: role of peripheral arterial disease. Kidney Int 2005;68:217-27. [Crossref] [PubMed]
24. Chou CY, Kuo HL, Yung YF, et al. C-reactive protein predicts vascular access thrombosis in hemodialysis patients. Blood Purif 2006;24:342-6. [Crossref] [PubMed]
25. Calès P, Boursier J, Ducancelle A, et al. Improved fibrosis staging by elastometry and blood test in chronic hepatitis C. Liver Int 2014;34:907-17. [Crossref] [PubMed]
26. Boursier J, Guillaume M, Leroy V, et al. New sequential combinations of non-invasive fibrosis tests provide an accurate diagnosis of advanced fibrosis in NAFLD. J Hepatol 2019;71:389-96. [Crossref] [PubMed]
27. Betts MJ, Kirilina E, Otaduy MCG, et al. Locus coeruleus imaging as a biomarker for noradrenergic dysfunction in neurodegenerative diseases. Brain 2019;142:2558-71. [Crossref] [PubMed]
28. Alldred SK, Takwoingi Y, Guo B, et al. First trimester ultrasound tests alone or in combination with first trimester serum tests for Down’s syndrome screening. Cochrane Database Syst Rev 2017;3:CD012600 [Crossref] [PubMed]

(English Language Editor: R. Scott)