Cross-cultural adaptation, validity, and reliability of the Chinese version of the Nurses’ Perceptions of Disaster Core Competencies Scale (NPDCC)

Introduction

Public emergencies refer to natural disasters, accident disasters, public health events, and social security events that occur suddenly and cause or may cause serious social hazards and require emergency response measures. The government of China refers to all disasters as “public emergencies” and divides them into four categories: natural disasters, accident disasters, public health events, and social security events (1). Public emergencies have frequently occurred, including severe acute respiratory syndrome (SARS) in 2003, human avian influenza in 2005, influenza A (H1N1) in 2010, avian influenza A (H7N9) in 2013, Ebola hemorrhagic fever in 2014, the Middle East respiratory syndrome (MERS) in 2015, and COVID19 in 2019 have emerged in the recent years. With COVID19, according to the global data released on the official website of the World Health Organization on July 15, 13,150,645 people have been confirmed, and 574,464 people have died (2). These emergencies have seriously threatened the health and economic development of people all over the world. Nurses are the leading force in the rescue of public emergencies, and the core emergency response ability directly affects the quality of the entire medical rescue, of considerable significance to the protection of public health, social stability and economic development (3). WHO claimed that nurses provide care in emergency settings and will be key to the achievement of universal health coverage (4). But global disaster nursing education and research is lacking now (5,6). The survey results by Jolyon May indicate that nurses’ participation is obviously less than doctors’ in the course of disaster management and training, and the degree of attention to disaster response ability is not enough (7). Recently, Chinese scholars have studied the strategies of nurses to cope with public emergencies, but there are insufficient specific measurement tools for nurses’ core capacity of disaster preparedness. Applicable to the evaluation of nurses, Turkish scholar Celik (8) developed and validated the Nurses’ Perceptions of Disaster Core Competencies Scale (NPDCC) according to International Nursing Coalition for Mass Casualty Education (INCMCE) Disaster preparedness framework in 2010. The scale has 45 items and five subscales: Critical Thinking Skills, Special Diagnostic Skills, General Diagnostic Skills, Technical Skills and Communication Skills. This is a Likert-type scale where each item is scored from 1 point for ‘This needs to be taught’ to 5 points for ‘I can do and teach it’. Minimum and maximum scores vary between 45 and 225. High scores signify higher perceptions of disaster core competencies. However, its adaption into the Chinese language has not been done yet. This study aims to translate the English version of NPDCC into Chinese combining with Chinese nurses’ status and cultural background and to test its reliability and validity among nursing staff, and to provide a reliable and practical measurement tool for assessing the core competence of nurses’ public emergency in China. At the same time, it is hoped that the promotion and application of assessment tools can arouse the attention of the global health system on the education and training of emergency response ability of nurses. We present the following article in accordance with the SURGE reporting checklist (available at http://dx.doi.org/10.21037/apm-20-1454).

Methods

The required permission has been obtained from the original author of the English version scale via e-mail. The translation and cultural adaptation were conducted according to the procedure established by Sousa et al. (9). According to the factor analysis sample scale estimation method: the sample size is 5–10 times of the number of items (10). EFA and CFA samples were separated. So at least 900 samples need to be included. A total of 911 nurses were recruited and completed the questionnaire in the Yangtze River Delta Regional Lung Disease Nursing Alliance Hospital (35 hospitals, covering 5 provinces) from February to March 2020. These hospitals are all tertiary hospitals, which require nurses to master the ability of disaster care. This study was conducted according to the Declaration of Helsinki (as revised in 2013) principles and the ethics was approved by the institutional review board of Shanghai Pulmonary Hospital (ethical ID number: K19-146). Informed consent was obtained from all individual participants included in the study.

Subjects

The inclusion criteria are as follows:

• Obtained a nurse qualification certificate;
• A nurse working in a clinical department for at least one year;
• Informed and agreed to take part in this research.

The exclusion criteria are as follows:

• Practitioner nurses;
• Training nurses;
• Those who were out-of-post during the investigation;
• Logistics or auxiliary department nurses.

Instrument

Celik et al. (8) developed and confirmed the NPDCC for these disaster competencies and literature of INCMCE. The scale has 45 items and five factors: critical thinking skills, special diagnostic skills, general diagnostic skills, technical skills, and communication skills (Table S1). The scale is Likert-type, where each item is scored from 1 point for ‘this needs to be taught’ to 5 points for ‘I can do it and teach it.’ Minimum and maximum scores vary between 45 and 225. High scores signify higher perceptions of disaster core competencies. Celik et al. (8) reported that the alpha coefficients of the subscales to vary between 0.81 and 0.92 and is 0.96 for the total scale.

Table S1 factors and items of NPDCC
Full table

Translation and cultural adaptation

We obtained permission to translate the English version of the scale into Chinese from the authors using forward and back-translation with reconciliation by a panel of experts. Two English-speaking nursing professionals translated the original questionnaires into Chinese versions, and then the research team discussed and merged the two translations to form a Chinese NPDCC. Two postgraduate English majors who are unfamiliar with the original scale and have bilingual abilities translated the Chinese translation of NPDCC back to English NPDCC. Finally, the expert group compared the back-translated version of NPDCC with the original English scale. The sentences that do not conform to the Chinese cultural background and customs were discussed and changed by members of the research group. The scale is less influenced by cross culture, and the definitions involved are basically the same

Forty-five nurses from Shanghai Pulmonary Hospital were selected to conduct a pilot study to understand the respondents’ understanding of the content of the scale and the problems and suggestions during the filling. With the feedback from the survey respondents and the recommendations of the expert group, the Chinese version of NPDCC was appropriately formed in the definitive version. According to the pre-survey results, the scale was adjusted as follows: The item 23 of Original scale“ I can explain the psychological effects of the disaster ……” is adjusted as “I can assess and diagnose the psychological effects of the disaster ……”; The item 45 of Original scale“ I can explain the appropriate coping strategies to provide support for myself and others against negative effects of disasters” is adjusted as “I can explain how to adopt appropriate coping strategies to help myself and others to resist the negative effects of public health emergencies”.

Statistical analysis

SPSS software (version 21.0) and Amos software (24.0) were used for the statistical analysis. Descriptive statistics are used to summarize the nurses’ demographic characteristics. The analyses were expressed as mean ± standard deviation and percentages. For the Chinese version NPDCC, assessments of reliability and validity, as well as explanatory factor analysis (EFA), were conducted. Item analysis includes critical ratio and correlation analysis: critical ratio test should reach the significance level, indicating the item has sufficient discrimination, and items should be deleted for the Pearson correlation coefficient below 0.60 (11). The internal consistency and half-split reliability analysis were used for reliability analysis: Cronbach’s alpha coefficient measuring the average correlation among the items in the scale was considered excellent for above 0.70. The split-half reliability coefficients were considered excellent for above 0.60 (12,13). Validity analysis includes content validity and constructs validity analysis. EFA and confirmatory factor analysis (CFA) are utilized for constructing the validity analysis. The significance level was set at 5% (P<0.05).

Results

The NPDCC was designed as an electronic version through a software named “questionnaire star”. After obtaining the consent of the nurses, the questionnaire survey was conducted by scanning the QR (Quick Response) code of the web questionnaire. Nurses can draw a lottery of gifts to thank them for their cooperation in completing the study. Each questionnaire needs to be completed completely, and any missing items will be eliminated. A questionnaire was conducted among 911 nurses, and the response rate (number of valid questionnaires/total number of respondents) was 100%.

Demographic characteristics

Nurses’ mean age was 34.26 and meant the professional experience was 15.54, most of them were women (97.1%), married (78.27%), and worked as service nurses (92.35%). Other demographic information of the research objects was shown in Table 1.

Table 1 Demographic characteristics of nursing staff (N=911)
Full table

NPDCC scores of nurses

The NPDCC total score was 133.82 [standard deviation (SD): 32.96], and the critical thinking skills, special diagnostic skills, general diagnostic skills, technical skills, and communication skills scores are presented in Table 2.

Table 2 NPDCC scores for total and subscales (N=911)
Full table

Item analysis

Critical ratio (CR value) was used to evaluate the differentiation of Chinese NPDCC entries. The total NPDCC scores of 911 sample data were ranked in descending order. The first 27% with the highest score was in the high group, and the last 27% with the lowest score was in the low group. The results showed that the CR values of all items were greater than 3 and reached the significance level (all P<0.01), and the correlation coefficient between the items and the total score of the scale was 0.600–0.800 (all P<0.01), indicating that the items in the scale had high discrimination, so all items in the scale were retained (Table 3).

Table 3 Validity and reliability evidence
Full table

Reliability

The Cronbach’s alpha of the total NPDCC score was 0.978, and 0.884–0.945 for the subscales. The split-half for the scale was 0.930, and for the subscales were 0.861–0.894 (Table 3).

Validity

Content validity

There are no clear recommendations on the number of experts to include. Lynn suggested including a minimum of 3 experts, but more than 10 was not considered helpful (14). Therefore, nursing experts from general or specialist hospitals in Shanghai, Zhejiang, Jiangsu, and Anhui provinces are five members of this research group. They are professors in respiratory medicine, infectious diseases, intensive care unit (ICU), and nursing management. They rated the unambiguity, familiarity, and appropriateness of the items of the CBS-G on a 4-point Likert scale (very unambiguous/familiar/appropriate =4; not at all ambiguous/unfamiliar/inappropriate =1) (15). And the results showed that the content validity index (S-CVI) of the scale was 0.925, and the content validity index (I-CVI) of each item was 0.800–1.000.

Construct validity

EFA for the original five-factor model

CFA for the original five-factor model

We used SPSS AMOS (Version 24.0) to evaluate the research model with the data collected from nursing staff in China. The model produced acceptable fit indices, as shown in Table 4. The results of CFA demonstrated that all scales used in this study formed adequate measurement models, and thus, provided evidence for the construct validity of the measures (17).

Table 4 Model fit indices
Full table

Discussion

Project analysis is to explore the differences between high- and low-scoring research objects on each item or to evaluate for homogeneity between items. The results can be used as the basis for item selection or modification (18). In this study, the extreme group comparison method and the total score correlation method were used to evaluate each item to screen the Chinese version of the NPDCC scale. The top 27% of the scale items with the highest score are high grouping, and the bottom 27% of the scale items are low grouping. The t-test results show that there is statistical significance between the high and low groupings of each item of the scale (P<0.01). The correlation coefficients between the items and the total scores are all above 0.04, indicating that all items in the Chinese version of the NPDCC scale have high discrimination, that is, all items have a strong discriminative ability to nurses’ core competence in disasters and are retained.

In terms of content validity, if the content validity index of the scale entry is ≥0.78, and the content validity index of the scale is ≥0.90, it can be considered the evaluation tool has good content validity (19). This study evaluated the content validity of the Chinese version of the NPDCC scale by five nursing experts in related fields. After two rounds of expert consultation, the content validity index of the scale was 0.925, and the content validity index of each item (I-CVI) was over 0.800. The results of this study show that the NPDCC scale indicates that the Chinese version of NPDCC content is valid.

In terms of structural validity, the Chinese version of the NPDCC scale was verified through exploratory factor analysis and CFA, and the common factors were extracted using principal component analysis. A screen diagram of the factor structure was generated, with factor load values and commonness ≥0.40. It can be considered the entry that is in this factor (20). The results of this study show that the cumulative contribution rate of the five common factors of critical thinking skills, special diagnostic skills, general diagnostic skills, technical skills, and communication skills is 68.289%, and the load value of each item on the corresponding factor is 0.403–0.756. The test results of the original scale are similar and have sufficient structural validity.

Reliability is an effective evaluation of tool stability. The higher the reliability, the greater the credibility (21). The Cronbach’s alpha coefficient of each dimension of the scale is >0.70, and the Cronbach’s alpha coefficient of the total scale is >0.80, indicating that the reliability of the scale is excellent (12,13). The results of this study show that the Cronbach’s alpha coefficient of the Chinese version of the NPDCC scale is 0.978, and the Cronbach’s alpha coefficients of the subscales are 0.884 to 0.945; the half-coefficient of the scale is 0.930, and the half-reliability of each dimension is 0.861–0.894. The Chinese version of the NPDCC scale has high reliability and stability.

G Taskiran used NPDCC to investigate 406 nurses in Turkey. The results show ‘Technical Skills’ scored highest across the subscales of the scale, and ‘Critical Thinking Skills’ scored lowest (5). And Park used NPDCC to explore Factors influencing disaster nursing core competencies of emergency nurses working in 12 hospitals in South Korea (22). In this study, we complete Cross-cultural adaptation and test of NPDCC. However, the further application of the scale needs further research.

Study limitation: The sample source for reliability and validity verification is relatively single, and it needs to be applied in other areas of China in the follow-up study.

Conclusions

The NPDCC scale covers five aspects of nurses’ core competencies in disasters, which can comprehensively measure the core competencies of nurses in response to disasters. The Chinese version of NPDCC has excellent reliability and validity and meets the requirements of measurement. It shows it is suitable for measuring the cognitive level of the nurse’s core public emergency.

Acknowledgments

Funding: This work was supported by The Fund of basic scientific research business expenses of Tongji University (No. 22120200017), Shanghai, China.

Footnote

Reporting Checklist: The authors have completed the SURGE reporting checklist. Available at http://dx.doi.org/10.21037/apm-20-1454

Data Sharing Statement: Available at http://dx.doi.org/10.21037/apm-20-1454

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/apm-20-1454). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was conducted according to the Declaration of Helsinki (as revised in 2013) principles and the ethics was approved by the institutional review board of Shanghai Pulmonary Hospital (ethical ID number: K19-146). Informed consent was obtained from all individual participants included in the study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Li YH, Li SJ, Chen SH, et al. Disaster nursing experiences of Chinese nurses responding to the Sichuan Ya’an earthquake. Int Nurs Rev 2017;64:309-17. [Crossref] [PubMed]
2. World Health Organization. Coronavirus disease 2019 (COVID-19) Situation Report – 76. Available online: https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200715-covid-19-sitrep-177.pdf?sfvrsn=b1a193f3_2
3. Tomblin Murphy G, Mackenzie A, Alder R, et al. Pilot-testing an applied competency-based approach to health human resources planning. Health Policy Plan 2013;28:739-49. [Crossref] [PubMed]
4. World Health Organization. Nursing and midwifery. Available online: https://www.who.int/news-room/fact-sheets/detail/nursing-and-midwifery
5. Taskiran G, Baykal U. Nurses’ disaster preparedness and core competencies in Turkey: a descriptive correlational design. Int Nurs Rev 2019;66:165-75. [Crossref] [PubMed]
6. Halstead JA. When Disaster Strikes: Are You and Your nursing students prepared? Nurs Educ Perspect 2013;34:213. [PubMed]
7. May J, Colbert D, Rea S, et al. Preparedness and training in staff responding to a burns disaster. Br J Nurs 2015;24:918-23. [Crossref] [PubMed]
8. Celik F. Disaster Preparedness Status of Nurses Working at Turkish Red Crescent. Istanbul: Istanbul University Institute of Health Sciences. 2010.
9. Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: a clear and user-friendly guideline. J Eval Clin Pract 2011;17:268-74. [Crossref] [PubMed]
10. Hair JF, Black WC, Babin BJ, et al. Multivariate data analysis, 7th Edition Prentice Hall 2009:100.
11. Hazar Kanik Z, Gunaydin G, Pala OO, et al. Translation, cultural adaptation, reliability, and validity of the Turkish version of the Penn Shoulder Score. Disabil Rehabil 2018;40:1214-9. [Crossref] [PubMed]
12. Creswell JW. Educational research: Planning conducting and evaluating quantitative and qualitative approaches to research (2nd ed.). Upper Saddle River, NJ: Merrill/Pearson Education, 2005.
13. Fleiss JL. Reliability of measurement. In: Fleiss JL. editor. The Design and Analysis of Clinical Experiments. New York: John Wiley & Sons, 1986:1-32.
14. Lynn MR. Determination and quantification of content validity. Nurs Res 1986;35:382-5. [Crossref] [PubMed]
15. Holle D, Köller L, Moniz-Cook E, et al. Translation and Linguistic Validation of the German Challenging Behavior Scale for Formal Caregivers of People With Dementia in Nursing Homes. J Nurs Meas 2018;26:544-65. [Crossref] [PubMed]
16. Kaiser HF. A second generation little jiffy. Psychometrika 1970;35:e4015. [Crossref]
17. Arpaci I, Baloglu M. The impact of cultural collectivism on knowledge sharing among information technology majoring undergraduates. Computers in Human Behavior 2016;56:65-71. [Crossref]
18. He W, Bonner A, Anderson D. Translation and psychometric properties of the Chinese version of the Leeds Attitudes to Concordance II scale. BMC Medical Informatics and Decision Making 2015;15:60. [Crossref] [PubMed]
19. Taskiran G, Baykal U. Nurses’ disaster preparedness and core competencies in Turkey: a descriptive correlational design. Int Nurs Rev 2019;66:165-75. [Crossref] [PubMed]
20. Fredrickson BL, Branigan C. Positive emotions broaden the scope of attention and thought-action repertoires. Cognition & Emotion 2005;19:313-32. [Crossref] [PubMed]
21. Lynn MR. Determination and quantification of content validity. Nurs Res 1986;35:382-5. [Crossref] [PubMed]
22. Park HY, Kim JS. Factors influencing disaster nursing core competencies of emergency nurses. Appl Nurs Res 2017;37:1-5. [Crossref] [PubMed]

(English Language Editor: J. Chapnick)