Development and assessment of a mental health preliminary screening questionnaire for cancer patients

Introduction

With rapid economic development, increased environmental pollution and lifestyle changes, malignant tumors have become a serious threat to human health (1). In recent years, the incidence of cancer has increased considerably, making it the second deadliest pathology after cardiovascular disease (2). Due to the particular features of cancer, affected patients face great pressure and are prone to complex psychological changes that may manifest as anxiety, fear, and depression (3). These psychological changes not only cause mental stress to patients, but also affect treatment effects and quality of life. Thus, the evaluation of cancer patients’ mental health is attracting increasing attention. In mental health assessments, inferential and quantitative analyses of psychological characteristics of representative human behaviors are conducted to facilitate difference comparisons and observations, and determine patients’ mental states (4-6).

Currently, preliminary screening methods for mental health evaluation include the Self-Rating Depression Scale (SDS), Self-Rating Anxiety Scale (SAS), Symptom Checklist (SCL-90), Hamilton Depression Scale (HAMD) and Beck Depression Inventory (BDI-II) (7-10). As no specific questionnaires have been designed for the mental health screening of cancer patients, despite the particularity of malignancies, these scales are applied to all patients. These self-rating questionnaires comprise 10 to 90 items each, and thus take a long time to complete, which reduces patient compliance and response accuracy. Thus, an easy-to-operate, time efficient and effective mental health preliminary screening questionnaire (MHPSQ) is urgently required for cancer patients.

This study sought to develop a convenient and reliable screening tool that could be used by medical staff to quickly assess the mental status of cancer patients. We developed the MHPSQ, and evaluated the specificity and sensitivity of the new scale to screen depression and anxiety in cancer patients. We present the following article in accordance with the MDAR reporting checklist (available at https://dx.doi.org/10.21037/apm-21-1252).

Methods

Ethical considerations

Patients at Fujian Cancer Hospital between May 2018 and June 2019 were enrolled in this study. To be eligible to participate in the study, patients had to meet the following inclusion criteria: (I) have been hospitalized for the first time for cancer; (II) be able to communicate efficiently and fluently in Mandarin; and (III) be willing to cooperate. Patients with a history of mental disorder were excluded. All procedures performed in this study involving human participants were in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the ethics committee of Fujian Cancer Hospital (No. KT2018-011-01) and informed consent was taken from all the patients.

The study was performed in three stages (see Figure 1).

Figure 1 Study flow chart. SDS, Self-Rating Depression Scale; SAS, Self-Rating Anxiety Scale.

Stage 1: development of questionnaire

In May 2018, 8 patients were interviewed to gather information on the topic. Individual interviews were conducted to determine the psychological characteristics of newly admitted cancer patients. The data were collected in face-to-face, semi-structured in-depth interviews. Qualitative interviews and a literature search (11-18) were used to form a questionnaire item pool. Following 2 rounds of expert consultations, the first draft of the questionnaire was developed.

From July to August 2018, cancer patients hospitalized for the first time were selected to participate in a project analysis and inspection. Then, the final draft of the MHPSQ was developed.

Stage 2: reliability and validity assessment of the new questionnaire

From September to December 2018, cancer patients hospitalized for the first time were selected by purposive sampling for the reliability and validity assessments of the MHPSQ.

Stage 3: comparison of the completion times of the SDS, SAS, and MHPSQ, assessments to determine the diagnostic value of the MHPSQ

Cancer patients hospitalized for the first time from January to April 2019 were selected using purposive sampling for the field investigation. All the study participants were evaluated based on the MHPSQ, SAS, and SDS (in this order) in a quiet environment. The times required to complete the three scales were compared.

Patients were diagnosed with anxiety or depression based on the gold standard; that is, the Chinese Classification and Diagnostic Criteria for Mental Disorders (CCMD-3) (19). Positivity in the MHPSQ was defined as evaluation results revealing mental health problems, while negativity was defined as an absence of mental health problems. The specificity and sensitivity of the MHPSQ were assessed against the gold standard.

Instruments

The SAS was developed by the Chinese professor Zung in 1971 (20). The SAS is a simple clinical tool for analyzing a patient’s subjective symptoms. The SDS was developed by the Chinese professor Zung in 1965 (21), and comprises 20 items reflecting subjective feelings of depression.

Statistical analysis

SPSS17.0 (SPSS, USA) was used for the data analysis. The demographic data are expressed as percentages or means ± standard deviations, and a 2-sided P<0.05 indicated statistical significance. Internal consistency was assessed by determining Cronbach’s α. When the deletion of an item resulted in a Cronbach’s α greater than that of the overall questionnaire, the indicated item was deleted (22). For the construct validity assessment, Pearson correlation coefficients were determined to test associations between the questionnaire items and the total scores (22). In the exploratory factor analysis, items with factor loadings ≥0.5 were considered acceptable (22). In the criterion validity analysis, Pearson correlation coefficients were determined to test the associations of the questionnaire with the SAS and SDS. For the reliability evaluation, Cronbach’s α (22) and test-retest reliability (22) were adopted. The completion times of the SAS, SDS, and MHPSQ groups were assessed using t-tests. The sensitivity and specificity of the new scale were analyzed using a 4-grid table.

Results

During this study, 150, 400, and 1,000 questionnaires were collected, respectively. The return rate was 100%.

Stage 1

8 patients (3 males and 5 females) took part in the semi-structured interviews. These patients had an average age of 51.60±9.00. Of these patients, 2, 3, and 3 patients had lung, gastrointestinal and breast cancer, respectively. 6 items were preliminarily selected for the item pool.

Next 2 rounds of expert consultations were carried out; the questionnaire return rates in both rounds were 100%. The coefficients of authority were 0.84 and 0.87 for the first and second rounds, respectively. The consistency of the experts’ opinion was assessed using Cronbach’s α values, and scores of 0.31 and 0.52, were obtained for the first and second rounds, respectively; the results were statistically significant (P<0.05). Finally, a first draft of the MHPSQ was developed, comprising 5 items in total, including items related to sleep, emotion, interactions with others, social functioning (work and study), and symptoms in daily life.

One hundred and fifty cancer patients were involved in the development of the final questionnaire. These patients aged 30–78 years old (average: 54.60±10.00 years). The final draft of the MHPSQ, comprising 4 items in total, was developed (see Table 1).

Table 1 Final version of the MHPSQ
Full table

Internal consistency

The Cronbach’s α for the whole questionnaire was 0.67, while that of the “symptoms and daily life” item was 0.68. Thus, the “symptoms and daily life” item was deleted.

Test validity

A principal component analysis was adopted in the exploratory factor analyses of the 5 items of the MHPSQ. After extracting 2 items, factor 1 was found to have close associations with “emotion,” “interpersonal relationships,” “social function,” and “sleep.” Thus, the item “physical discomfort” was deleted, and a MHPSQ that comprised 4 items was established as the final version (see Table 1).

Application of the new questionnaire

MHPSQ was evaluated using hospital HIS system. According to the Likert-scale scoring rule, a measuring scale is generally based on a 5- or 4-point scoring principle. The total possible score for the MHPSQ is 20. A score of 16 to 20 indicates a great psychological condition. A score of 13 to 15 indicates a good psychological condition. A score of 9 to 12 indicates a psychological condition that requires psychological support. If a patient scored 9 to 12, SAS and SDS evaluations were then conducted simultaneously. The results of which were reported to the mental liaison officer of the hospital, and a mental counselor was consulted as necessary. A score of 8 points or less indicates a poor psychological status. If a patient scored less than 8, SAS and SDS evaluations were then conducted simultaneously, and a risk assessment for suicide prevention was also conducted. The results were reported to the chief physician, the head nurse of the department, the mental counselor of the hospital, and a psychiatric department outside the hospital was approached for further consultation if necessary. Under the MHPSQ, scores ≤12 indicate mental health problems and are considered positive. Further, the SDS and SAS were available for evaluations, and corresponding psychological intervention measures were applied in cases of confirmed psychological problems.

Stage 2

Among the 400 cancer patients who participated in the reliability and validity assessments of the new questionnaire, 198 (49.5%) were male and 202 (50.5%) were female. The patients were aged 54.20±13.40 on average. Data about patients’ general characteristics are set out in Table 2.

Table 2 Characteristics of test participants
Full table

Test validity

In relation to content validity, the coefficients between factor pairs were 0.45–0.64, and the coefficient between each factor and the overall score was 0.78–0.85. No item was deleted. In relation to criterion validity, the results showed that the questionnaire had significant negative correlations with the SAS and SDS; the Pearson correlation coefficients were between −0.45 and −0.71 (P<0.05; see Table 3).

Table 3 Calibration correlation validity of the final MHPSQ based on the SAS and SDS
Full table

Reliability of the MHPSQ

The overall Cronbach’s α obtained for the questionnaire was 0.76, and the test-retest reliability was 0.92. Specifically, the test-retest reliability coefficients for “emotion,” “interpersonal relationship,” “social function” and “sleep” were 0.77, 0.84, 0.90, and 0.86, respectively.

Stage 3

A total of 1,000 patients (482 males and 518 females) were included in the comparison of the 3 scales. The average age of these patients was 46.00±13.40 years (see Table 2).

Comparing completion times

The results of the SAS, SDS, and MHPSQ in terms of completion time are set out in Table 4. Notably, the screening and completion times for the MHPSQ were far shorter than those of the SAS and SDS.

Table 4 Evaluation times for the MHSQ, SAS, and SDS
Full table

Diagnostic value of the MHPSQ

One hundred and forty-seven (14.7%) patients scored ≤12 based on the MHPSQ. Compared with the CCMD-3, the sensitivity and specificity were 92.4% and 98.5%, respectively.

Discussion

This study developed a new questionnaire for mental health screening among cancer patients. The questionnaire developed was found to have good reliability and validity. It comprises only 4 items, is easy to operate, can effectively identify patients with psychological abnormalities, and thus could improve patient care.

Clinical screenings based on the MHPSQ were highly consistent with final diagnoses, and the MHPSQ had high specificity and sensitivity. Additionally, the MHPSQ was significantly superior to the SAS and SDS in terms of completion time.

The proposed MHPSQ not only relied on the SAS, SDS, HAMD for the development of its theoretical basis, but also on the opinions of psychologists, clinical nurse specialists, and research groups. Thus, the quality of the MHPSQ is guaranteed.

The MHPSQ has reliable psychometric characteristics. First, the questionnaire has good validity. As discussed above, the associations among the items of the MHPSQ and the whole questionnaire were statistically significant (P<0.01). Additionally, this study used the SAS and SDS as standards to assess the criterion validity of the MHPSQ. The overall score of the MHPSQ was negatively correlated with the standard scores of the SAS and SDS, indicating that the MHPSQ has good criterion validity.

Next, the reliability, stability, and consistency of the questionnaire were examined. Cronbach’s α is the coefficient of internal consistency, which indicates the intrinsic relevance of multiple items that are used to evaluate a certain topic. The greater the value of the α, the better the correlations among items. In general, the internal consistency coefficient of Cronbach’s α for a questionnaire is acceptable if it is above 0.7 (23). The Cronbach’s α for the questionnaire in this study was 0.77. Further, the test-retest reliability of the scores obtained from the questionnaire at the first investigation and 2 weeks later, as well as the test-retest reliabilities of 4 items (“emotion,” “interpersonal relationship,” “social function,” and “sleep”) were all between 0.70 and 0.90. Thus, the newly developed MHPSQ has good reliability, stability, and consistency.

Finally, the clinical application of the questionnaire was preliminarily evaluated, and a relatively high positive rate was obtained, which was similar to that of the CCMD-3. Thus, the MHPSQ can be reliably employed to assess cancer patients’ mental status.

As discussed above, the MHPSQ takes less time to complete than the SAS and SDS, but achieves similarly accurate screening results. This is a great advantage, as it could change the nursing workflow. Patients could be quickly screened at the time of hospital admission, and those at high-risk of mental health problems (anxiety and/or depression) could be identified in a timely and effective manner. Additionally, this questionnaire could achieve a dynamic psychological assessment of cancer patients during the 4 periods of hospitalization (i.e., at admission, before treatment, upon condition changes, and before discharge). This could reveal the psychological changes of inpatients throughout the treatment in a timely manner.

By reducing the screening time, the MHPSQ would also reduce the clinical nursing time. In the preliminary clinical application, the MHPSQ only too approximately 84 seconds to complete, which is much shorter than the time it takes to complete other questionnaires. Thus, the MHPSQ could not only reduce the workload of clinical nurses and improve their work efficiency, but could also detect psychological problems in cancer patients and improve patient compliance and care in a timely manner.

At present, Mental health screening and instruments have been widely used in many groups. This paper summarizes different scales to provide references for cancer rehabilitation and promotion of cancer patients’ mental health. The scales are divided into four categories: emotional distress, anxiety, depression, and post-traumatic stress disorder. The distress thermometer and the general health questionnaire are widely used in the screening for emotional distress problems. The Hospital Anxiety and Depression Scale Anxiety Scale and the Baker Anxiety Scale are good tools for screening anxiety problems (24). questionnaires such as the SAS, SDS, and SCL-90 are commonly used in clinics to assess anxiety and depression in cancer patients throughout treatment (25). The SAS and SDS are clinical tools for analyzing individual subjective symptoms; however, they are self-rating scales, which are susceptible to factors such as the patient’s educational background, degree of cooperation, and understanding of questionnaire instructions. Thus, it is not easy for busy clinical nursing staff to quickly perform screenings and evaluations of many patients. Additionally, patients with low education levels can find it difficult to understand and answer the questions on these scales. The MHPSQ developed in this study has fewer items, and is easy to operate, time-saving and labor-saving. The new questionnaire is characterized by the combination of objective and subjective parameters, quantitative qualitative measures, and analytical and synthetic factors. Consequently, psychological interventions could be performed as early as possible to help patients improve their psychological status.

The concept of patient-centered nursing requires us to pay attention to the patient’s disease diagnosis and treatment, but also pay attention to the patient's illness process and feelings, especially the anxiety and other psychological states of the disease. Effective preliminary screening of cancer patients’ mental health can better implement patient-centered nursing.

This study confirmed that the MHPSQ has good reliability and validity; however, some limitations should be noted. First, the sample size was relatively limited. In addition, this study only assessed patients at a tertiary hospital in a city. Thus, similar studies should be carried out in different areas of mainland China. Further, this study employed purposive sampling; however, as the backgrounds of the different populations varied, the representativeness of the target population is deficient. To enable a wide clinical application of the MHPSQ, future studies should expand the scope of the samples and reduce differences among samples to confirm our findings. Moreover, the questionnaire in this study is only used for rapid preliminary screening of the mental health of cancer patients. SAS and SDS are still needed for self-assessment when patients with mental health problems are screened.

The current study developed a new MHPSQ with good reliability and validity that is easy to operate, time-saving, labor-saving, and effective. It can be conveniently used to quickly evaluate the psychological state of cancer patients, and thus has the ability to identify psychological problems in a timely and effective manner.

Acknowledgments

Funding: None.

Footnote

Reporting Checklist: The authors have completed the MDAR reporting checklist. Available at https://dx.doi.org/10.21037/apm-21-1252

Data Sharing Statement: Available at https://dx.doi.org/10.21037/apm-21-1252

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://dx.doi.org/10.21037/apm-21-1252). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All procedures performed in this study involving human participants were in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the ethics committee of Fujian Cancer Hospital (No. KT2018-011-01) and informed consent was taken from all the patients.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Mannucci PM, Franchini M. Health Effects of Ambient Air Pollution in Developing Countries. Int J Environ Res Public Health 2017;14:1048. [Crossref] [PubMed]
2. You W, Henneberg M. Cancer incidence increasing globally: The role of relaxed natural selection. Evol Appl 2018;11:140-52. [Crossref] [PubMed]
3. Bai LJ, Liu Q, Wang M, et al. Evaluation of the psychological and biological changes of patients diagnosed with benign and malignant breast tumors. Int J Biol Markers 2012;27:e322-30. [Crossref] [PubMed]
4. Zhao H, Zhao Z, Chen C. Prevalence, risk factors and prognostic value of anxiety and depression in cervical cancer patients underwent surgery. Transl Cancer Res 2020;9:65-74. [Crossref]
5. Yen JY, Ko CH, Yen CF, et al. Quality of life, depression, and stress in breast cancer women outpatients receiving active therapy in Taiwan. Psychiatry Clin Neurosci 2006;60:147-53. [Crossref] [PubMed]
6. Henoch I, Bergman B, Gustafsson M, et al. The impact of symptoms, coping capacity, and social support on quality of life experience over time in patients with lung cancer. J Pain Symptom Manage 2007;34:370-9. [Crossref] [PubMed]
7. Feng Y, Wang XY, Li SD, et al. Clinical research of acupuncture on malignant tumor patients for improving depression and sleep quality. J Tradit Chin Med 2011;31:199-202. [Crossref] [PubMed]
8. Zhang L, Huang Z, Wu H, et al. Effect of swallowing training on dysphagia and depression in postoperative tongue cancer patients. Eur J Oncol Nurs 2014;18:626-9. [Crossref] [PubMed]
9. Xiao N, Zhu D, Xiao S. Effects of continued psychological care toward brain tumor patients and their family members' negative emotions. J Cancer Res Ther 2018;14:S202-7. [Crossref] [PubMed]
10. Han J, Nian H, Zheng ZY, et al. Effects of health education intervention on negative emotion and quality of life of patients with laryngeal cancer after postoperative radiotherapy. Cancer Radiother 2018;22:1-8. [Crossref] [PubMed]
11. Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry 1960;23:56-62. [Crossref] [PubMed]
12. Friborg O, Hjemdal O, Rosenvinge JH, et al. A new rating scale for adult resilience: what are the central protective resources behind healthy adjustment? Int J Methods Psychiatr Res 2003;12:65-76. [Crossref] [PubMed]
13. Jansen CE, Miaskowski C, Dodd M, et al. Chemotherapy-induced cognitive impairment in women with breast cancer: a critique of the literature. Oncol Nurs Forum 2005;32:329-42. [Crossref] [PubMed]
14. Duprez C, Vanlemmens L, Untas A, et al. Emotional distress and subjective impact of the disease in young women with breast cancer and their spouses. Future Oncol 2017;13:2667-80. [Crossref] [PubMed]
15. Christophe V, Duprez C, Congard A, et al. Evaluate the subjective experience of the disease and its treatment in the partners of young women with non-metastatic breast cancer. Eur J Cancer Care (Engl) 2016;25:734-43. [Crossref] [PubMed]
16. Perz J, Ussher J, Gilbert E. Loss, uncertainty, or acceptance: subjective experience of changes to fertility after breast cancer. Eur J Cancer Care (Engl) 2014;23:514-22. [Crossref] [PubMed]
17. Ohnishi N, Kataoka T, Okamura H. Relationships between roles and mental states and role functional QOL in breast cancer outpatients. Jpn J Clin Oncol 2011;41:1112-8. [Crossref] [PubMed]
18. Kroenke K, Spitzer RL, Williams JB, et al. The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: a systematic review. Gen Hosp Psychiatry 2010;32:345-59. [Crossref] [PubMed]
19. Chinese Society of Psychiatry. Chin J Neurol 2001;34:184-8.
20. Zung WW. A rating instrument for anxiety disorders. Psychosomatics 1971;12:371-9. [Crossref] [PubMed]
21. Zung WW. A self-rating depression scale. Arch Gen Psychiatry 1965;12:63-70. [Crossref] [PubMed]
22. Souza AC, Alexandre NMC, Guirardello EB. Psychometric properties in instruments evaluation of reliability and validity. Epidemiol Serv Saude 2017;26:649-59. [Crossref] [PubMed]
23. Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007;60:34-42. [Crossref] [PubMed]
24. Chen WL, Yan R, Wang JY, et al. Progress in researches on mental health screening instruments for cancer patients. Chinese Journal of Public Health 2020;36:1389-92.
25. Li L, Wang BQ, Gao TH, et al. Assessment of psychological status of inpatients with head and neck cancer before surgery. Zhonghua Er Bi Yan Hou Tou Jing Wai Ke Za Zhi 2018;53:21-6. [PubMed]

(English Language Editor: L. Huleatt)