PROutine: a feasibility study assessing surveillance of electronic patient reported outcomes and adherence via smartphone app in advanced cancer

Introduction

Quality of life (QoL): a major goal of care

Cancer is frequently associated with pain and other conditions that impair the patients’ QoL and result in significant suffering of the affected patients (1,2). This is relevant for patients in curative and palliative stages of the disease (1,2). QoL and the reduction or prevention of suffering is an important goal of care for all cancer patients in curative and palliative situations (2). It is known that already at diagnosis, many patients with incurable cancer suffer from burdensome symptoms (3). All too often, this suffering remains unidentified and thus untreated despite readily available therapeutic options (i.e., cancer pain management) (1,2).

Importance of patient-reported-outcomes (PROs)

A major reason for such unnecessary suffering is that burdening symptoms or other sources of distress can ideally be assessed via PROs, but despite their validated usefulness such PROs are not routinely documented in clinical routine (4). For example, te Boveldt et al. (5) found that pain in medical oncology outpatients is not systematically registered in their medical records. Basch et al. (6) reported findings from a recently performed randomized, controlled trial (RCT) that showed that if PROs were obtained and recorded routinely, QoL and other clinical outcomes of cancer patients improved substantially compared to those who received standard care. This could be due to both patient and clinician focusing more on actual symptoms, distress and in their development (worsening/improving). Clinicians react on basis of this information with adjusting symptom control management and as a result of that, QoL and other clinical outcomes improved. The same applies to research in the field of anti-cancer therapy (7). Here, too often clinical trials abstain from assessing, reporting and discussing PROs (7). This substantially impairs the appraisal of the study findings, because such information is essential to judge outcomes that are of utmost importance for the patients (8). Furthermore, PROs play a crucial role in the process of evaluating the benefit of new anti-cancer-therapies: the European Society for Medical Oncology (ESMO) as well as the American Society of Clinical Oncology (ASCO) have implemented PROs in their framework for evaluation of new therapies.

The potential role of mobile Health (mHealth)

Large studies show that paper- and web-based surveys provide data that are essentially equivalent (9). It is known that the documentation of PROs via conventional paper-pencil questionnaires is associated with significant shortcomings (10). For example, such paper-pencil solutions are prone to recall bias, and their use in clinical trials is associated with a large amount of missing data and transcription errors (10). In addition to that paper-pencil versions require printed questionnaires not being ecofriendly and being costly. The advantages of electronic PRO (ePRO) are that symptom data could automatically be scored and is available in easily interpretable reports to be viewed when the clinician meets the patient, data entry by staff is not required, data can be automatically transferred in real time to a computer server, which can lead to many different clinically relevant actions based on PRO scores, results can be added to the patient’s electronical medical record, score alerts can be generated to notify clinical staff of acute patient needs and patients can be immediately provided electronically with educational material or advices (11). Furthermore, a clinician can easily review a patient’s symptom data over time. Meanwhile, it is known that electronic versions of validated paper-pencil versions of simple narrative questionnaires that do not rely on the use of complex visual material (i.e., Rorschach test) do not need to be further tested for validity. In addition to overcoming the methodological problems mentioned above (recall bias etc.), mHealth solutions, supported by mobile devices facilitate the continuous surveillance of the patients’ well-being in the home environment without relying on costly and difficult-to-schedule visits in the out-patient clinic (or study center) or follow-up phone calls. This, for example, has been reported for the use of e-mail- or browser-based solutions (11). Because of the potential correlation of PROs to the adherence to the medication plan [i.e., pain intensity/breakthrough pain episodes (BPE) correlated to the correct use of opioids on demand], the real-time recording and visualization of, for example, pain intensity, pain episodes and (under-) use of on-demand medication, a mobile version like a smartphone app to collect these data may improve adherence and facilitate patient guidance (12). The same applies to the aspects of pharmacovigilance (i.e., side effects, or incorrect use) (13). Comprehensive and role-specific training for using ePRO systems are necessary (14).

Aim of the study

The aim of this study was to test whether the routine assessment of PROs via a newly developed smartphone application (MeQoL^®) is feasible for outpatients with advanced cancer.

Methods

After registration of the study [German Clinical Trials Register (DRKS), ID: DRKS00008761] and approval by the local ethic committees, an open label prospective multicenter study was performed. The ethics committees of University Medical Center Göttingen, Klinikum rechts der Isar of the Technical University Munich and of the University Medical Center Berlin gave positive statements (ID: Göttingen, 13/11/14; Munich, 190/50S; Berlin, agreement to statement of Göttingen). Adult (>18 years) outpatients with advanced, solid cancer were eligible if they were routinely scheduled to receive at least monthly appointments in the outpatient clinic of the three participating cancer centers. The ability to use a smartphone app or the self-confidence and ability to learn it were preconditions to take part. In each center ten patients ought to be included. The key exclusion criteria were diagnosis of severe psychiatric comorbidities, psychomotor impairments, or inability to speak and write German. Also, patients near the end of life were not approached because of their vulnerability, their focus on other existential things other than taking part in a study and because of assumed drop out before end of study. “End of life” was defined via the “surprise question” for three months (“Would you be surprised if your patient died within the next three months?”) (15). Sampling was performed on several randomly chosen days by asking the oncologist responsible for outpatient clinic which patients of the present day would meet inclusion criteria.

After providing informed consent, patients were instructed how to use the smartphone application. Instructions were given by a physician in a face to face conversation demonstrating the use and the content of the app going through the program in a live demonstration. There were no process level changes. Patients who agreed to take part in the study were familiar with computer literacy including smartphone use. The time needed for this instruction was recorded. Additionally, each patient received a handbook explaining the usage of the device and the app.

Daily, patients recorded the degree of perceived distress (measure: NCCN Distress Thermometer^®) via the smartphone app MeQoL^® (Figure 1). Additionally, pain intensity (average and worst), the number of BPE and ten questions from a shortened Minimal Documentation System (MIDOS) questionnaire, which is a simplified version of the Edmonton Symptom Assessment Scale (ESAS), were asked to be documented. Additionally, patients recorded five questions [from the Short-form 8 questionnaire (SF-8)] concerning different domains of QoL (reduction of activities, pain intensity, reduction of social contact, distress by emotional problems, restrictions in everyday life) weekly. For these recordings, the patients were reminded via push-up notifications. Patients were allowed to record PROs on more than these predefined occasions. Monitoring of symptoms was not followed by clinical actions because feasibility of using the app itself by the patient was in focus of this study. Analyzing doctor-patient interaction with the help of this app and effect on medication adjustment by the physician and symptom relief of the patient will be next steps in a new study. According to the main scope of the trail (feasibility), no clinical outcome was defined as a primary endpoint. Rather, the following main outcome measures were assessed descriptively: missing data, drop-out- and acceptance-rate, patient satisfaction and patient judgement of practicability, patients’ and physicians’ suggestions for improvement, demographic and clinical data of the patients. Testing for statistical significance of association between different parameters was not attempted, in order to avoid multiple testing errors in the absence of an obvious a priori hypothesis. Rather, the results were evaluated descriptively.

Figure 1 PROutine study design, screenshots and starter pack of MeQoL^®. QoL, quality of life; SF-8, short-form 8; MIDOS, Minimal Documentation System.

Results

In the period from March 2015 to December 2015, 40 patients from three German cancer centers were included in the study. One institution had previously canceled participation in the study due to organizational reasons. Because of pragmatically considerations, no additional center was included in order to carry on in time. One center doubled therefore the number of patients included so the number of patients was 40. From 40 devices that were sent by mail, three devices were not delivered and thus could not be evaluated since the data was stored only in the device. The data from all received devices could be analyzed. Missing feedback sheets were observed and registered (Table 1).

Table 1 Sample and evaluable material
Full table

Forty of the requested 56 patients (71%) participated in the study. Thirty-seven devices (92.5%) could be evaluated. Eighty-five percent (34/40) of the record sheets and 75% (30/40) of the feedback forms were available and could be evaluated (Table 1). The reported duration for the training of the patients in the handling of the device was 20 to 30 minutes. The reasons for refusing participation in the study were mostly personal in nature or justified with an uncertainty regarding the device and the technology. Reasons for refusal of participation were not correlated with age or sex of the patient. The majority of patients was between 50 and 70 years old [average, 57 years (range, 27–73 years; SD, 12)] and was female (28/40), four patients were older than 70 years (Figure 2).

Figure 2 Age of the participating patients (n=40). Ø, mean (age).

The median overall investigation period per device was 99.5 days [defined as days between the first and last active recording in the MeQoL^® (SD, 31; range, 16–175 days)]. In the median daily reviews (defined as a day on which an active use of a MeQoL^® component was recorded) were performed on 70 days (70%) (range, 9–119 days; SD, 29). On 4% of the days the app was used more than once (Table 2). Median weekly recordings were performed on 13 weeks (87%) (Table 3). Also, the patients frequently used the possibility of weekly recordings (Table 3). Both daily and weekly recordings seem to be appropriate. Daily recording makes sense to be performed in the evening.

Table 2 Evaluation of the average app usage for the daily recording compared to the total investigation period, treatment centers as mean value
Full table

Table 3 Evaluation of the average app usage for the weekly recording, compared to the total investigation period (weeks, treatment centers as mean value)
Full table

Evaluation and feedback

Feedback was given by 30 patients but not everybody answered every question. The majority of patients, 26 of 29 (86%), rated the usability as good to very good, and 27 of 30 patients (90%) would use the mobile device based app again. The personal benefit from monitoring was rated good and very good by 16 (53%) patients (Table 4).

Table 4 Feedback of patients
Full table

Five physicians gave feedback. They rated usability as good to very good, all would want to use the mobile device based app again, and everybody saw a personal benefit by using it. They appreciated the most the fact that such an app is available and graphic data analysis.

To improve the app, it was suggested to enlarge writing, a possibility of type in text, to add questions concerning weight, health/feeling good, sexuality and collateral symptoms as mucositis. Patients recommended adding an acoustic signal to be reminded to use the app.

Exemplary extract of the data evaluation of a device

The recorded values (QoL, pain episodes, pain rescue medication, MIDOS questions, SF-8) can be reproduced and displayed in any combination. This allowed a direct comparison of individual values (two or more). QoL, pain intensity and pain episodes as well as taking rescue medication for pain were recorded and were visualized in graphs (Figure 3). It shows that rescue medication for pain was taken when pain intensity and pain events increased and QoL had declined. With the use of the rescue medication for pain QoL improved, and pain intensity decreased. (Figure 3).

Figure 3 Individual example for pain, distress and adherence (patient NO. 22, day 54–67). The figure shows levels of pain intensity and distress, number of pain episodes/day and adherence to pain rescue medications are shown for this patient during 2 weeks. NRS, numerical rating scale; n, number of pain episodes/day.

Discussion

Summary and interpretation of findings

Increasing use of smartphones and apps in the general population suggests the collection of individual health data, in order to optimize treatment in pain patients, for example. In this feasibility trial, adherence for the smartphone application was high, while missing data and dropout rate were low, even though a quite frequent (daily) recording was demanded (Tables 2,3). These findings met our a priori definition (less than 30% refusal) of feasibility. Comparison of the rates of missing data and drop out using conventional paper-pencil versions of PRO questionnaires (10) with the results of this study shows that the smartphone version is at least comparable or even superior to conventional PRO assessment. In addition, it has already been demonstrated that in contrast to electronic recording, paper-pencil questionnaires are prone to recall bias for example because of lost questionnaire papers and illegible writing (10). Furthermore, recall bias could be due to patients who did not remember their symptoms well enough in the time of recording them. In their feedback, patients mentioned that it would be helpful to have f.e. acoustic reminders to fill out the questionnaire. Patients’ satisfaction with MeQoL^® was high and many participants recorded symptoms and distress more often than in the predefined intervals (Tables 2,3). Also, individual assessment of visualized correlation of pain intensity, pain episodes and intake of on-demand medication (Figure 3) can be used to identify shortcomings in the patients’ adherence to treatment and to facilitate communication about these issues.

A simple and inexpensive iPod based mHealth solution (MeQoL^®) enables the patient to routinely document the current QoL corresponding to intensity of symptoms and intake of rescue medication.

Cloud-based solutions could have reduced the number of missing data related to lost devices and data which is accessible from everywhere any time by authorized persons. Furthermore, it would have offered a brought range of possibilities for evaluation and interaction.

Improvement of MeQoL^®-app and implications for further research

In order to improve user friendliness for patients, reminding functions for data input could be adjusted. For doctors, “red flags”, e.g., for high symptom scores and frequently used rescue medication, might be helpful to react quickly to adjust medication or to arrange diagnostic procedures. A future goal is to further individualize apps adapted to the needs of patients and care givers. Accessibility of data for physicians independent of interface meetings with patients could be simplified by clouds.

Further research could be performed concerning how patients and physicians or other care givers could ideally be motivated and trained in using such apps for collecting health information and how to interpret the data.

Limitations of the study

Limitations of this study were especially the open label design and low number of patients. Furthermore, only German speaking people could take part because no other language options for the app were provided. The individual benefit of data documentation could not be evaluated because data was not routinely discussed with the doctor and therefore had no impact on therapeutically decisions.

Conclusions

Using digital documentation of PROs via smartphone app is characterized by a high patient satisfaction and judgment of practicability. Routine screening for symptom burden and QoL issues demonstrates significantly improved clinical outcomes, and the insufficient assessment of PROs in cancer research massively impairs the interpretability of study results. For the individual patient, MeQoL^® may also allow for monitoring adherence to pharmacotherapy and can facilitate patient guidance.

Acknowledgements

We thank every team member in Berlin, Göttingen and Munich who has made it possible that this study could be performed, either by active contribution to the study itself or by allowing investigators to have time resources for working on the study. We thank Gerhild Becker for proofreading the final manuscript.

Funding: The conduction of this study was promoted by a research grant from TEVA Specialties.

Footnote

Conflicts of Interest: B Alt-Epping received honoraria from TEVA, Kyowa Kirin, Roche, BMS, Amgen, and Novartis; T Bauknecht was employed as senior medical scientific liaison by Teva Germany; G Benze declares no conflicts of interest; J Ettl received honoraria from TEVA, Pfizer, and Roche Pharma; J Gaertner received research grants and honoraria from TEVA, Kyowa Kirin, Mundipharma, Pfizer, Sanofi-Aventis; G Gianni is employee of AppPeople; L Kretzschmar declares no conflicts of interest; A Munte declares no conflicts of interest; F Nauck received honoraria from Roche Pharma, Mundipharma, Archimedes and TEVA.

Ethical Statement: The ethics committees of University Medical Center Göttingen, Klinikum rechts der Isar of the Technical University Munich and of the University Medical Center Berlin gave positive statements (ID: Göttingen, 13/11/14; Munich, 190/50S; Berlin, agreement to statement of Göttingen). All patients have provided written informed content.

References

1. Kurita GP, Sjøgren P. Pain management in cancer survivorship. Acta Oncol 2015;54:629-34. [Crossref] [PubMed]
2. Kelley AS, Morrison RS. Palliative Care for the Seriously Ill. N Engl J Med 2015;373:747-55. [Crossref] [PubMed]
3. Alt-Epping B, Seidel W, Vogt J, et al. Symptoms and Needs of Head and Neck Cancer Patients at Diagnosis of Incurability - Prevalences, Clinical Implications, and Feasibility of a Prospective Longitudinal Multicenter Cohort Study. Oncol Res Treat 2016;39:186-91. [Crossref] [PubMed]
4. Basch E. Toward a Patient-Centered Value Framework in Oncology. JAMA 2016;315:2073-4. [Crossref] [PubMed]
5. te Boveldt ND, Vernooij-Dassen MJ, Jansen A, et al. Pain is not systematically registered in Dutch medical oncology outpatients. Pain Pract 2015;15:364-70. [Crossref] [PubMed]
6. Basch E, Deal AM, Kris MG, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol 2016;34:557-65. [Crossref] [PubMed]
7. Weingärtner V, Dargatz N, Weber C, et al. Patient reported outcomes in randomized controlled cancer trials in advanced disease: a structured literature review. Expert Rev Clin Pharmacol 2016;9:821-9. [Crossref] [PubMed]
8. Basch E, Dueck AC. Patient-reported outcome measurement in drug discovery: a tool to improve accuracy and completeness of efficacy and safety data. Expert Opin Drug Discov 2016;11:753-8. [Crossref] [PubMed]
9. Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health 2008;11:322-33. [Crossref] [PubMed]
10. Gaertner J, Elsner F, Pollmann-Dahmen K, et al. Electronic pain diary: a randomized crossover study. J Pain Symptom Manage 2004;28:259-67. [Crossref] [PubMed]
11. Bennett AV, Jensen RE, Basch E. Electronic patient-reported outcome systems in oncology clinical practice. CA Cancer J Clin 2012;62:337-47. [Crossref] [PubMed]
12. Johansen MA, Henriksen E, Horsch A, et al. Electronic symptom reporting between patient and provider for improved health care service quality: a systematic review of randomized controlled trials. part 1: state of the art. J Med Internet Res 2012;14:e118. [Crossref] [PubMed]
13. Hoppe C, Obermeier P, Muehlhans S, et al. Innovative Digital Tools and Surveillance Systems for the Timely Detection of Adverse Events at the Point of Care: A Proof-of-Concept Study. Drug Saf 2016;39:977-88. [Crossref] [PubMed]
14. Rose M, Bezjak A. Logistics of collecting patient-reported outcomes (PROs) in clinical practice: an overview and practical examples. Qual Life Res 2009;18:125-36. [Crossref] [PubMed]
15. Moroni M, Zocchi D, Bolognesi D, et al. The 'surprise' question in advanced cancer patients: A prospective study among general practitioners. Palliat Med 2014;28:959-64. [Crossref] [PubMed]