Outcome measurement in paediatric palliative care: lessons from the past and future developments

Introduction

The need for palliative care in children is great, with an estimated 21.6 million children globally needing access to palliative care (1). In many countries, paediatric palliative care (PPC) is a relatively new discipline. A review of the status of PPC provision globally (2) found that 65.6% of countries had no known PPC activity, with only 5.7% having mainstream provision. Whilst there have been developments since then, access to PPC remains a challenge for many. Indeed, a study in Kenya, Zimbabwe and South Africa (3,4) found that <1% of children needing PPC could access it in Kenya, and <5% in Zimbabwe and South Africa. The Global Atlas of Palliative Care at the end-of-life reinforces this gap, noting that low and middle income countries (LMICs) have the greatest number of children in need of end-of-life palliative care, but also the least provision (5,6). Likewise, the Lancet Commission (7) noted that >98% of the 2.5 million children dying annually with serious health related suffering (SHS) are from LMICs, with deaths in high income countries, where the majority of PPC services are found, accounting for <1% of all deaths with SHS. Whilst much of the evidence on the impact and outcomes of palliative care comes from high income countries, similar findings are being reported in LMICs (8), hence demonstrating PPC can be implemented in a wide variety of settings and cultures.

While the need for PPC is undisputable, there is limited evidence of the quality or outcomes of the care provided. A review of PPC in sub-Saharan Africa (9,10) found only 5 peer reviewed papers published on PPC in the region, reporting on Uganda and South Africa and only one of these looked at outcomes for children. The report recommended researchers need to provide evidence for PPC, including the development of appropriate outcome tools as “the evidence base in Africa has not progressed for paediatric as it has for adult palliative care. A fundamental reason for this is the lack of locally relevant, validated tools to measure outcomes for children”. This was echoed by the Lancet Commission (7) which recommended the implementation of a “rigorous, vigorous and substantive research agenda” that provides the tools to measure the outcomes of the care provided.

The absence of an outcome measure for PPC has been consistently highlighted. First, a systematic review of outcome measures in PPC identified 27 potential instruments (11). However, the domains, recall and response format were not considered appropriate, and no measures scored at least “fair” on the COSMIN checklist. A referral tool exists (12), but nothing measures outcomes. The paediatric version of the Palliative Outcome Scale (POS) for sub-Saharan Africa is the only available measure (13). Second, the Oxford patient reported outcome measures (PROMS) group report to the Department of Health (14) concluded there is inadequate attention to children and parent-reported measures, stating that “a number of key conceptual and methodological complexities must be carefully considered”, including acceptability, and need for child self-report. Third, the Medical Research Council (MRC) scoping workshop report on PROM methods identified two UK gaps: end-of-life, and childhood issues (15).

Methods

The absence of scientific activity to develop and implement PROMS for children and young people (CYP) with life-limiting and life-threatening conditions is lagging far behind the rapid progress made for adults (16-22). This absence is recognized as an impediment to generating robust evidence of effectiveness of PPC services (23). Advancing the science is a key priority—a systematic review of studies that investigated symptoms and concerns among CYP with life-limiting and life-threatening conditions found an over-reliance on proxies, poor description of methods to engage and promote participation for CYP, and a focus on malignant disease in high income countries (24). This paper sets out to review the current status of outcome measurement in PPC, reviewing lessons learnt and looking at the ongoing need for future development. Whilst it is recognized that there is a gap in the literature on outcome measurement on PPC, it is important that any focused issue looking at outcome measurements, addresses that of PPC even though the status of the science lags behind that in adults. This paper will discuss the need to measure outcomes in PPC, outcomes and quality measures in PPC, and the development of the African Children’s Palliative Outcome Scale (C-POS). Lessons learnt from the past will then be explored before looking ahead at the need for future developments in outcome measures in PPC.

A narrative review was undertaken utilising search terms such as: outcome measures palliative care +/− paediatric; PROMs children’s/PPC; outcome tools children’s/PPC; quality of life—children’s/paediatrics’. Databases searched included PubMed and CINHAL. Alongside this the authors have drawn upon reflective insights from their collective experiences.

Results

Why we need to measure outcomes in PPC

The measurement of outcomes of care is essential to ensure quality and efficacy of the care provided. Outcomes need to be measured across different diagnosis, in different settings and for routine clinical practice as well as audit and research. There is a lack of robust evidence within PPC (25,26) with practice being based on evidence from adult palliative care or on clinical/expert practice, or on service provision in high income countries which may not relate to that in LMICs. Thus, it is essential we increase and broaden the evidence base for PPC, including outcomes of care, to enable us to improve care as needed, and demonstrate the quality of the care that we are providing, and the impact of such services. All of these are essential if we are to continue to develop service provision globally, and ensure we are providing the quality care that is required. A Delphi study identifying priorities for global research into PPC (27) identified the need to measure outcomes of care within its top ten priorities. Likewise, patient outcomes and effectiveness was identified as a topic for adult and PPC research in Africa (28), and in Canada, where the establishment of core quality indicators for PPC and the evaluation of outcomes appeared within the top 20 priorities as did the utilisation of the outcome measures to compare outcomes of care in different settings (29).

The global policy context

Palliative care is a core component of universal health coverage (UHC) (30) and an essential part of working towards meeting the sustainable development goals (31), Likewise, the integration of palliative care, including PPC, is part of the World Health Assembly Resolution on palliative care (8) which urges all member states to “integrate evidence-based, cost-effective and equitable palliative care services in the continuum of care, across all levels.” Alongside this it promotes “the development and implementation of evidence-based guidelines and tools on palliative care…. In adults and children” (8). More and more, funders, governments and other stakeholders, including patients and families, are insisting on the provision of outcomes data (32). Without the measurement of outcomes in PPC the field will fall further behind in demonstrating its effectiveness, the impact it can have on CYP and their families, its role in UHC, and its role in health systems strengthening.

Outcome and quality measures in PPC

An outcome can be described as “the change in a patients current and future health status that can be attributed to preceding healthcare (33)”. Outcomes are what directly affect the CYP and their family, such as change in pain levels, improved quality of life etc. (32). Outcomes can be measured in a variety of ways, some of which are easier than others. For example, an outcome of a surgical intervention could be the number of patients who die due to the surgery, a simple measure that can be easily recorded. Whereas, due to the multi-dimensional nature of PPC, outcome measures will inevitably be more complex and harder to measure. Whilst a variety of such PROMs have been developed for use in adult palliative care, such as: the Palliative care Outcome Scale (POS) (34); the Integrated Palliative care Outcome Scale (I-POS) (35); the Distress Thermometer (36); the Edmonton Symptom Assessment Scale (ESAS) (37); the Memorial Symptom Assessment Scale (MSAS) (38); and the EORTC QLQ-C30 (39); a similar range of tools does not exist within PPC.

A literature review was conducted in 2009 by the African Palliative Care Association (APCA) prior to the development of a PROM for CYP (40). The review aimed to identify the important outcome domains in PPC and what outcome tools exist for PPC. Three main outcome domains were identified—physical care, spiritual care and psychosocial care, with the latter extending to the families as well as the child. Three main categories of assessment tools were identified: self-report, behavioural and physiological, with self-report being seen as the gold standard. A variety of validated pain assessment tools are available for different age ranges and other tools are being utilized even if not specifically developed for the paediatric or palliative care population, e.g., the MSAS (38). Likewise, various spiritual assessment tools exist that can be utilized in a paediatric care context (41), such as FICA (42) and BELIEF (41). Psychological assessment tools were also identified, including the Depression, Anxiety and Stress Scale (DASS) (43) and the Beck Youth Inventories (BYI) (44). Thus, the majority of the tools identified were unidimensional, with only a few, such as the PedsQL (45) being multi-dimensional addressing physical, emotional, social and school functioning. The review identified gaps in outcome measurement for CYP including that: there was literature about pain, but less about spiritual and psychosocial care; most of the literature related to cancer; and that there were several unidimensional tools, and multi-dimensional tools for quality of life, but no specific outcome measure for PPC.

Coombes et al. (11) undertook a systematic literature review and analysis of psychometric properties for health-related quality of life (QoL) outcome measures in PPC. They identified 22 health-related QoL measures and found that the quality of the studies varied greatly, with missing data and limited analysis of measurement error and responsiveness. They found that the domains of health related QoL measures were not all relevant for PPC and some items were disease specific. They also concluded that there was no ‘ideal’ PPC outcome measurement tool. Thus, the challenge remains within PPC to develop and implement a multi-dimensional, age appropriate, easy to use, outcome measure.

Development of the APCA C-POS

In recognition of the need for an outcome scale specifically for CYP and following on from their success in developing, piloting and validating an outcome scale for Africa—the APCA African POS (46,47), the African Palliative Care Association set out to develop a multi-dimensional outcome scale for PPC—the African C-POS (C-POS). The process involved organizations from across eight African countries (Kenya, Malawi, Namibia, South Africa, Swaziland, Uganda, Zambia, Zimbabwe) alongside the International Children’s Palliative Care Network and King’s College London. Starting in 2009, an initial tool was developed and revised in 2014, with validation and psychometric testing being completed in 2017. The tool development and validation followed the COSMIN guidance for the development and testing of health measurement tools (48,49).

It was important to identify the domains of care that needed to be covered within such a multi-dimensional tool. PPC experts from the region identified key domains including pain, symptoms, distress, QoL, communication and family support. Further domains were discussed and identified such as education and finances, but it was not clear how to include these, or whether they could be seen as ‘outcomes’ of PPC. Having identified the domains, questions were developed. It was decided that where existing validated questions could be used then they should be included and tested within the paediatric population. There were discussions with regards to whether there should be separate tools for young children and adolescents, and for verbal and non-verbal children, and whether there should be one for the child to complete and one to be completed by a proxy.

Based on a similar format to the APCA African POS, the initial C-POS consisted of 14 questions, nine in Section A for the child and five in Section B aimed at the parent or carer. Answers were scored using Likert-type scales ranging from 0 to 5. Questions could be scored using numerical and descriptive labels, the hand scale (50,51) or the faces scale (50-52). The initial tools were piloted in four sites in Kenya, South Africa and Uganda (13), following which the results were reviewed by PPC experts from across SSA, after which the tool was revised and the two versions combined.

Following this, work has been ongoing on the utility of the tool, its acceptability and feasibility in practice, along with validation and psychometric testing, establishing face, content and construct validity, reliability and acceptability. Work has been completed ensuring the tool is fit for practice, validated and reliable. Whilst the tool can be found in Figure 1, it is currently being reviewed and finalized based on the validation and psychometric testing completed in 2017. This has demonstrated the need for a separate outcome measure for adolescents, to that of both children and adults, and through the mapping of identified needs with the tool, it is anticipated that a couple of questions may be revised. Thus, it is anticipated that the finalized C-POS will be available in 2018.

Figure 1 The draft African Children’s Palliative Outcome Scale (C-POS).

The C-POS is the first specifically designed multi-dimensional outcome measure for PPC. It is not possible to measure everything, so identifying the priorities is key, along with ensuring that the domains/questions included can be impacted by the care that is being provided by the PPC service so that changes seen over time are as a result of the care provided. Thus, it is not perfect, and no outcome measure is perfect (32), but it is a good start in the measurement of outcomes of PPC, and forms a basis upon which other measures can be developed. This C-POS has already been used in different clinical and research settings and is being used as a basis for the development of a similar outcome measure for PPC in Europe.

Discussion

Lessons from the past

Over the past few years there has been a plethora of papers written about outcome measures as their importance has been recognized within palliative care. Lessons have been learnt and recommendations provided, such as through the European Association for Palliative Care (EAPC) White Paper on outcome measurement in palliative care (19) which identifies 12 recommendations covering different aspects of the utilisation of outcome measures, focusing on their use in clinical care and the impact of implementing their use in palliative care practice. Whilst generic, these recommendations apply within the field of PPC (Figure 2). However, specific to the field of PPC, many lessons have been learnt in the development of the C-POS.

Figure 2 Recommendations on outcome measurement in palliative care from the EAPC White Paper on outcome measurement (19).

Undertaking research in PPC

One of the challenges to the development of a PROM for PPC has been the barriers associated with undertaking research with such a vulnerable population. A tool such as this cannot be developed without involving CYP with life-limiting and life-threatening conditions, and doing this can be difficult. There are numerous challenges including: small sample sizes; difficulty in recruitment; challenges with getting ethical approval; the unpredictable nature of the child’s condition; and societies perceptions of the potential burden for children and their families (25,26). Alongside these, a survey conducted in July 2015 also identified: a lack of time and other resources as barriers along with small PPC teams with limited capacity to take on any additional work, such as research and audit; clinician’s attitudes towards research; and clinician’s perceptions of patients and their families, such as a concern to overburden the child and their families (26).

Changing attitudes to research in PPC is essential, not just for the development and validation of such outcome measures, but also in their ongoing utilisation for clinical practice, audit and research. Developing and increasing the PPC evidence base must be a priority, and outcome measures have a key role in this. Challenges, such as those listed above, can be overcome and experience has shown that CYP and families are happy to be involved in research, despite their vulnerable situations (25). Local, national and international collaborations, such as was seen in the development of the C-POS, are essential in order that experiences can be shared and barriers to undertaking research in PPC overcome.

Definition of PPC

There are various definitions of PPC, which need to be taken into consideration if developing an outcome measure that can be applied across different contexts (Figure 3). Whilst the definition is important, and it is from this that the outcomes of what we perceive PPC to be should form the basis of any PROM, it is also important not to get too ‘bogged down’ in the definition. Each of the definitions above has the holistic components of physical, psychological, social and spiritual suffering, so in reviewing the domains of outcomes of care required in PPC, these are similar. Thus, it is these domains and outcomes that are important and they can be mapped across the differing definitions.

Figure 3 Definitions of PPC and SHS. PPC, paediatric palliative care; SHS, serious health related suffering(7,53,54).

If you ask a child what their concerns are they will tell you

One of the important things about an outcome measure is that it measures the outcomes that are important to the CYP and their family, and not those important to the health professional. Thus, an essential component of the development of an outcome measure for PPC is to interview the CYP and their families and find out what is important for them. A misconception in research in paediatrics is that children are unable to tell you what they want, that some of the topics might be too sensitive for the child, that you will upset the child etc. This is not the case, and there have been many instances of qualitative research with children as young as five on sensitive issues (55).

Experience in interviewing children during the development of the C-POS is that they are able to tell you what is important to them, the outcomes of the care that they would like to see, being realistic that for many of them, cure is not an option. Throughout the development of the C-POS children have been interviewed across a range of ages with a variety of life-limiting and life-threatening conditions such as HIV/AIDS, cancer, organ failure etc. Whilst many of the issues identified by the CYP map well with the initial C-POS, there are areas that have been identified that have not been included, and work is ongoing as to how these can be added, for example the importance of education, of going to school, of keeping up with school work whilst in hospital, and of the social element of school.

Child and/or proxy report?

One of the areas of contention in the development of PROMs is the issue of ‘patient-reported’ versus ‘proxy-reported’ instruments (56). Whilst the gold standard for outcome measurement is patient-reported (32), this may not always be possible in PPC, due for example to: the age of the child; the condition of the child such as if they are unconscious; if they do not wish to open up to a stranger; if they have limited vocabulary; and whether the child is able to talk, or communicate in other ways such as by pointing (57). PROMs are usually self-completed or can be facilitated by an interviewer. The reality within PPC is that the child will be helped, either by their family, or the health care professional to complete the outcome measure.

Research has been undertaken into the use of proxies in completing assessment tools and outcome measures and shows that health professionals tend to underestimate the severity of a patient’s symptoms, particularly in terms of pain, fatigue and breathlessness (33,58,59). The debate continues as to how accurate proxy ratings by parents or carers is in terms of reflecting the child’s views. Davis et al. (56) undertook comprehensive enquiry into the discordance of the self and proxy views and noted that a discrepancy between parents and children in terms of the rating scales and the reasons for their answers. Where appropriate both the child and the carers estimation can be included, so that a comparison can be seen, so if the child is no longer able to complete the outcome measure, the carer can do so, and there is some evidence as to how these two correlate (32). However, if the same proxy is responding and completing the outcome measure on each occasion then change over time should still be seen, regardless as to whether the scores correlate exactly. Within PPC, whilst wanting to ensure accuracy, it is important to also be pragmatic, and where the only option for completion of an outcome measure is that of a proxy then it is important to ensure that that option is available. One plausible recommendation is to use observable indicators, which call for less reasoning to make proxy indicators more accurate (60,61). Theunissen et al. (62) compared child and proxy reports and concluded that both are valid, increasing the essence of their use in PPC (57,62).

Different tools for different ages?

In the development of the C-POS there was an attempt to keep it simple, with just one child outcome scale alongside the adult scale, with adolescents and young people having the choice of which to use. However, following on from the qualitative interviews it appears that there may be a need for an addition of a few items targeting the priorities of adolescents in the social and spiritual domains. This is still under review and brings with it the challenges as to at what stage does a child stop using the outcome measure for children and start using the one for adolescents, and then again moving on from adolescents to the adult version. This has always been an issue in the field of ‘Orphans and Vulnerable Children’ with best practice being to allow experts to judge what is suitable for the child’s development stage. This may differ per setting (rural/urban) and progress towards cognitive development. There is also a question mark as to the usefulness of the current C-POS in neonates, babies and very young children.

What methods of scoring should be used?

When the C-POS was first developed, children were given the option of scoring the questions via a numerical rating scale, via verbal descriptors, using the faces scale, or using the hands scale (13). The hands scale is one that is used regularly within the African context both for adults and children, particularly with regards to pain scores. Work has been undertaken in adults with regards to the utilisation of different pain scales including comparing numerical scoring to using the hand scale, the faces scale or a jerry can. Results suggest that using the hands and faces scoring methods correlates well with using numerical scaling, although there was not the same evidence for the use of a jerry can (50). Whilst this work was completed with adults, it is the best evidence available to suggest that using the hands and faces correlates well with numerical scoring, and there is no reason to suggest this would be otherwise in children. The important thing to remember is the need for explanation as to how to uses these scales, with the most common misconception being that the faces scale is about emotions rather than pain.

Outcome tool or assessment tool, or both?

Whilst the C-POS has been developed as an outcome measure, it has also been useful within the context of assessment of the CYP and their family. At all stages of development health professionals have felt that utilizing the tool has given them insight into the child’s condition and how their carer/family were feeling, and has encouraged them to think more broadly about the care of the CYP and their family (13). The recommendation is that outcome measures can be used for clinical, audit and research purposes—they are helpful with assessment, monitoring and reporting (32). Evidence from the development of outcome measures for adults also suggests its usefulness in regular clinical practice and in ascertaining a baseline against which to measure change (46,47,63). In the development process of the APCA African POS the nurses commented how utilizing the outcome measure helped to give some structure to their assessment and helped them to ask questions they would otherwise have found challenging (46). Thus, many services will use an outcome measure such as the APCA African POS routinely on admission to hospital or during the first visit at home. It can then be used at regular intervals to assess change from a clinical perspective, but also for audit and research purposes (64).

Length of the outcome measure

The length of an outcome measure and hence the time that it takes to complete is important. Research has shown that the length of an outcome measure is crucial in its usability and acceptance, with the length of time it takes to complete an outcome measure impacting on its use (65). When asked about the ideal outcome measure for use in the African context, 73% said that it should have between 6 and 15 questions, i.e., long enough to be multi-dimensional, but short enough so that it is not a burden either to the patient or staff (65). Generally, outcome measures that are too long for patients to answer, or that require a lot of time for administration, are not utilized regularly, with both issues being important within palliative care where time is limited and the individuals have complex needs and may be very frail. Thus, it is important to get the balance between the length of the outcome measure, its sound psychometrics and the feasibility of its use (65). Most importantly for children, younger children need shorter interviews whilst older children (10 and above) can stand longer interviews and this is an important consideration for outcome measure development in this population (66).

Development of an outcome measure takes time

The development, validation and assessment of psychometric properties of the C-POS has taken many years and it is important to allow adequate time for development. Whilst the development of this initial outcome measure has taken 7–8 years, it is anticipated that future measures could be developed in a much shorter time, with 3–4 years being a realistic time frame. Challenges that can delay progress are varied and include issues such as: the length of time it takes to get ethical approval for a study involving PPC; if working in different countries, ethical approval is needed in each country; changes in personnel in the study team; unrest or natural disasters; interviewing paediatric patients being labour intensive; ensuring good representation of all age brackets, and disease conditions and phases of disease; and, where the numbers of children requiring PPC are small, it may take time to recruit enough CYP and their families into the study.

It won’t be perfect!

If we are looking for the perfect outcome measure for PPC we will be disappointed and will never get one. Due to the nature of outcome measures, the nature of PPC, the need to have a multi-dimensional measure; and the individual aspect of PPC, it will not be possible to develop an outcome measure that accurately measures the outcomes of PPC in all instances. However, as we develop outcome measures, and refine the process and refine what we are looking for, the validity and reliability of the measures will improve. Whilst the same outcome measures are used to measure change over time, it will be possible to measure some, if not all, of the outcomes of PPC, thus impacting on the care that we provide, helping us to provide quality PPC.

Future developments

Whilst the development of outcome measures in adults has progressed greatly, there is still a long way to go in the development of outcome measures for PPC. The development of the C-POS is an important step further, but ongoing work is needed, learning from the work with the C-POS along with the use of outcome measures in adult palliative care. Some future developments include:

Finalisation and publication of the African C-POS

The finalisation of the C-POS is in its final stages, with the publication of the final tool being anticipated in 2018. Meanwhile information about its validity and psychometric properties has been submitted for publication. The finalisation of this tool is urgent, as many organizations are looking to utilize it, both in routine clinical practice and research.

Utilisation of the C-POS in clinical practice

As PPC services are developing, the routine use of an outcome measure can be integrated into their practice, thus becoming the norm rather than the exception. Through doing this a baseline assessment followed by regular reviews will be undertaken on all children accessing a service, providing routine audit data along with longer term research data. Importantly, though, the routine use of an outcome measure in PPC can influence the quality of the care provided.

Utilisation of the C-POS in research and audit

As has already been noted, there is a paucity of evidence within the field of PPC. One of the reasons cited behind the lack of evidence was the lack of outcome measures (7,8,10). Thus researchers should be encouraged to use outcome measures such as the C-POS in their research in order to begin to develop the evidence base. Examples demonstrate the use of an outcome measure for research/audit in palliative care has led to improvements in the care being provided, e.g., in the USA (67) and Uganda (68).

Collation and review of data sets

Work is being undertaken nationally, regionally and internationally to look at the use of minimum data sets for palliative care, including the use of outcome measures. To date it has not been possible to include paediatrics in this due to the lack of a validated outcome measure for PPC. With the generation of such measures, and their integration into routine clinical practice, this will change, and it will be possible to incorporate paediatrics firmly into this work.

Development of C-POS in different settings

The C-POS has been developed specifically within the sub-Saharan Africa region, thus it is important to either validate it, or adapt it for different settings. Whilst it is important not to reinvent the wheel, and the core principles and philosophy of PPC may be similar in different countries, it is essential that any outcome measure is appropriately validated in different contexts and languages. Guidelines are available for the cultural adaptation of outcome measures which can be applied both to the paediatric as well as adult settings (69). Work is currently underway to develop a C-POS in Europe, drawing on the innovation and the tool development led from Africa.

Conclusions

The measurement of outcomes in PPC is an imperative in order to assess the care that we are giving, review the outcomes of care and ensure that we are providing a quality PPC service to CYP and their families. Whilst there are challenges in identifying priority outcomes, and in undertaking research in the PPC population, these should not stop us from developing outcome measures. The development of the first outcome measurement tool for PPC—the African C-POS is a key milestone in the ongoing development and utilisation of outcome measurements in the field. It is essential that as a PPC community we build on this, and strengthen both the development and utilisation of outcome measures in PPC practice, audit and research.

Acknowledgements

We acknowledge everyone who has been involved in the ongoing development of the C-POS.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

References

1. Connor SR, Downing J, Marston J. Estimating the global need for palliative care for children: A cross-sectional analysis. J Pain Symptom Manage 2017;53:171-7. [Crossref] [PubMed]
2. Knapp C, Woodworth L, Wright M, et al. Pediatric Palliative Care Provision Around the World: A Systematic Review. Pediatr Blood Cancer 2011;57:361-8. [Crossref] [PubMed]
3. Connor SR, Sisimayi C. Assessment of the Need for Palliative Care for Children: Three Country Report: South Africa, Kenya and Zimbabwe. UNICEF and ICPCN, London, 2013.
4. Connor S, Sisimayi C, Downing J, et al. Assessment of the need for palliative care for children in South Africa. Int J Palliat Nurs 2014;20:130-4. [Crossref] [PubMed]
5. Connor SR, Sepulveda Bermedo MC. Global Atlas of Palliative Care at the End of Life. London: Worldwide Palliative Care Alliance, 2014.
6. Downing J, Powell RA, Marston J, et al. Children’s palliative care in low and middle-income countries. Arch Dis Child 2016;101:85-90. [Crossref] [PubMed]
7. Knaul FM, Farmer PE, Krakauer EL, et al. Alleviating the access abyss in palliative care and pain relief-an imperative of universal health coverage: the Lancet Commission report. Lancet 2018;391:1391-454. [Crossref] [PubMed]
8. World Health Assembly. Strengthening of palliative care as a component of integrated treatment within the continuum of care. 134th session. EB134.R7. 2014.
9. Harding R, Sherr L, Albertyn R. The status of paediatric palliative care in sub-Saharan Africa – An Appraisal. Executive Summary. King’s College London and the Diana, Princess of Wales Memorial Fund, London, 2010.
10. Harding R, Albertyn R, Sherr L, et al. Pediatric palliative care in Sub-Saharan Africa: a systematic review of the evidence for care models, interventions and outcomes. J Pain Symptom Manage 2014;47:642-51. [Crossref] [PubMed]
11. Coombes LH, Wiseman T, Lucas G, et al. Health-related quality-of-life outcome measures in paediatric palliative care: A systematic review of psychometric properties and feasibility of use. Palliat Med 2016;30:935-49. [Crossref] [PubMed]
12. Bergstraesser E, Hain RD, Pereira JL. The development of an instrument that can identify children with palliative care needs: the Paediatric Palliative Screening Scale (PaPaS Scale): a qualitative study approach. BMC Palliat Care 2013;12:20. [Crossref] [PubMed]
13. Downing J, Atieno M, Powell RA, et al. Development of a palliative care outcome measure for children in sub-Saharan Africa: findings from early phase instrument development. Eur J Palliat Care 2012;19:292-5.
14. Morris C, Gibbons E, Fitzpatrick R. Child and Parent reported outcome measures: a scoping report focusing on feasibility for routine use in the NHS: A Report to the Department of Health. University of Oxford, 2009.
15. Patient-Reported Outcome Measures (PROMs): Identifying UK Research Priorities. Report of a MRC Workshop. 12th January 2009, Royal College of Physicians. London: MRC, 2009.
16. Antunes B, Harding R, Higginson IJ, et al. Implementing patient-reported outcome measures in palliative care clinical practice: A systematic review of facilitators and barriers. Palliat Med 2014;28:158-75. [Crossref] [PubMed]
17. Daveson BA, Alonso JP, Calanzani N, et al. Learning from the public: citizens describe the need to improve end-of-life care access, provision and recognition across Europe. Eur J Public Health 2014;24:521-7. [Crossref] [PubMed]
18. Harding R, Simms V, Calanzani N, et al. If you had less than a year to live, would you want to know? A seven-country European population survey of public preferences for disclosure of poor prognosis. Psychooncology 2013;22:2298-305. [PubMed]
19. Bausewein C, Daveson BA, Currow DC, et al. EAPC White Paper on outcome measurement in palliative care: Improving practice, attaining outcomes and delivering quality services - Recommendations from the European Association for Palliative Care (EAPC) Task Force on Outcome Measurement. Palliat Med 2016;30:6-22. [Crossref] [PubMed]
20. Collins ES, Witt J, Bausewein C, et al. A Systematic Review of the Use of the Palliative Care Outcome Scale and the Support Team Assessment Schedule in Palliative Care. J Pain Symptom Manage 2015;50:842-53. [Crossref] [PubMed]
21. Etkind SN, Daveson BA, Kwok W, et al. Capture, transfer, and feedback of patient-centered outcomes data in palliative care populations: does it make a difference? A systematic review. J Pain Symptom Manage 2015;49:611-24. [Crossref] [PubMed]
22. van Vliet LM, Harding R, Bausewein C, et al. How should we manage information needs, family anxiety, depression, and breathlessness for those affected by advanced disease: development of a Clinical Decision Support Tool using a Delphi design. BMC Med 2015;13:263. [Crossref] [PubMed]
23. Harding R, Wolfe J, Baker JN. Outcome Measurement for Children and Young People. J Palliat Med 2017;20:313. [Crossref] [PubMed]
24. Namisango E, Murtagh F, Bristowe K, et al. Symptoms and concerns among children and young people with life-limiting and life-threatening conditions: a systematic review. 3rd ICPCN Conference Abstract Book. 2018:14.
25. Downing J. Editorial: To research or not to research and concerns among children and young people with life-limiting and life-threaten. Palliat Med 2016;30:902-3. [Crossref] [PubMed]
26. Beecham E, Hudson BR, Oostendorp L, et al. A call for increased paediatric palliative care research: Identifying barriers. Palliat Med 2016;30:979-80. [Crossref] [PubMed]
27. Downing J, Knapp C, Mukaden MA, et al. Priorities for global research into children’s palliative care: results of an International Delphi Study. BMC Palliat Care 2015.14:36.
28. Powell RA, Harding R, Namisango E, et al. Palliative Care Research in Africa: Consensus Building for a Prioritized Agenda. J Pain Symptom Manage 2014;47:315-24. [Crossref] [PubMed]
29. Baker JN, Levine DR, Hinds PS, et al. Research Priorities in Pediatric Palliative Care. J Pediatr 2015;167:467-70. [Crossref] [PubMed]
30. World Health Organization. Health financing for universal health coverage. Available online: http://www.who.int/health_financing/universal_coverage_definition/en/
31. The United Nations Development Programme (UNDP) Sustainable Development Goals. Geneva: UNDP, 2000.
32. Bausewein C, Daveson B, Benalia H, et al. Outcome Measurement in Palliative Care: The Essentials. PRISMA, 2009.
33. Donabedian A. Explorations in quality assessment and monitoring. Michigan: Ann Arbor Health Administration Press, 1980.
34. Hearn J, Higginson IJ. Development and validation of a core outcome measure for palliative care: the palliative care outcome scale. Palliative Care Core Audit Project Advisory Group. Qual Health Care 1999;8:219-27. [Crossref] [PubMed]
35. Schildmann EK, Groeneveld EI, Denzel J, et al. Discovering the hidden benefits of cognitive interviewing in two languages: The first phase of a validation study of the Integrated Palliative care Outcome Scale. Palliat Med 2016;30:599-610. [Crossref] [PubMed]
36. Gessler S, Low J, Daniells E, et al. Screening for distress in cancer patients: is the distress thermometer a valid measure in the UK and does it measure change over time? A prospective validation study. Psychooncology 2008;17:538-47. [Crossref] [PubMed]
37. Bruera E, Kuehn N, Miller M, et al. The Edmonton Symptom Assessment System (ESAS): a simple method for the assessment of palliative care patients. J Palliat Care 1991;7:6-9. [PubMed]
38. Portenoy RK, Thaler HT, Kornblith AB, et al. The Memorial Symptom Assessment Scale: an instrument for the evaluation of symptom prevalence, characteristics and distress. Eur J Cancer 1994;30A:1326-36. [Crossref] [PubMed]
39. Aaronson NK, Ahmedzai S, Bergman B, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 1993;85:365. [Crossref] [PubMed]
40. Matthews S, Downing J. Paediatric Palliative Care: Domains, Tools and Research. Kampala: APCA, 2009.
41. Elkins M, Cavendish R. Developing a plan for pediatric spiritual care. Holist Nurs Pract 2004;18:179-84. [Crossref] [PubMed]
42. Puchalski C. Spiritual assessment tool: FICA. Innovations in End-of-Life Care 1999;1:6.
43. Depression, Anxiety, Stress Scales. Available online: http://www2.psy.unsw.edu.au/groups/dass//
44. Steer RA, Kumar G, Beck JS, et al. Evidence for the Construct Validities of the Beck Youth Inventories with Child Psychiatric Outpatients. Psychol Rep 2001;89:559-65. [Crossref] [PubMed]
45. Sherman SA, Eisen S, Burwinkle TM, et al. The PedsQLTM present functioning visual analogue scales: preliminary reliability and validity. Health Qual Life Outcomes 2006;4:75. [Crossref] [PubMed]
46. Powell RA, Downing J, Harding R, et al. Development of the APCA African Palliative Outcome Scale. J Pain Symptom Manage 2007;33:229-32. [Crossref] [PubMed]
47. Harding R, Selman L, Agupio G, et al. Validation of a core outcome measure for palliative care in Africa; the APCA African Palliative Outcome Scale. Health Qual Life Outcomes 2010;8:10. [Crossref] [PubMed]
48. Mokkink LB, Terwee CB, Knol DL, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: A clarification of its content. BMC Med Res Methodol 2010;10:22. [Crossref] [PubMed]
49. Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 2010;63:737-45. [Crossref] [PubMed]
50. Blum D, Selman LE, Agupio G, et al. Self-report measurement of pain and symptoms in palliative care patients: a comparison of verbal, visual and hand scoring methods in sub-Saharan Africa. Health Qual Life Outcomes 2014;12:118. [Crossref] [PubMed]
51. Hospice Africa Uganda. Palliative Medicine: Pain and symptom control in the cancer and/or AIDS patient in Uganda and other African countries, 4th edn. Kampala, Uganda: HAU, 2006.
52. Bieri D, Reeve R, Champion GD, et al. The Faces Pain Scale for the self-assessment of the severity of pain experienced by children: Development, initial validation and preliminary investigation for ratio scale properties. Pain 1990;41:139-50. [Crossref] [PubMed]
53. . Available online: www.who.int/cancer/palliative/definition/en/WHO Definition of Palliative Care for Children.
54. Together for Short Lives. Children’s Palliative Care Definitions. Available online: http://www.togetherforshortlives.org.uk/professionals/childrens_palliative_care_essentials/definitions
55. Bikaako-Kajura W, Luyirika E, Purcell DW, et al. Disclosure of HIV Status and Adherence to Daily Drug Regimens Among HIV-infected Children in Uganda AIDS Behav 2006;10:S85-93. [Crossref] [PubMed]
56. Davis E, Nicolas C, Waters E, et al. Parent-proxy and child self-reported health-related quality of life: using qualitative methods to explain the discordance. Qual Life Res 2007;16:863-71. [Crossref] [PubMed]
57. Bevans KB, Riley AW, Moon J, et al. Conceptual and methodological advances in child-reported outcomes measurement. Expert Rev Pharmacoecon Outcomes Res 2010;10:385-96. [Crossref] [PubMed]
58. Nekolaichuk CL, Bruera E, Spachynski K, et al. A comparison of patient and proxy symptom assessments in advanced cancer patients. Palliat Med 1999;13:311-23. [Crossref] [PubMed]
59. Laugsand EA, Sprangers MAG, Bjordal K, et al. Health care providers underestimate symptom intensities of cancer patients: A multicentre European study. Health Qual. Life Outcomes 2010;8:104. [Crossref] [PubMed]
60. Matza LS, Swensen AR, Flood EM, et al. Assessment of health-related quality of life in children: A review of conceptual, methodological, and regulatory issues. Value Health 2004;7:79-92. [Crossref] [PubMed]
61. Matza LS, Patrick DL, Riley AW, et al. Pediatric patient-reported outcome instruments for research to support medical product labeling: report of the ISPOR PRO good research practices for the assessment of children and adolescents task force. Value Health 2013;16:461-79. [Crossref] [PubMed]
62. Theunissen NCM, Vogels TGC, Koopman HM, et al. The proxy problem: Child report versus parent report in health-related quality of life research. Qual Life Res 1998;7:387-97. [Crossref] [PubMed]
63. Higginson IJ, Simon ST, Benalia H, et al. Republished: Which questions of two commonly used multidimensional palliative care patient reported outcome measures are most useful? Results from the European and African PRISMA survey. Postgrad Med J 2012;88:451-7. [Crossref] [PubMed]
64. Defilippi K, Downing J. Commentary: Feedback from African palliative care practitioners on the use of the APCA POS. Int J Palliat Nurs 2013;19:577-8. [Crossref] [PubMed]
65. Downing J, Simon ST, Mwangi-Powell FN, et al. Outcomes an palliative care practitioners on the use of the APCA POS. Int J Palliat lliative care in Africa. BMC Palliat Care 2012;11:1. [Crossref] [PubMed]
66. Clark A, Moss P. Listening to young people: The Mosaic approach Joseph Rowntree and The National Children's Bureau, London, 2001.
67. Wolfe J, Orellana L, Cook EF, et al. Improving the care of children with advanced cancer by using an electronic patient-reported feedback intervention: results from the PediQUEST randomized controlled trial. J Clin Oncol 2014;32:1119-26. [Crossref] [PubMed]
68. Asiimwe BM. Quality of life for paediatric cancer patients at New Hope Childrened cancer Abstract Book for the UCI-PCAU Joint International Conference on Cancer and Palliative Care and the 7th Palliative Care Conference. Uganda: UCI-PCAU, 2017.
69. The Palliative care Outcome Scale family of measures; Manual for cross- cultural adaptation and psychometric validation. Available online: http://pos-pal.org/ Resources.php