Quality appraisal of clinical practice guidelines on physical restraints in ICU: a systematic review

Introduction

Patients in the intensive care unit (ICU) often require invasive catheters to assist in their treatment due to their condition, such as tracheal intubation, central venous catheter (CVC), and urinary catheters (1). As it is believed to limit the patient’s movement (2), physical restraints have long been used as a protective measure for patients in the ICU to ensure that patients’ life-support catheters are not accidentally removed (3), which has resulted in physical restraints rate 23.4 times higher in the ICU than in the general wards (4). The rate of physical restraint use varies between countries. In the United States, a study that included 68 ICUs showed that the physical restraints rate in ICUs was 33% (5,6). In China, where several scholars have conducted localized surveys over the past decade, the physical restraints rate fluctuated between 35.1% and 77.2% (7-10). Physical restraints are widely used in ICU around the world (11).

With the establishment of the “biopsychosocial medical model”, the care of patients has gradually changed from “disease-centered” to holistic care centered on the physical and psychological health of patients (12). Therefore, when using physical restraints on ICU patients, medical staff should consider the physical and psychological effects of physical restraints. Unfortunately, it has been proven that physical restraints may lead to delirium (13), neurovascular complications (e.g., redness, limb movement, oedema, and colour complication) (14) and worsen agitation (15). Besides, while the main reason for using physical restraints in the ICU is to prevent the extubation (16), some studies (17-19) have pointed out that physical restraints can increase the level of anxiety and irritability of patients, but instead increase the risk of unplanned extubation and falls, which cannot ensure patient safety.

High quality clinical practice guidelines (CPGs) can enhance the health care quality through presenting recommendations to decision makers (20). Nurses are the primary decision-makers in applying physical restraints (3,21). So, reducing the use of physical restraints requires changing nurses’ perceptions and increasing knowledge of the proper use of physical restraints, guided by high-quality CPGs (22). Although physical restraints CPGs have been published by different organizations, the quality of available ones remains unknown. Our study aims to evaluate the quality of physical restraints guidelines from methodological and reporting perspective and to inform future practice improvement and guideline development. We conducted the systematic review following the PRISMA reporting checklist (available at https://apm.amegroups.com/article/view/10.21037/apm-21-2851/rc) (23).

Methods

Data sources and search strategy

On Nov 21, 2021, two reviewers (RL, YL) independently searched MEDLINE (via PubMed), Web of Science, Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL), China National Knowledge Infrastructure (CNKI), and Wanfang Data since inception using search terms, such as “physical restraint”, “intensive care unit”, “critical care”, “acute care”, “practice guideline”, etc. The details of the search strategies are shown in Appendix 1. We also searched Google and some relevant websites (GIN, NICE, SIGN, RNAO, AHRQ, AACN) using “physical restraint” as search terms.

Inclusion and exclusion criteria

We included guidelines based on the following inclusion criteria: (I) articles met the definitions of the guidelines proposed by the institute of medicine (IOM) in 1990 or 2011 (24,25). When evidence quality is low or very low, the guideline panels label them as consensus statements (26). So, in this study, we included statements based on evidence; (II) guidelines focus on physical restraint; (III) the settings of guidelines include critical care and acute care settings (general residential care setting were excepted); (IV) guidelines focus on adult patients; (V) guidelines were published between January 2001 and November 2021 in Chinese or English.

Our exclusion criteria were: (I) articles focus on chemical restraints or recommendations of physical restraints cannot be clearly distinguished; (II) previous version of guidelines, if there were two same guidelines by the same group/organization; (III) translated versions, brief versions or interpretations of the guidelines.

Study selection

Two reviewers (RL and YL) independently screened the titles, abstracts based on the inclusion and exclusion criteria through bibliographic software EndNote and then screened the full text. Disagreements were resolved by consensus and discussion with a third reviewer (XJ). Prior to the screening, a pilot test was performed until agreement on the screening process was reached.

Data extraction

Data were extracted by two reviewers (RL and YL) using a standardized form: organization, publication date, country/region, systematic literature search, recommendation formulation method, evidence quality grading, conflicts, funding.

Quality appraisal of guidelines

Two reviewers (RL and YL) independently used the Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool (27-30) to assess the methodological quality of the included guidelines. According to the AGREE II manual (27), it contains 23 items in six domains. Each item was rated on a seven-point scale (1-strongly disagree to 7-strongly agree). The domain quality score (between 0 to 100%) was calculated by aggregating the individual scores of the reviewers according to the formula provided in AGREE II handbook, scaled by the percentage of the highest possible score, and averaging the scores of the two reviewers. In addition, the mean scores of the six standardized domains were used to calculate the overall guideline assessment and to classify guidelines (31,32): “high quality” was score >80%; “moderate quality” was score between 50–80%; “low quality” was score <50%.

Two reviewers (RL and YL) independently used the Reporting Items for Practice Guidelines in Healthcare (RIGHT) checklist (33) to assess the reporting quality of the included guidelines. It contains 22 items in seven domains. Items rated as “reported” are scored as 1, items rated as “not reported” are scored as 0. Calculate the percentage of the total score to obtain the overall report assessment is divided into: “well-reported” was score >80%; “moderate-reported” was score between 50–80%; “low-reported” was score <50%.

Statistical analysis

Using SPSS 25.0 to calculate intra-class correlation coefficients (ICC) of two reviewers in AGREE II scores and RIGHT reporting scores to test agreement among reviewers. ICC >0.75 shows good reliability (34).

Results

A total of 635 articles were retrieved, and 74 duplicate articles were excluded; 542 articles were excluded after reading the title and abstract. Based on full-text screening, 13 articles were excluded, and six guidelines were included (35-40). The selection process was shown in Figure 1.

Figure 1 PRISMA 2020 flow diagram for new systematic reviews which included searches of databases and registers only. *, consider, if feasible to do so, reporting the number of records identified from each database or register searched (rather than the total number across all databases/registers); **, if automation tools were used, indicate how many records were excluded by a human and how many were excluded by automation tools.

The ICC values for two reviewers were 0.820 (95% CI: 0.757–0.868) in the AGREE II and 0.837 (95% CI: 0.792–0.873) in the RIGHT checklist, both of which indicate good reliability.

Characteristics of guidelines

The six guidelines were published in the United States (37,38,40), the United Kingdom (35,36), and Canada (39), with only the guideline developed by The University of Iowa (38), which were updated in 2016 after being published in 2012, and the guidelines developed by the Intensive Care Society (35), which were published in 2021 and are scheduled to be updated in 2024; none of the remaining guidelines have been updated or plan to update. Five guidelines (36-40) mentioned systematic literature retrieval, but only one (39) described the guideline development process and literature search strategies in detail. Three guidelines (37,39,40) formulated recommendations through expert consensus and used the evidence grading system, including Cochrane, SIGN, and self-defined system. The rest guidelines were not mentioned recommendation formulation methods and evidence grading system. One guideline (38) reported no relevant conflicts of interest for the developers, and two guidelines (39,40) reported funding sources. Specific information was shown in Table 1.

Quality of included guidelines

Methodological quality

The mean AGREE II score of the included guidelines were 41.92% with a range of 31.89–69.50%. No guideline was “high quality”. Only one guideline developed by the Registered Nurses’ Association of Ontario (RNAO) with a mean AGREE II score of 69.50% was “moderate quality”, and the remaining five guidelines were of “low quality”. The overall AGREE II scores (average of six domains) for each guideline were shown in Figure 2. “Clarity of Presentation” was the highest score (69.91%), and “Applicability” was the lowest score (21.53%). Each domain AGREE II scores were shown in Figure 3. And the scores of all guidelines in six domains were shown in Table 2.

Figure 2 AGREE II mean scores and RIGHT reporting rates of each guideline. AGREE II, Appraisal of Guidelines for Research and Evaluation II; RIGHT, Reporting Items for Practice Guidelines in Healthcare.

Figure 3 AGREE II mean scores of each domain for all included guidelines. AGREE II, Appraisal of Guidelines for Research and Evaluation II.

Reporting quality

The mean reporting rate for guidelines was 41.0% with a range of 24.7–77.7%. Only one guideline developed by RNAO was “moderate-reported” with a mean reporting rate of 77.7%, and the remaining five guidelines were all rated as “low-reported”. The overall reporting rates of six guidelines were shown in Figure 2. Among the seven domains, “Information” was the highest reporting rate (66.7%), while “Funding and conflict-of-interest statements and management” was the lowest reporting rate (16.7%). Among 10 key items, the reporting rates of item 14c “other consideration", item 18b “role of funder” and item 19b “management of conflict of interest” were 0%, with the highest reporting rates of item 12 “evidence quality assessment approach” at 50%. Each domain reporting rates were shown in Figure 4. And the reporting rates of all guidelines in seven domains were shown in Table 3.

Figure 4 RIGHT reporting rates of each domain for all included guidelines. RIGHT, Reporting Items for Practice Guidelines in Healthcare.

Discussion

CPGs, developed based on the best research evidence, provide clinicians with clear and comprehensive recommendations and are an important bridge between research evidence and clinical practice (41). AGREE II and RIGHT checklist focus on different aspects of quality, although some items overlap, in general can be a more comprehensive reflection of the quality of a guideline. It has been shown that AGREE II scores and RIGHT reporting rates have a high positive correlation, and writing CPGs based on the RIGHT checklist can improve AGREE II scores and thus improve the quality of guidelines (32,42). Therefore, this study provides suggestions for the future development of high-quality CPGs by evaluating the quality of physical restraints CPGs.

The overall methodological and reporting quality of the six guidelines included in this study was low. Only the guideline developed by RNAO in 2012 were rated as “moderate quality” in terms of methodological and reporting quality. The quality of CPGs affects clinical practice (41), so there is a need to develop high-quality CPGs in physical restraints to better guide clinical practice. There are two categories of CPGs: evidence-based CPGs (EB-CPGs) and non-EB-CPGs (43). Study found that the quality of EB-CPGs was significantly higher than the quality of non-EB-CPGs (26). If only low-quality evidence is available, EB-CPGs can be developed as well (43). Thus, in the future, high-quality physical restraints CPGs should be developed based on the best available evidence and in strict accordance with the methodology of evidence-based guidelines.

Applicability of recommendations in clinical practice is a key element of guideline translation (44). The overall mean score for “Applicability” of physical restraints CPGs in AGREE II was only 17.36%, with 4 guidelines (36-38,40) scoring <10%, which was serious neglect of guideline applicability. Studies have summarized facilitators of guideline application (44), including the provision of guideline implementation tools (e.g., executive summaries, brochures), presentation of guidelines in a short format, and presentation of guidelines in a digital format. It is recommended that future CPGs focus on the evidence as well as the application aspects. In particular, it should pay attention to analyzing what the advantages and disadvantages of applying the recommendations of this guideline, providing supporting tools, potential resource implications of applying the recommendations, and proposing monitoring and/or auditing criterion.

For both funding and conflict of interest statements, physical restraints CPGs scored low quality for both AGREE II and RIGHT checklist corresponding items. Three guidelines (35-37) did not report either financial support or information about conflict of interest. Conflicts of interest are one of the most important factors affecting the reliability of guidelines (24,45). Studies have shown that financial ties exist between guideline authors, panelists, and pharmaceutical companies (46). Therefore, guideline developers should guideline developers should regulate the management and reporting of funding and conflicts of interest, and researchers and administrators need to actively improve management policies and develop corresponding reporting specifications to promote physically binding CPGs toward greater objectivity, fairness, and transparency.

Opinions and preferences of the target population, procedures for updating the guideline, sources and evaluation of evidence were factors that affect the quality of the guidelines. The overall mean score for 14a “Describe whether values and preferences of the target population(s) were considered in the formulation of each recommendation” of physical restraints CPGs in RIGHT checklist was 33.3%. Few guidelines clearly describe how to consider patients’ perspectives and preferences, but clinical experience suggests that patients’ feelings and perceptions and close collaboration with health care professionals play a critical role in physical restraints practice. Thus, without knowledge of patient preferences, the implementation of the guidelines is definitely affected. Similar to the results of other guideline evaluations (47), descriptions of the updating procedures of the guidelines were poor, with only two guidelines (37,39) mentioning an update schedule. The WHO guideline handbook stated that although there is no maximum duration validity of recommendations, the minimum period for guideline updates is 2 years, and the maximum period is 5 years (48). Therefore, it is extremely important to update the guidelines in a timely manner. The mean score of “Rigor of development” in AGREE II (36.46%) and “Evidence” in RIGHT checklist (30.00%) were low, which reflect that the guidelines for “sources and evaluation of evidence” are not very good. Inclusion and exclusion criteria for evidence should be clear and strictly implemented, and formal tools or methods (e.g., Jadad scale, GRADE method) should be used to assess the strength of the evidence, clearly state the limitations of the evidence, balance the pros and cons of the available evidence, and give supporting data. A clear presentation and description of the evidence will help clinical staffs make good decisions based on a synthesis of the evidence when applying the recommendations. In addition, attention should be paid to the completeness of guideline reporting. For example, for process of systematic review, the entire process of literature search should be described in detail, and a complete search strategy should be provided in an appendix rather than just a few key words.

Limitation

This study provided a systematic literature retrieve to comprehensively explore the methodological and reporting quality of physical restraints CPGs and to make recommendations. Our findings provide clinical experts and methodologists with an overview of methodological and reporting quality of physical restraints guidelines, which may contribute to the development and updating of future guidelines and promote standardization of physical restraints practice. However, this study had several limitations. First, we only have two researchers involved in the quality appraisal process, which may have been problematic in terms of the accuracy of the results. Second, we only searched guidelines published in Chinese and English. So, we may not have included all of the guidelines.

Conclusions

In general, the methodological and reporting quality of physical restraints guidelines were low, and future development or updating of high-quality guidelines to guide clinical practice is needed. The domains of applicability, funding and conflict of interest statement, opinions and preferences of the target population, procedures for updating the guideline, sources and evaluation of evidence still need improvement. In the development of guidelines, more detailed and specific methodological descriptions are needed.

Acknowledgments

Funding: Chongqing Medical Scientific Research Project (Joint Project of Chongqing Health Commission and Science and Technology Bureau) (No. 2019ZDXM024).

Footnote

Reporting Checklist: The authors have completed the PRISMA reporting checklist. Available at https://apm.amegroups.com/article/view/10.21037/apm-21-2851/rc

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://apm.amegroups.com/article/view/10.21037/apm-21-2851/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Devlin JW, Skrobik Y, Gélinas C, et al. Clinical Practice Guidelines for the Prevention and Management of Pain, Agitation/Sedation, Delirium, Immobility, and Sleep Disruption in Adult Patients in the ICU. Crit Care Med 2018;46:e825-73. [Crossref] [PubMed]
2. Benbenbishty J, Adam S, Endacott R. Physical restraint use in intensive care units across Europe: the PRICE study. Intensive Crit Care Nurs 2010;26:241-5. [Crossref] [PubMed]
3. Teece A, Baker J, Smith H. Identifying determinants for the application of physical or chemical restraint in the management of psychomotor agitation on the critical care unit. J Clin Nurs 2020;29:5-19. [Crossref] [PubMed]
4. Souza LMDS, Santana RF, Capeletto CDSG, et al. Factors associated with mechanical restraint in the hospital environment: a cross-sectional study. Rev Esc Enferm USP 2019;53:e03473. [Crossref] [PubMed]
5. Kirk AP, McGlinsey A, Beckett A, et al. Restraint Reduction, Restraint Elimination, and Best Practice: Role of the Clinical Nurse Specialist in Patient Safety. Clin Nurse Spec 2015;29:321-8. [Crossref] [PubMed]
6. Pun BT, Balas MC, Barnes-Daly MA, et al. Caring for Critically Ill Patients with the ABCDEF Bundle: Results of the ICU Liberation Collaborative in Over 15,000 Adults. Crit Care Med 2019;47:3-14. [Crossref] [PubMed]
7. Zhu S, Jin Y, Xu Z, et al. The characteristics and nursing status of physical restraint use in ICU. Chin J Nurs 2009;44:1116-8.
8. Chen L, Xi X, Chen X. The current status and analysis of physical restraints usage in Intensive Care Unit. Chin Nurs Manag 2014;14:1022-4.
9. Hao W, Jiang Z, Zeng H, et al. Physical restraints characteristics of adult patients in ICU. Guangdong Med J 2016;37:2525-7.
10. Cui N, Gan X, Zhou S, et al. The investigation of body constraint related characteristic and influence factors for patients in intensive care unit. J Nurs Train 2018;33:483-6.
11. Perez D, Peters K, Wilkes L, et al. Physical restraints in intensive care-An integrative review. Aust Crit Care 2019;32:165-74. [Crossref] [PubMed]
12. Jasemi M, Valizadeh L, Zamanzadeh V, et al. A Concept Analysis of Holistic Care by Hybrid Model. Indian J Palliat Care 2017;23:71-80. [Crossref] [PubMed]
13. Pan Y, Jiang Z, Yuan C, et al. Influence of physical restraint on delirium of adult patients in ICU: A nested case-control study. J Clin Nurs 2018;27:1950-7. [Crossref] [PubMed]
14. Ertuğrul B, Özden D. The effect of physical restraint on neurovascular complications in intensive care units. Aust Crit Care 2020;33:30-8. [Crossref] [PubMed]
15. Perreault M, Talic J, Frenette AJ, et al. Agitation after mild to moderate traumatic brain injury in the intensive care unit. Crit Care 2017;21:219.
16. Crutchfield P, Gibb TS, Redinger MJ, et al. The Conditions for Ethical Application of Restraints. Chest 2019;155:617-25. [Crossref] [PubMed]
17. LeLaurin JH, Shorr RI. Preventing Falls in Hospitalized Patients: State of the Science. Clin Geriatr Med 2019;35:273-83. [Crossref] [PubMed]
18. Abraham J, Kupfer R, Behncke A, et al. Implementation of a multicomponent intervention to prevent physical restraints in nursing homes (IMPRINT): A pragmatic cluster randomized controlled trial. Int J Nurs Stud 2019;96:27-34. [Crossref] [PubMed]
19. Silva PS, Reis ME, Aguiar VE, et al. Unplanned extubation in the neonatal ICU: a systematic review, critical appraisal, and evidence-based recommendations. Respir Care 2013;58:1237-45. [Crossref] [PubMed]
20. Woolf SH, Grol R, Hutchinson A, et al. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ 1999;318:527-30. [Crossref] [PubMed]
21. Maiden MJ, Bone A, Fitzpatrick M, et al. Physical restraint of patients in Australia and New Zealand intensive care units. Intensive Care Med 2021;47:234-6. [Crossref] [PubMed]
22. Cui N, Qiu R, Zhang Y, et al. "Patient Comfort Can Be Sacrificed for Patient Safety"-Perception and Practice Reported by Critical Care Nurses Toward Physical Restraints: A Qualitative Descriptive Study. Front Med (Lausanne) 2021;8:573601. [Crossref] [PubMed]
23. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372: [PubMed]
24. Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Graham R, Mancher M, Miller Wolman D, Greenfield S, Steinberg E. editors. Clinical Practice Guidelines We Can Trust. Washington, DC, USA: National Academies Press, 2011.
25. Institute of Medicine (US) Committee to Advise the Public Health Service on Clinical Practice Guidelines, Field MJ, Lohr KN. editors. Clinical Practice Guidelines: Directions for a New Program. Washington, DC, USA: National Academies Press, 1990.
26. Zhang R, Yan S, Wang Y, et al. Analysis of the Status and Trends of Chinese Clinical Practice Guideline Development Between 2010 and 2020: A Systematic Review. Front Med (Lausanne) 2021;8:758617. [Crossref] [PubMed]
27. Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ 2010;182:E839-42. [Crossref] [PubMed]
28. Simancas-Racines D, Montero-Oleas N, Vernooij RWM, et al. Quality of clinical practice guidelines about red blood cell transfusion. J Evid Based Med 2019;12:113-24. [Crossref] [PubMed]
29. Ghanbari A, Rahmatpour P, Jafaraghaee F, et al. Quality assessment of diabetic foot ulcer clinical practice guidelines. J Evid Based Med 2018;11:200-7. [Crossref] [PubMed]
30. Wu D, Jiang W, Yu L, et al. Quality assessment of clinical practice guidelines for infectious diseases in China. J Evid Based Med 2018;11:95-100. [Crossref] [PubMed]
31. Li Q, Zhou Q, Xun Y, et al. Quality and consistency of clinical practice guidelines for treating children with COVID-19. Ann Transl Med 2021;9:633. [Crossref] [PubMed]
32. Maes-Carballo M, Mignini L, Martín-Díaz M, et al. Quality and reporting of clinical guidelines for breast cancer treatment: A systematic review. Breast 2020;53:201-11. [Crossref] [PubMed]
33. Chen Y, Yang K, Marušic A, et al. A Reporting Tool for Practice Guidelines in Health Care: The RIGHT Statement. Ann Intern Med 2017;166:128-32. [Crossref] [PubMed]
34. Lin I, Wiles LK, Waller R, et al. Poor overall quality of clinical practice guidelines for musculoskeletal pain: a systematic review. Br J Sports Med 2018;52:337-43. [Crossref] [PubMed]
35. Society IC. Guidance For: The Use of Physical Restraints in UK Intensive Care Units. Intensive Care Society 2021. Available online: https://www.ics.ac.uk/Society/Guidance/PDFs/Physical_Restraints_Guidance
36. Bray K, Hill K, Robson W, et al. British Association of Critical Care Nurses position statement on the use of restraint in adult critical care units. Nurs Crit Care 2004;9:199-212. [Crossref] [PubMed]
37. Maccioli GA, Dorman T, Brown BR, et al. Clinical practice guidelines for the maintenance of patient physical safety in the intensive care unit: use of restraining therapies--American College of Critical Care Medicine Task Force 2001-2002. Crit Care Med 2003;31:2665-76. [Crossref] [PubMed]
38. Lach HW, Leach KM, Butcher HK. Evidence-Based Practice Guideline: Changing the Practice of Physical Restraint Use in Acute Care. J Gerontol Nurs 2016;42:17-26. [Crossref] [PubMed]
39. Promoting safety: alternative approaches to the use of restraints. Registered Nurses' Association of Ontario (RNAO). Available online: http://rnao.ca/bpg/guidelines/promoting-safety-alternative-approaches-use-restraints
40. Bradas CM, Sandhu SK, Mion LC. Physical restraints and side rails in acute and critical care settings. In: Evidence-based geriatric nursing protocols for best practice. New York, NY, USA: Springer Publishing Company, 2012.
41. Lockwood C, Lizarondo L, Harrison MB, et al. Initial perceptions of, and intention to use, an online guideline adaptation framework: a descriptive survey. Int J Evid Based Healthc 2018;16:214-26. [Crossref] [PubMed]
42. Zhang S, Wu L, Wang Y, et al. Methodological and reporting quality of pediatric clinical practice guidelines: a systematic review. Ann Transl Med 2021;9:1258. [Crossref] [PubMed]
43. Djulbegovic B, Guyatt GH. Progress in evidence-based medicine: a quarter century on. Lancet 2017;390:415-23. [Crossref] [PubMed]
44. Chen Y, Shi Q, Zhao J, et al. From Knowledge to Action: Bridging the Gap Between Theory and Practice in Clinical Practice Guidelines. Medical Journal of Peking Union Medical College Hospital 2020;11:746-53.
45. Iannone P, Montano N, Minardi M, et al. Wrong guidelines: why and how often they occur. Evid Based Med 2017;22:1-3. [Crossref] [PubMed]
46. Shnier A, Lexchin J, Romero M, et al. Reporting of financial conflicts of interest in clinical practice guidelines: a case study analysis of guidelines from the Canadian Medical Association Infobase. BMC Health Serv Res 2016;16:383. [Crossref] [PubMed]
47. Li W, Li Y, Zhang X, et al. Evaluating nine clinical practice guidelines for vitamin D supplementation in children and adolescents with AGREE II: recommendations and controversies. Chin J Evid Based Pediatr 2012;07:372-9.
48. World Health Organization. WHO Handbook of Guideline Development. World Health Organization 2013. Available online: https://www.who.int/publications/i/item/9789241548960