Quality and fragility of meta-analyses assessing the efficacy and safety of Ginkgo biloba preparation: protocol for a methodological study

Introduction

Ginkgo biloba L. (GB) is an ancient plant with several parts (e.g., leaves and fruits) containing multiple active substances (e.g., biflavonoids and ginkgolides) with high medicinal value (1,2). According to the “Chinese Pharmacopoeia 2020 edition” (3), dried ginkgo seeds and leaves both have the functions of astringing the lung and relieving dyspnea; while seeds can improve leukorrhea and reduce polyuria, and leaves can activate blood and resolve stasis, dredge collaterals, relieve pain, eliminate turbidity, and lower the blood lipid level. Relevant research (2) indicated that the health effects of ginkgo are primarily associated with metabolites such as flavonoids and terpenoids. Ginkgolides belong to terpenoid derivatives and are active ingredients in GB leaves. Based on the differences in molecular structure of the active ingredients, there are several different ginkgolides in GB (4). Evidence showed that biflavonoids and ginkgolides have many biological activities such as anti-inflammatory, anti-allergic, antioxidant, anti-cancer, and neuroprotective activities (2,5). The extracts from GB have been converted to drugs, such as ginkgo lactone injection, ginkgolide injection, and ginkgo leaf dropping pills (4). These are now widely used for treating numerous clinical conditions, such as angina pectoris (6), ischemic stroke (4,7), and dementia (8,9).

With the rapid popularization of evidence-based medicine, systematic reviews and meta-analyses play a critical role in medical decision-making and guiding future research (10). Numerous meta-analyses (4,6-8,11) evaluating the clinical efficacy and safety of GB preparations have been published in recent years. For instance, in 2022, Zhao et al. (4) published a meta-analysis focusing on the clinical effects of ginkgo terpene lactone preparations against ischemic stroke. The quality of conduct and reporting of meta-analyses has been recognized to affect the reliability and clinical usability of pooled results (10,12,13). However, the methodological and reporting quality of the meta-analyses on GB preparations has not been comprehensively assessed to date. Therefore, in the present study, the methodological and reporting quality of eligible meta-analyses regarding the effects of GB preparations for clinical conditions will be assessed. The “A Measurement Tool to Assess Systematic Reviews 2” (AMSTAR-2) tool (14) and the “Preferred Reporting Items for Systematic reviews and Meta-Analyses” (PRISMA 2020) statement (15) will be used, respectively. The correlation between the methodological and reporting quality, as well as the factors affecting the methodological quality will be assessed. Moreover, the fragility index (FI) will be utilized to evaluate the robustness of statistically significant binary outcomes (16,17). We present the following article in accordance with the PRISMA-P reporting checklist (available at https://apm.amegroups.com/article/view/10.21037/apm-22-795/rc) (18).

Methods

Study design

The methodological study will investigate the quality of conduct and reporting of meta-analyses on GB preparations. The study has been registered on the INPLASY website (registration number: INPLASY202260092). The details and reasons for any changes will be reported in the final full text for deviations from this protocol. Ethical approval and informed consent were not required because this study was an overview of published literature.

Search strategy

An extensive literature search will be performed in PubMed, Embase, CNKI, WanFang, and the Chinese Biomedical Literature Database from the inception to June 2022. In this study, Medical Subject Heading terms and keywords will be used to establish retrieval strategies. The primary search terms included will be “ginkgo biloba”, “gingko”, “ginkgo biloba extract”, “systematic reviews as topic”, “systematic review”, “meta-analysis as topic”, and “meta-analysis”. No other language or publication type restrictions will be imposed. In addition, the reference list of selected meta-analyses will be manually checked to retrieve potentially eligible studies. The preliminary search strategies are presented in detail in Appendix 1.

Study selection

Studies meeting the following criteria will be included: (I) type of study: meta-analyses of GB preparations and published in either English or Chinese peer-reviewed journals; a meta-analysis is defined as in our previous studies (10,12); (II) subjects: humans suffering from any disease (e.g., angina pectoris, ischemic stroke, and dementia), and treated using any GB preparations; (III) interventions: a control group treated with placebo or conventional therapy, and an interventional group treated with any GB preparations (e.g., ginkgolide injection, ginkgo leaf dropping pills, etc.) alone or in combination with the same treatments as the control group; (IV) outcome: any outcome regarding efficacy and safety will be considered. We will exclude duplicate publications or inaccessible full texts, as well as protocols, narratives, or qualitative systematic reviews, letters, conference abstracts, network meta-analyses, and individual participant data meta-analyses.

The literature obtained through database search will be imported into EndNote (X9, Clarivate Analytics) software for screening. To exclude irrelevant records, two independent reviewers will conduct the literature selection by reading titles and abstracts. The full text of the remaining papers will be assessed to determine the final eligibility. Any disagreement will be resolved by consulting a third reviewer.

Quality assessment

Two trained reviewers will use AMSTAR-2 and PRISMA 2020 to evaluate the methodological and reporting quality of included meta-analyses. Any conflict will be resolved through discussion. AMSTAR-2 consists of 16 items, where items 2, 4, 7, 9, 11, 13, and 15 are critical items. AMSTAR-2 provides three answers to the question of the items for a meta-analysis, including “Yes”, “Partial Yes” and “No.” Based on existing methodological weaknesses in each meta-analysis, the overall methodological quality can be evaluated as high, moderate, low, or critically low.

PRISMA 2020, an updated version of PRISMA 2009, is a reporting checklist for systematic reviews and meta-analyses (19). It consists of seven sections with a total of 42 sub-items. PRISMA was initially developed to improve the reporting of systematic reviews and meta-analyses (19). However, it has been widely employed to evaluate the reporting quality of published systematic reviews or meta-analyses. In the current study, “Yes” and “No” will be used to assess whether the included meta-analyses satisfy the reporting requirements for each item. In order to facilitate the statistical analysis, 1, 0.5, and 0 scores will be assigned to “Yes”, “Partial Yes” and “No”, respectively, in AMSTAR-2 and without “Partial Yes” in PRISMA 2020.

Data extraction

Two independent investigators will extract data from each meta-analysis using a predesigned data extraction form. Any disagreement will be discussed and solved by consensus. Extracted data will include the following parameters: title, first author, number of authors, disease, language, year of publication, registration information, country, PRISMA tool used, journal name, journal impact factor, number of pages, intervention and control description, database searched, searching timeframe, total sample size, study design of original studies, number of original studies, quality assessment tool, and funding information.

Statistical analysis

Compliance rate of AMSTAR-2 or PRISMA2020 will be determined through the number and percentage of “Yes”, “Partial Yes”, or “No.” Moreover, a total quality score for each meta-analysis will be calculated. The relationship between methodological and reporting quality will be assessed using the Spearman correlation coefficient based on the total score of each meta-analysis. In addition, univariate and multivariate linear analyses will be utilized to assess whether the five predefined factors (i.e., year of publication, number of authors, number of pages, registration, and PRISMA used) would affect the methodological quality. The quality assessment results presented with a bubble chart. This chart will consist of two axes and some bubbles. The x-axis will be divided into four sections representing the results of AMSTAR-2 (“high”, “moderate”, “low”, and “critically low”) for each article. The y-axis will describe the total score based on PRISMA 2020 for each meta-analysis. Each bubble will represent a meta-analysis article, the size of the bubbles will reflect the total number of participants included in each meta-analysis, and the color of the bubbles will represent the year of publication. Moreover, the FI of each statistically significant binary outcome will be utilized to assess the robustness of the pooled data, where a larger FI represents a more robust estimate (20). Data analysis will be performed with Stata 16.0 (StataCorp, College Station, TX, USA) and Excel 2016 (Microsoft Corporation, WA, USA) software. Two-sided P<0.05 will be considered statistically significant.

Discussion

GB is a plant with high medicinal value and various pharmacological activities, including anti-inflammatory, anti-allergic, antioxidant, and neuroprotective activities. GB preparations have been used against cardiovascular and nervous system disorders (21), and other conditions, such as symptoms of influenza and COVID-19 infection (22,23). Generally, meta-analyses are regarded as the most substantial in the hierarchy pyramid of evidence (24), but the quality of meta-analyses in certain clinical areas is suboptimal (25-28). Many meta-analyses of GB preparations have been conducted to assess the efficacy and safety of GB preparations under various conditions. The methodological and reporting quality of these meta-analyses has not yet been thoroughly evaluated and reported. Therefore, the present study aspires to fill this gap.

To the best of our knowledge, this review will be the first to assess the methodological and reporting quality of meta-analyses concerning GB preparations using AMSTAR-2 and PRISMA 2020. In addition, the correlation between methodological and reporting quality will be identified, and the potential factors affecting the methodological quality will be explored. Further, the robustness of pooled results for each binary outcome will be assessed via the FI, which is defined as the minimum number of patients with an event-status modification that can change a statistically significant result to a nonsignificant pooled effect (20). However, this study will have certain limitations. First, the scores assigned to answers may not indicate the actual quality. Therefore, the number will be provided with a percentage for each answer to help readers fully understand the quality assessment results. Second, the judgment regarding the research quality will only be based on the included publications and supplementary materials (29,30). Differences could emerge between what was reported and what was conducted by the authors; therefore, the results of methodological quality may be biased. In summary, the final results of this methodological study of meta-analyses on GB preparations will help to improve the quality of relevant future studies and clinical decision-making.

Acknowledgments

The authors would like to thank MogoEdit (https://www.mogoedit.com) for its English editing during the preparation of this manuscript.

Funding: The present study was supported by CACMS Innovation Fund (No. CI2021A00702).

Footnote

Reporting Checklist: The authors have completed the PRISMA-P reporting checklist. Available at https://apm.amegroups.com/article/view/10.21037/apm-22-795/rc

Peer Review File: Available at https://apm.amegroups.com/article/view/10.21037/apm-22-795/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://apm.amegroups.com/article/view/10.21037/apm-22-795/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study has been registered on the INPLASY website (registration number: INPLASY202260092). Ethical approval and informed consent were not required because this study was an overview of published literature.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Singh B, Kaur P. Biology and chemistry of Ginkgo biloba. Fitoterapia 2008;79:401-18. [Crossref] [PubMed]
2. Šamec D, Karalija E, Dahija S, et al. Biflavonoids: Important Contributions to the Health Benefits of Ginkgo (Ginkgo biloba L.). Plants (Basel) 2022;11:1381. [Crossref] [PubMed]
3. Lan F, Hong XX, Song ZH, et al. Overview of the Chinese Pharmacopoeia 2020 Edition. Drug Standards of China 2020;21:185-8.
4. Zhao H, Guo Q, Li B, et al. The Efficacy and Safety of Ginkgo Terpene Lactone Preparations in the Treatment of Ischemic Stroke: A Systematic Review and Meta-Analysis of Randomized Clinical Trials. Front Pharmacol 2022;13:821937. [Crossref] [PubMed]
5. Gachowska M, Szlasa W, Saczko J, et al. Neuroregulatory role of ginkgolides. Mol Biol Rep 2021;48:5689-97. [Crossref] [PubMed]
6. Tan D, Wu JR, Cui YY, et al. Ginkgo Leaf Extract and Dipyridamole Injection as Adjuvant Treatment for Angina Pectoris: A Meta-Analysis of 41 Randomized Controlled Trials. Chin J Integr Med 2018;24:930-7. [Crossref] [PubMed]
7. Chong PZ, Ng HY, Tai JT, et al. Efficacy and Safety of Ginkgo biloba in Patients with Acute Ischemic Stroke: A Systematic Review and Meta-Analysis. Am J Chin Med 2020;48:513-34. [Crossref] [PubMed]
8. Liao Z, Cheng L, Li X, et al. Meta-analysis of Ginkgo biloba Preparation for the Treatment of Alzheimer's Disease. Clin Neuropharmacol 2020;43:93-9. [Crossref] [PubMed]
9. Yuan Q, Wang CW, Shi J, et al. Effects of Ginkgo biloba on dementia: An overview of systematic reviews. J Ethnopharmacol 2017;195:1-9. [Crossref] [PubMed]
10. Lu C, Ke L, Li J, et al. Saffron (Crocus sativus L.) and health outcomes: a meta-research review of meta-analyses and an evidence mapping study. Phytomedicine 2021;91:153699. [Crossref] [PubMed]
11. Tabrizi R, Nowrouzi-Sohrabi P, Hessami K, et al. Effects of Ginkgo biloba intake on cardiometabolic parameters in patients with type 2 diabetes mellitus: A systematic review and meta-analysis of clinical trials. Phytother Res 2020; Epub ahead of print. [Crossref] [PubMed]
12. Chen J, Wang H, Lu X, et al. Safety and efficacy of stem cell therapy: an overview protocol on published meta-analyses and evidence mapping. Ann Transl Med 2021;9:270. [Crossref] [PubMed]
13. Hoffmann TC, Oxman AD, Ioannidis JP, et al. Enhancing the usability of systematic reviews by improving the consideration and description of interventions. BMJ 2017;358:j2998. [Crossref] [PubMed]
14. Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017;358:j4008. [Crossref] [PubMed]
15. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372: [PubMed]
16. Atal I, Porcher R, Boutron I, et al. The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses. J Clin Epidemiol 2019;111:32-40. [Crossref] [PubMed]
17. Baer BR, Gaudino M, Charlson M, et al. Fragility indices for only sufficiently likely modifications. Proc Natl Acad Sci U S A 2021;118:e2105254118. [Crossref] [PubMed]
18. Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;350:g7647. [Crossref] [PubMed]
19. Page MJ, Moher D, McKenzie JE. Introduction to preferred reporting items for systematic reviews and meta-analyses 2020 and implications for research synthesis methodologists. Res Synth Methods 2022;13:156-63. [Crossref] [PubMed]
20. Linden A. Computing the fragility index for randomized trials and meta-analyses using Stata. The Stata Journal 2022;22:77-88. [Crossref]
21. Wu DD, Qu C, Liu XQ, et al. A simple high performance liquid chromatography method for the assay of flavonoids in Ginkgo biloba leaves. World J Tradit Chin Med 2021;7:47-53. [Crossref]
22. Ibrahim MA, Ramadan HH, Mohammed RN. Evidence that Ginkgo Biloba could use in the influenza and coronavirus COVID-19 infections. J Basic Clin Physiol Pharmacol 2021;32:131-43. [Crossref] [PubMed]
23. Nawrot J, Gornowicz-Porowska J, Budzianowski J, et al. Medicinal Herbs in the Relief of Neurological, Cardiovascular, and Respiratory Symptoms after COVID-19 Infection A Literature Review. Cells 2022;11:1897. [Crossref] [PubMed]
24. Brunström M, Thomopoulos C, Carlberg B, et al. Methodological Aspects of Meta-Analyses Assessing the Effect of Blood Pressure-Lowering Treatment on Clinical Outcomes. Hypertension 2022;79:491-504. [Crossref] [PubMed]
25. Lu C, Ke L, Li J, et al. Chinese Medicine as an Adjunctive Treatment for Gastric Cancer: Methodological Investigation of meta-Analyses and Evidence Map. Front Pharmacol 2022;12:797753. [Crossref] [PubMed]
26. Storman M, Storman D, Jasinska KW, et al. The quality of systematic reviews/meta-analyses published in the field of bariatrics: A cross-sectional systematic survey using AMSTAR 2 and ROBIS. Obes Rev 2020;21:e12994. [Crossref] [PubMed]
27. Lu TT, Lu CC, Li MX, et al. Reporting and methodological quality of meta-analyses of acupuncture for patients with migraine: A methodological investigation with evidence map. J Integr Med 2022;20:213-20. [Crossref] [PubMed]
28. Xu C, Furuya-Kanamori L, Kwong JSW, et al. Methodological issues of systematic reviews and meta-analyses in the field of sleep medicine: A meta-epidemiological study. Sleep Med Rev 2021;57:101434. [Crossref] [PubMed]
29. Baek Y, Ademi Z, Paudel S, et al. Economic Evaluations of Child Nutrition Interventions in Low- and Middle-Income Countries: Systematic Review and Quality Appraisal. Adv Nutr 2022;13:282-317. [Crossref] [PubMed]
30. Lu T, Lu C, Li H, et al. The reporting quality and risk of bias of randomized controlled trials of acupuncture for migraine: Methodological study based on STRICTA and RoB 2.0. Complement Ther Med 2020;52:102433. [Crossref] [PubMed]