Coronavirus disease (COVID 2019): protocol for a living overview of systematic reviews
Original Article

Coronavirus disease (COVID 2019): protocol for a living overview of systematic reviews

Cuncun Lu1,2,3, Tingting Lu1,2,3, Bei Pan4, Qi Wang2,5, Liangying Hou2,3, Qian Zhang6, Yunhua Wang2, Yihui Wang2, Xiuxia Li2,3,5, Yue Ruan6, Lian Chen6, Honghao Lai2, Tianzhu Qin2, Long Ge2,3,5, Kehu Yang1,3,5

1Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China;2Department of Social Science and Health Management, School of Public Health, Lanzhou University, Lanzhou, China;3Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou, China;4Gansu Provincial Hospital, Lanzhou, China;5Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou, China;6Evidence-Based Nursing Center, School of Nursing, Lanzhou University, Lanzhou, China

Contributions: (I) Conception and design: C Lu, L Ge, K Yang; (II) Administrative support: X Li; (III) Provision of study materials or patients: Q Wang, B Pan, L Ge, K Yang, T Lu, L Hou, YuH Wang, Y Ruan, L Chen, C Lu; (IV) Collection and assembly of data: B Pan, L Hou, YiH Wang, H Lai, T Qin, Q Wang, T Lu, Q Zhang, L Ge, C Lu; (V) Data analysis and interpretation: C Lu, L Ge, K Yang, T Lu; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

Correspondence to: Long Ge, Kehu Yang. Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou, China. Email: gelong2009@163.com; kehuyangebm2006@126.com.

Background: The coronavirus disease 2019 (COVID-19) pandemic continues to grow worldwide, and systematic reviews (SRs)/meta-analyses (MAs) on COVID-19 can efficiently guide evidence-based clinical practice. However, SRs/MAs with weaknesses can mislead clinical practice and pose harm to patients, and too many useless SRs/MAs could pose confusion and waste sources. A “living” overview of SRs/MAs aims to provide an open, accessible and frequently updated resource summarizing the highest-level evidence of COVID-19, that can help evidence-users to quickly identify trusted evidence to guide the practice. This study aims to systematically give an overview SRs/MAs of COVID-19, assess their quality, and identify the best synthesis of evidence.

Methods: Databases including Medline, EMBASE, Web of Science, China National Knowledge Infrastructure (CNKI), China Biology Medicine (CBM) and WanFang were systematically searched on May 1, 2020 using relevant terms for identify SRs/MAs related to COVID-19. The study selection, data extraction and quality assessment will be performed by independent reviewers, and results will be cross-checked. The authoritative tools (AMSTAR-2, PRISMA and its extensions) will be used to assess the methodological quality and reporting quality of included SRs/MAs, and potential influence factors will be explored. The consistency of conclusions will be compared among reviews and the best evidence will be summarized. In addition, we will conduct exploratory meta-analyses (MAs) of individual studies when applicable. Data will be reported as number with (or) percentage, risk ratio (RR) or odds ratio (OR), mean difference (MD) or standardized mean difference (SMD) with 95% confidence interval (CI) according to the specific results. R3.6.1 and Microsoft Excel 2016 will be used to analyze and manage data.

Results: The results of this overview will be submitted to a peer-reviewed journal for publication.

Conclusions: In this study, we will present for the first time, an overview of SRs/MAs, which provides a comprehensive, dynamic evidence landscape on prevalence, prevention, diagnosis, treatment, and prognosis of COVID-19.

Keywords: COVID-19; living overview; systematic review (SR); meta-analysis; evidence quality; AMSTAR-2; PRISMA

Submitted May 18, 2020. Accepted for publication Jul 10, 2020.

doi: 10.21037/apm-20-1130

Introduction

At the end 2019, the novel coronavirus disease 2019 (COVID-19) that could spread from person to person was firstly identified and reported in the world (1). In addition to severe acute respiratory syndrome coronavirus (SARS) and middle east respiratory syndrome coronavirus (MERS), COVID-19 is the third coronavirus that is severely harmful to human health (2). On February 1, 2020, the World Health Organization (WHO) declared COVID-19 as a public health emergency of international concern that presently has spread around the world (3). According to WHO report published at May 1, 2020, the cumulative number of confirmed cases and deaths worldwide reached 3145407 and 221823, respectively (https://covid19.who.int/).

After the COVID-19 outbreak, the number of research evidence of COVID-19 is rapidly increasing (4-11), includes basic research, epidemiology research, clinical studies, narrative reviews, bibliometric studies, systematic reviews (SRs)/MAs, guidelines or recommendations, etc. The topics of these publications cover but are not limited to the underlying mechanism of action, prevention, diagnosis, treatment and prognosis (4). However, in several studies, concerns have been expressed on quality, reliability and normalization of COVID-19 studies (12-14). Evidence based medicine (EBM) emphasizes that all clinical decisions should be based on the best evidence currently available (15). In addition, SR/MA as a key tool in EBM, that through summarizing multiple individual studies on same or similar clinical question including etiology, prevention, diagnosis, treatment and prognosis topics, provide guidance for EBM clinical practice (16). However, poor methodological guidance, selection report and duplications will pose significant confusion, and may even mislead practice, thus sources waste will be companied (16-18). Recently, many SRs/MAs on COVID-19 have been published (9,10,19-21), however, the quality of the studies remains unknown. Moreover, these SRs/MAs may have posed confusion for relevant stakeholders in applying these evidences. An overview could summarize multiple SRs/MAs into a single usable document and provide a comprehensive ‘friendly front end’ of the evidence for clinicians, decision-makers, and patients (22-24). Evidence users be able to quickly capture and understand the evidence by reading the overview, and do not have to spend much time to assimilate data from separate SRs/MAs (24).

By increasing of SRs/MAs on COVID-19 and substantially increased the clinical workload, it has been a challenge for clinicians to update their knowledge on COVID-19 for evidence-based practice. Therefore, we plan this “living” overview (which will be updated regularly) to summarize evidence of SRs/MAs on COVID-19 to guide clinical practice, assess their quality, and identify current gaps in evidence to inform future studies.

Methods

In this study, we reported this protocol by referencing the Preferred Reporting Items for Systematic Review and Meta-analysis Protocols (PRISMA-P) guidelines (25). Considering the urgency of the study, we have not yet registered this in a public website, but if some modifications were to occur, any changes as well as the reasoning will be reported in the main text.

Eligibility criteria

We will include all types of SRs/MAs (such as qualitative SR, traditional MA, network MA) on COVID-19 that have been published in the Chinese or English language, and they should conform the following definition (26): “A review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant studies, and collect and analyze data from the studies included in the review (qualitative SR), or the statistical combination of at least two studies to produce a single estimate (such as traditional MA or network MA)”. The clinical topics could involve all aspects of COVID-19, including prevalence, prevention, diagnosis, treatment and prognosis. We will exclude preclinical studies, narrative reviews, overviews, guidelines, and other publications that do not present key information, such as protocols, letters, abstracts, newsletters, etc.

Search method

A systematically search in Medline (Ovid), EMBASE (Ovid), Web of Science, China National Knowledge Infrastructure (CNKI), China Biology Medicine (CBM) and WanFang databases was performed on May 1, 2020. Main search terms included “coronavirus”, “COVID-19”, “2019-nCoV”, “SARS-CoV-2”, “novel coronavirus”, “WN-CoV”, “systematic”, “systematic review”, “systematic literature review”, “meta-analysis”, “meta analyses”, and “evidence synthesis”, etc. The search was performed by QW and BP with the guidance of two senior researchers (LG, KHY) (27). The initial database search yielded 447 records, and this search will be updated monthly. The detailed search strategy is shown in Appendix 1. To identify any potentially eligible studies, the references list of included SRs/MAs will be manual checked. We did not search preprints websites, such as medRxiv (https://www.medrxiv.org/), which has the following statement: “Preprints are preliminary reports of work that have not been certified by peer review. They should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.”

Study selection

All identified records will be imported into Endnote X9 software (Thomson Reuters, NY, USA). After removing duplications, teams of three reviewers (TTL, LYH, YuHW, YR, LC, CCL) will independently screen titles and abstracts according to the eligibility criteria. Potential studies and conflicted studies will be subjected to full-text review. Any disagreement or conflict will be solved through discussion, and in any case the conflict exists, a senior reviewer (LG) will be making an arbitrary decision.

Data extraction

For data extraction, teams of two reviewers (BP, LYH, YuHW, TZQ, HHL, QW, TTL, QZ) will independently extract key information from included SRs/MAs. This information includes the following: title, first author, publication date or online date, language, journal and impact factor, number of authors, country of the corresponding author or the last author, number of patients (participants) and their details (age, gender, clinical symptoms, co-morbidities, diagnostic methods, interventions, risk factors, and disease severity, etc.), clinical topics, data sources, number of primary studies, type of primary studies, assessment tools for the quality of primary studies, synthesis methods, outcomes, and their effect sizes, and key findings or conclusions, etc. Any conflict will be resolved by consensus.

Quality assessment

The tools, Assessment of Multiple Systematic Reviews 2 (AMSTAR-2) (28) and Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) (29) and its extensions (such as PRISMA-DTA for diagnostic MA, PRISMA-NMA for network MA) (30,31), will be used to evaluate the methodological quality and reporting quality of included SRs/MAs, respectively. AMSTAR-2 consists of 16 items, among which seven (item 2, 4, 7, 9, 11, 13 and 15) are considered critical items. The overall confidence on quality of SRs/MAs will be classified into four categories: (I) high quality: no or only one non-critical weakness, (II) moderate quality: >1 non-critical weakness, (III) low quality: only one critical weakness, with or without non-critical weakness, and (IV) critically low quality: > one critical weakness, with or without non-critical weakness. However, considering that AMSTAR-2 was developed for interventional SRs/MAs (28), modifications will be conducted when necessary. The PRISMA consists of 27 items, and we will assess the compliance of SRs/MAs against each item using “Yes”, “Partial”, “No”, or “Not applicable”. In addition, the aforementioned four options for its extensions are appropriate. Prior to the assessment, all investigators will discuss the items of the tools, and all reviewers will use 5‒10 SRs/MAs to test the consistency until a consensus is reached. Then, teams of two reviewers (BP, LYH, YiHW, HHL, TZQ, QW, TTL, QZ) will assess the quality of included SRs/MAs. Any conflict will be resolved by discussion or by adjudication of a senior reviewer (LG, or CCL). In addition, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) will be used to assess evidence quality based on each of the outcomes of MAs that focus on randomized controlled trials (32), when applicable.

Data analysis

In this study, qualitative description and quantitative synthetization will be used. The results of quality assessment will be reported as number with (or) percentage. We will explore the differences of the quality of included SRs/MAs by subgroup analysis based on the type of reviews, country, journals’ impact factor, and clinical topic, etc., and the results will be reported as the risk ratio (RR) with 95% confidence interval (CI). In addition, based on AMSTAR-2 assessment, we will categorize reviews into “Trusted evidence” (high and moderate quality) and “Questionable evidence” (low and critically low quality). For “Trusted evidence”, we will summarize the conclusion of each review in tabular format and will stratify according to the type of SR. Where MA is performed within a review, we will also extract forest plot and pooled effect sizes. Where no quantitative pooling of effect sizes is reported, we will use standardized language indicating the direction of the effect and statistical significance. For continuous outcomes, we will summarize data using the mean difference (MD) or standardized mean difference (SMD) with 95% CI as reported in the included reviews. For dichotomous outcomes, we will present the RR or odds ratio (OR) or absolute effect and 95% CI as appropriate. For “questionable evidence”, we will try to reconduct an exploratory MA or indicate the necessary of future research. For all analyses, significance level will be set at 0.05 (two-sided test). R 3.6.1 (R Foundation for Statistical Computing, Vienna, Austria) and Microsoft Excel 2016 (Microsoft, Richmond, WA, USA) will be used to analyze and manage the data obtained.

Discussion

SRs/MAs as high-level evidence source for supporting clinical practice, their methodology and reporting can influence their applicability in clinical practice (16,17). At present, many SRs/MAs related to COVID-19 have been published, however, their quality remains unknown. Moreover, many reviews may have posed confusion on relevant stakeholders. To fill this gap, the present study was designed to present comprehensive evidence from SRs/MAs, and the results of this study will be updated regularly. The study will employ AMSTAR-2, PRISMA and its extensions to assess the quality of included SRs/MAs, and data will be re-synthesized when applicable. Furthermore, because the number of SRs/MAs is growing, we will keep updating our results and this involves a dynamic process that we named “living overview”. We believe this living overview will promote the use of SRs/MAs evidence to improve clinical service quality while reducing research waste (18).

Strengths and limitations

This research has several strengths and limitations. Firstly, this is the first living overview of SRs/MAs related to COVID-19. This study will provide a comprehensive and dynamic evidence landscape on prevalence, prevention, diagnosis, treatment and prognosis of COVID-19. Secondly, the validated tools, AMSTAR-2, PRISMA and its extensions will be used to judge the quality of included SRs/MAs, and data will be re-synthesized when applicable. Regarding limitations, we will only include SRs/MAs published in the English and Chinese language, therefore, language bias may be an important limitation.

Presenting and reporting of results

The results of this study will be reported by referencing the PRISMA statement (29). A flow diagram will be used to show the process of the screening and selection of the eligible SRs/MAs. Finally, this work will be submitted to a peer-reviewed journal for publication.

Acknowledgments

Funding: This study was supported by a special funding for prevention and control of emergency of COVID-19 from Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province (No. GSEBMKT-2020YJ01), and by the Fundamental Research Funds for the Central Universities (lzujbky-2020-sp14).

Footnote

Peer Review File: Available at http://dx.doi.org/10.21037/apm-20-1130

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/apm-20-1130). The authors have no conflicts of interest to declare.

Ethical Statement: All authors are accountable for all aspects of this work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All data used to support the present research will be from published publications, and there will be no any identifiable patient information. No ethical approve or informed consent is required for the purposes of the present study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

  1. Li Q, Guan X, Wu P, et al. Early Transmission Dynamics in Wuhan, China, of Novel Coronavirus-Infected Pneumonia. N Engl J Med 2020;382:1199-207. [Crossref] [PubMed]
  2. Peeri NC, Shrestha N, Rahman MS, et al. The SARS, MERS and novel coronavirus (COVID-19) epidemics, the newest and biggest global health threats: what lessons have we learned? Int J Epidemiol 2020;49:717-26. [Crossref] [PubMed]
  3. Guan WJ, Ni ZY, Hu Y, et al. Clinical Characteristics of Coronavirus Disease 2019 in China. N Engl J Med 2020;382:1708-20. [Crossref] [PubMed]
  4. Lv M, Luo X, Estill J, et al. Coronavirus disease (COVID-19): a scoping review. Euro Surveill 2020;25:2000125. [Crossref] [PubMed]
  5. Walls AC, Park YJ, Tortorici MA, et al. Structure, Function, and Antigenicity of the SARS-CoV-2 Spike Glycoprotein. Cell 2020;181:281-92.e6. [Crossref] [PubMed]
  6. Bi Q, Wu Y, Mei S, et al. Epidemiology and transmission of COVID-19 in 391 cases and 1286 of their close contacts in Shenzhen, China: a retrospective cohort study. Lancet Infect Dis 2020;20:911-9. [Crossref] [PubMed]
  7. Sanders JM, Monogue ML, Jodlowski TZ, et al. Pharmacologic Treatments for Coronavirus Disease 2019 (COVID-19): A Review. JAMA 2020;323:1824-36. [PubMed]
  8. Tao Z, Zhou S, Yao R, et al. COVID-19 will stimulate a new coronavirus research breakthrough: a 20-year bibliometric analysis. Ann Transl Med 2020;8:528. [Crossref] [PubMed]
  9. Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ 2020;369:m1328. [Crossref] [PubMed]
  10. Rodriguez-Morales AJ, Cardona-Ospina JA, Gutiérrez-Ocampo E, et al. Clinical, laboratory and imaging features of COVID-19: A systematic review and meta-analysis. Travel Med Infect Dis 2020;34:101623. [Crossref] [PubMed]
  11. Alhazzani W, Møller MH, Arabi YM, et al. Surviving Sepsis Campaign: guidelines on the management of critically ill adults with Coronavirus Disease 2019 (COVID-19). Intensive Care Med 2020;46:854-87. [Crossref] [PubMed]
  12. Ioannidis JPA. Coronavirus disease 2019: the harms of exaggerated information and non-evidence-based measures. Eur J Clin Invest 2020. [Epub ahead of print]. [Crossref]
  13. Wolkewitz M, Puljak L. Methodological challenges of analysing COVID-19 data during the pandemic. BMC Med Res Methodol 2020;20:81. [Crossref] [PubMed]
  14. London AJ, Kimmelman J. Against pandemic research exceptionalism. Science 2020;368:476-7. [Crossref] [PubMed]
  15. Lu C, Li X, Yang K. Trends in Shared Decision-Making Studies From 2009 to 2018: A Bibliometric Analysis. Front Public Health 2019;7:384. [Crossref] [PubMed]
  16. Ioannidis JP. The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses. Milbank Q 2016;94:485-514. [Crossref] [PubMed]
  17. Glasziou P, Meats E, Heneghan C, et al. What is missing from descriptions of treatment in trials and reviews? BMJ 2008;336:1472-4. [Crossref] [PubMed]
  18. Siontis KC, Ioannidis JPA. Replication, Duplication, and Waste in a Quarter Million Systematic Reviews and Meta-Analyses. Circ Cardiovasc Qual Outcomes 2018;11:e005212. [Crossref] [PubMed]
  19. Cortegiani A, Ingoglia G, Ippolito M, et al. A systematic review on the efficacy and safety of chloroquine for the treatment of COVID-19. J Crit Care 2020;57:279-83. [Crossref] [PubMed]
  20. Yang J, Zheng Y, Gou X, et al. Prevalence of comorbidities and its effects in patients infected with SARS-CoV-2: a systematic review and meta-analysis. Int J Infect Dis 2020;94:91-5. [Crossref] [PubMed]
  21. Bao C, Liu X, Zhang H, et al. Coronavirus Disease 2019 (COVID-19) CT Findings: A Systematic Review and Meta-analysis. J Am Coll Radiol 2020;17:701-9. [Crossref] [PubMed]
  22. Xiu-xia L, Ya Z, Yao-long C, et al. The reporting characteristics and methodological quality of Cochrane reviews about health policy research. Health Policy 2015;119:503-10. [Crossref] [PubMed]
  23. Ge L, Tian JH, Li YN, et al. Association between prospective registration and overall reporting and methodological quality of systematic reviews: a meta-epidemiological study. J Clin Epidemiol 2018;93:45-55. [Crossref] [PubMed]
  24. Hunt H, Pollock A, Campbell P, et al. An introduction to overviews of reviews: planning a relevant research question and objective for an overview. Syst Rev 2018;7:39. [Crossref] [PubMed]
  25. Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;350:g7647. [Crossref] [PubMed]
  26. Chapman SJ, Drake TM, Bolton WS, et al. Longitudinal analysis of reporting and quality of systematic reviews in high-impact surgical journals. Br J Surg 2017;104:198-204. [Crossref] [PubMed]
  27. Li L, Tian J, Tian H, et al. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol 2014;67:1001-7. [Crossref] [PubMed]
  28. Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017;358:j4008. [Crossref] [PubMed]
  29. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009;339:b2700. [Crossref] [PubMed]
  30. McInnes MDF, Moher D, Thombs BD, et al. Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: The PRISMA-DTA Statement. JAMA 2018;319:388-96. [Crossref] [PubMed]
  31. Hutton B, Salanti G, Caldwell DM, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med 2015;162:777-84. [Crossref] [PubMed]
  32. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336:924-6. [Crossref] [PubMed]
Cite this article as: Lu C, Lu T, Pan B, Wang Q, Hou L, Zhang Q, Wang Y, Wang Y, Li X, Ruan Y, Chen L, Lai H, Qin T, Ge L, Yang K. Coronavirus disease (COVID 2019): protocol for a living overview of systematic reviews. Ann Palliat Med 2021;10(2):1488-1493. doi: 10.21037/apm-20-1130

Article Options

Download Citation

Share