Reduction in antimicrobial use associated with a multifaceted antimicrobial stewardship programme in a tertiary teaching hospital in Shanghai: a segmented regression analysis

Introduction

The irrational use of antimicrobial agents causes antimicrobial resistance (AMR), which has become a global health problem, presenting a huge challenge for infectious disease treatment. The World Health Organization (WHO) has adopted ‘antimicrobial resistance’ as the theme of the World Health Day since 2011, and called governments for policy and action to halt the spread of drug resistance (1). In China, one of the world’s largest producers and consumers of antibiotics, the AMR burden has increased in recent years (2). AMR monitoring data from the China Antimicrobial Surveillance Network (CHINET) showed that cases of infections with carbapenem-resistant Klebsiella pneumoniae (CRKP) increased from 3.0% to 20.9% during 2005–2017 (3). With the increase of AMR and its adverse economic impact, there is an urgent need to take action to minimize the spread of AMR in China (2). Therefore, antimicrobial stewardship has become one of the most important programmes in hospital management (4).

In an attempt to fight against AMR, the National Health and Family Planning Commission (NHFPC, originally called the Ministry of Health) of China announced the ‘National Special Stewardship in the Clinical Use of Antibiotics’ in 2011, which issued standards for strict management of antibiotics and set goals for clinical antimicrobial use (4). In 2015, the NHFPC updated the Guidelines for the Clinical Use of Antibiotics, describing the details of all types of antibiotics and infectious diseases. All these special stewardship policies aimed to decrease AMR and the expenditure of public health systems resulting from the irrational use of antimicrobial agents. Since then, many hospitals in China have implemented antimicrobial stewardship (AMS) programmes, which comprise several management practices to facilitate the rational use of antibiotics. Consequently, most studies have shown a reduction in the length of hospital stays and antibiotic expenditure (5). However, the overall quality of AMS studies was low (6) and robust evidence on the impact of specific interventions was lacking (7). Most of these studies mainly concentrated on the impact of policies set by the general government (5-9). Although previous studies have focused on evaluating and improving hospital management through education (4,10-12), it was found that no management and control mode was generally accepted by hospitals. As a result, a good tool for complete and comprehensive antimicrobial stewardship (AMS) assessment was lacking (13,14).

To address these issues, our study attempted to implement multifaceted antimicrobial stewardship (MAMS) in a tertiary hospital in Shanghai in 2019. Segmented regression analysis of interrupted time series (ITS) was used to evaluate the effect of MAMS and standardise its model for large tertiary hospitals. We present the following article in accordance with the MDAR reporting checklist (available at https://dx.doi.org/10.21037/apm-21-700).

Methods

Study design and setting

This study was conducted in a 2,200-bed academic teaching hospital in Shanghai, China. In accordance with management requirements, our hospital established a series of administrative policies, in particular the quality premium, to improve the rational use of antimicrobial agents on 1 January, 2019. Therefore, we used an ITS segmented regression model to evaluate the impact of MAMS. The study duration (1 January 2017 to 31 December 2020) was divided into two periods as follows: period 1, baseline phase (1 January 2017 to 31 December 2018) and period 2, intervention phase (1 January 2019 to 31 December 2020). As antibiotic consumption and the percentage of antibiotic prescriptions were critical targets for the antimicrobial stewardship programme, data were extracted from the hospital information system (HIS) retrospectively and analysed for these two outcomes. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). No specific patients were involved in this study; thus, the ethical approval and consent were waived.

Intervention

A MAMS programme on antimicrobial use was conducted, which mainly involved quality premium with financial incentives, antibiotic restriction, audit and feedback, and education. All of the interventions were performed on January 2019 and lasted the whole intervention period and still in progress till this article published. A flowchart and detailed MAMS interventions are shown in Figure 1.

Figure 1 Standardized intervention model of MAMS. MAMS, multifaceted antimicrobial stewardship.

Organization construction

The Chinese Ministry of Health launched a long-term national antimicrobial stewardship programme to promote the appropriate use of antibiotics in 2011. Our hospital formed antimicrobial stewardship programme committees in response to the issue of the rational use of antibiotics raised by the government of China. These committees consisted of hospital administrators, infection prevention practitioners, pharmacists, clinical microbiologists, doctors, and information technology staff; the hospital also set up a leadership group including the president of the hospital. The aim was to ensure the implementation of the antimicrobial stewardship programme based on local conditions.

Antibiotic restriction

The antimicrobial stewardship programme committee formulated specific restrictions on antibiotic use according to national guidelines. Antibiotics were divided into three catalogues with different levels of restrictions (unrestricted antimicrobials, restricted antimicrobials, and special antimicrobials) according to antimicrobial resistance, safety, effectiveness, and price. Special antimicrobials, such as carbapenems, tigecycline, vancomycin, and echinocandin, are normally used for severely resistant bacterial infections. Restricted antibiotics are extensively applied, the most representative of this kind being moxifloxacin; however, strict administration control is required because of their increased abuse and resistance. Doctors with different professional titles were matched to the corresponding grade of antibiotic prescribing privileges.

Quality premium

Standardized measurable targets assessing antibiotic application were established according to the National Antibiotic Stewardship Program (15). The overall goals for antimicrobial prescription percentage (APP) and antibiotic consumption were in accordance with the recommendations of the NHFPC (16) (Table S1). In addition, the directors of the clinical departments were responsible for antibiotic application. Individualised goals for different departments were established according to historical antimicrobial use. The detailed rewards and penalty measures were as follows: (I) reduction in both APP and antibiotic consumption by 10% (or greater) from the prior month value; (II) reduction in the prescription of special antibiotics as a percentage of the total number of antibiotics by 10% (or greater) from the prior month value; (III) increase in the prescription of unrestricted antimicrobials as a percentage of the total number of antibiotics by 10% (or greater) from the prior month value. The clinical department would be rewarded or fined for achieving the goals or failure to achieve the goals, respectively. The performance score would be deducted 0.5–2 points depending on the increase of antibiotic consumption compared with prior month. Conversely, the performance score would be increased 1 point according to the growth of prescription of unrestricted antimicrobials. The performance score was associated with bonus. Detailed measurements for quality premium of antimicrobial stewardship were shown in Table S2.

Audit and feedback

Audit and feedback were used to confirm the proper use of antimicrobials. Pharmacists who majored in antibiotics with certificates issued by the Chinese Hospital Association were in charge of this programme. The rational use of perioperative prophylactic antibiotics and special antimicrobials was emphasized. The evaluations of the rationality of antibiotic prescriptions were issued according to local guidelines, which recommended appropriate indication, dosage, and duration of antibiotic use, especially special restricted antibiotics (15,16). A retrospective evaluation was performed monthly, and the antimicrobial stewardship programme committees provided regular verbal and written feedback to doctors, while inappropriate prescriptions would be shown in the hospital OA system. Doctors would analyse the underlying reasons for irrational antimicrobial prescriptions and make plans for modifying their prescription habits. Further assessment would be conducted to evaluate the improvement of clinical departments in antimicrobial prescription.

Education

A total of 18 clinical pharmacists were responsible for all the clinical wards. They were responsible for ward rounds, counselling, medication record audits and feedback, providing pharmaceutical care and antimicrobial education (lectures, online teaching, appraisals). Educational intervention was implemented to help physicians, surgeons, nurses, and pharmacists to understand and master the rational use of antibiotics. Meanwhile, the patients were divided into different groups and persuasive education including the indications, dosage, cautions and common adverse drug events of antibiotics was provided to them, if necessary.

Data collection and outcome measures

The primary outcomes were antibiotic consumption and the percentage of antibiotic prescriptions. These two targets were selected to assess rational antibiotic use according to the recommendations of the NHFPC (16). Total antibiotic use (TAU), based on antibiotic consumption, was measured by defined daily doses (DDDs) per 100 patient-days (PD) according to the WHO Collaborative Centre for Drug Statistics Methodology recommendation (15). The DDD was retrieved from the Anatomical, Therapeutic, and Chemical Classification System (ATC) index with DDDs recommended by WHO, indicating the mean sustained dose per day of an agent during its use in adults as the main indication. The DDDs of antibiotics were evaluated using the formula: antibiotic usage amount × pack size/DDD (17). The percentage of antibiotic prescriptions = (number of inpatients receiving antibiotics/number of inpatients during the same period) ×100. These data were collected directly from HIS.

Statistical analysis

ITS analysis was used to evaluate the effects of the interventions. The segmented regression model for each indicator was fitted as follows: Y = β₀ + β₁ × time + β₂ × intervention + β₃ × time × intervention + β₄₁ × covid19_1 + β₄₂ × covid19_2.

Y represents the monthly antibiotic consumption or the percentage of antibiotic prescription, β₀ represents the baseline level at time =0, β₁ is the baseline trend before interventions, β₂ is the level change following the interventions, β₃ is the trend change following the interventions, β₄₁ is the level change during the COVID-19 period, and β₄₂ is the level change post-COVID-19. Time is the number of months elapsed since the start of the study (January 2017). Intervention is a dummy variable for before (intervention =0) and after (intervention =1) interventions. COVID-19 is a dummy variable indicating the pre-COVID-19 period (January 2017 to December 2019, coded 0), COVID-19 period (January 2020 to March 2020, coded 1), and post-COVID-19 (coded 2). We used harmonic terms specifying two sine and cosine pairs to adjust for seasonality. After controlling seasonality, residual autocorrelation was assessed by examining the plot of residuals and the partial autocorrelation function. Comparisons of the monthly values of the measures for antibiotic use were conducted using ANOVA or t-test. Box plots were used for the data visualisation. All statistical analyses were performed using R software (version 3.6.1; The R Foundation, Vienna, Austria). Differences were considered statistically significant when P<0.05.

Results

Change trends of antibiotic use in the period of 2017–2020

Monthly changes in antibiotic consumption and APP during the baseline phase (January 2017 to December 2018) and the intervention phase (January 2019 to December 2020) are shown in Figure 2, spanning 48 calendar months. The highest antibiotic consumption (77.0 DDDs/100 PD) was recorded in February 2018 and the lowest (37.4 DDDs/100 PD) in August 2020. The highest APP (47.2%) was recorded in December 2017 and the lowest (41.2%) in October 2020.

Figure 2 Monthly change of antibiotic consumption (A) and percentage of antibiotic prescriptions (B) pre- and post-intervention. DDD, defined daily doses.

Comparison of outcomes before and after intervention

The total values for antibiotic consumption and APP were 55.9 (43.6, 63.1) DDDs/100 PD and 44.2% (43.2%, 45.1%), respectively. Comparison (Table 1 and Figure 3) revealed that after MAMS intervention, antibiotic consumption decreased significantly from 63.1 (59.3, 67.2) DDDs/100 PD to 43.3 (39.0, 49.8) DDDs/100 PD (P<0.001) and APP decreased significantly from 44.8% (44.1%, 45.4%) to 43.3% (42.2%, 44.3%) (P<0.001).

Table 1 Comparison of antibiotic indicators before and after intervention
Full table

Figure 3 Comparisons of antibiotic consumption (A) and percentage of antibiotic prescriptions (B) pre- and post-intervention. ***, P<0.001; ****, P<0.0001. DDD, defined daily doses.

Comparison of outcomes in different phases considering coefficient of COVID-19

COVID-19 was first identified in early January 2020 in the Chinese city of Wuhan, and it rapidly spread across the country and subsequently, the world, emerging into a global health threat (18). As the Chinese government’s responses focused on traditional public health outbreak response tactics (19), the spread of COVID-19 slowed down at the end of March. Based on the above information, we divided the study into three phases: pre-COVID-19 (January 2017 to December 2019), COVID-19 (January 2020 to March 2020), and post-COVID-19 (April 2020 to Dec 2020). We compared the consumption and APP in the three phases. During the pre-COVID-19 phase, the antibiotic consumption and APP were 59.1 (54.4, 64.8) DDDs/100 PD and 44.3% (43.4%, 45.2%), respectively; during the COVID-19 phase, 44.0 (43.3, 44.9) DDDs/100 PD and 43.8% (43.7%, 44.8%), respectively; and during the post-COVID-19 phase, 38.2 (37.5, 39.0) DDDs/100 PD and 44.2% (43.2%, 45.1%), respectively. The results showed antibiotic consumption (P<0.001) and percentage of antibiotic prescriptions (P=0.026) were significantly different among pre-COVID-19 phase, COVID-19 phase, and the post-COVID-19 phase (Table 2). Antibiotic consumption during pre-COVID-19 phase was significantly more than antibiotic consumptions in COVID-19 phase and the post-COVID-19 phase (P<0.01 and P<0.0001, respectively), while antibiotic consumption in COVID-19 phase was notably higher than antibiotic consumption in post-COVID-19 phase (P<0.01). Similarly, there was a downward trend in percentage of antibiotic prescriptions during pre-COVID-19 phase, COVID-19 phase and the post-COVID-19 phase, and percentage of antibiotic prescriptions in pre-COVID-19 phase was significantly lower than percentage of antibiotic prescriptions in post-COVID-19 phase (P<0.01, Figure 4).

Table 2 Comparison of antibiotic indicators at different stages of COVID-19
Full table

Figure 4 Comparisons of antibiotic consumption (A) and percentage of antibiotic prescriptions (B) among pre-COVID-19, COVID-19, and post-COVID-19. **, P<0.01; ****, P<0.0001. DDD, defined daily doses.

Results of the segmented regression analysis

ITS analysis indicated an upward trend for monthly antibiotic consumption (0.12 DDDs/100 PD, P=0.378) before MAMS intervention and a significant downward trend (−12.54 DDDs/100 PD, P<0.001) after the first month of MAMS intervention (Table 3, Figure 5A). In the time series analysis, APP demonstrated no significant difference before (0.05% per month, P=0.155) and after interventions (−0.98% per month, P=0.226) (Table 4, Figure 5B). However, a significant change in the slope before and after intervention was observed (−0.17% per month, P=0.049) in APP (Table 4, Figure 5B). No significant change was observed in either antibiotic consumption or APP during the COVID-19 period.

Table 3 Interrupted time series analysis of intensity of antibiotic consumption
Full table

Figure 5 Interrupted time series for antibiotic consumption (A) and percentage of antibiotics prescription (B). Harmonic terms specifying two sine and cosine pairs were used to adjust for seasonality based on regression model. Lines were predicted trends and black dots were observed antibiotic consumption or proportion of antibiotics prescription. DDD, defined daily doses.

Table 4 Interrupted time series analysis of the proportion of antibiotics prescription
Full table

Discussion

MAMS was initiated on 1 January 2019. After two years of antibiotic management interventions, there was a significant overall reduction in antibiotic consumption [from 63.1 (59.3, 67.2) DDDs/100 PD to 43.3 (39.0, 49.8) DDDs/100 PD] and the APP [from 44.8% (44.1%, 45.4%) to 43.3% (42.2%, 44.3%)]. This apparent decline in both indicators was due to MAMS. Segmented regression analysis in our study found an increasing slope in the two indicators before interventions, and a downward slope, after the intervention. This is similar to findings from previous studies that antimicrobial restriction strategies can reduce the use of antibiotics, although most studies were not related to the quality premium (20). Our study suggests that there are potential long-term effects of interventions implemented by multidisciplinary hospitals with a quality premium.

The MAMS in our study contained many interventions, among which the quality premium played an important role. Quality premium, which attempts to motivate individuals to reach the target of the organisation, has been extensively used for hospital management (21). The crucial part of the quality premium in the AMS is a financial reward or penalty issued by healthcare commissioners to improve the quality of antimicrobial use based on objective or subjective elements (22). AMS with financial initiatives has shown positive effects in previous studies (20,22-24). Clinical commissioning groups (CCGs) in England suggested that AMS with financial incentives reduced the total number of antibiotics prescribed and the percentage of broad-spectrum antibiotics used from 2015 to 2016 (25). A study in the USA investigated two pay-for-performance programmes in healthcare quality, and the results showed that there was a 6% improvement in the rate of rational prescription of antibiotics (24). A Swedish study showed that the pay-for-performance of antibiotic prescription can encourage narrow-spectrum antibiotic use in children with respiratory tract infections (23). However, other studies suggest that the quality premium in AMS is neither easy to enact nor necessarily productive (20,23). The quality premium in our study have been doubted to increase in dissatisfaction, lack of motivation especially on some clinical departments with higher infectious patient admission rate. In order to address the resistance and refusal, the MAMS programme strengthen communication with clinic and make prompt adjustments to the measurements of quality premium according to physician’s suggestions. Moreover, the quality premium model differed from study to study, and the quality premium was always used in conjunction with other important interventions, such as policy restrictions.

In addition to the quality premium, the implementation of MAMS in our study also included regular management strategies such as including clinical pharmacists and epidemiologists in multidisciplinary consultation, prescription comments, and education on antibiotic use. Long-term educational interviews for prescribers were reported to improve the rational use of antibiotics, resulting in the reduction of infections caused by β-lactamase (ESBL)-producing E. coli (11). Therefore, one-to-one educational interviews may provide a new method for MAMS.

The spread of COVID-19 was effectively under control after 3 months within China (18). Flu-like symptoms such as fever, headache, and cough were most common in moderately infected patients, and the rapid progression of severe lung injury has led to emergency treatment in severe cases, which made it difficult to analyse whether viral and bacterial co-infection was present or not in the first few days (26). We were concerned that the sudden outbreak of COVID-19 would confuse the choice of antimicrobial therapy to be administered and lead to an increase in antibacterial agents used; however, our study showed that there was no significant difference in both antibiotic consumption and APP during the COVID-19 period compared with those in the pre- and post-COVID-19 periods. This may be because patients with COVID-19 received treatment and quarantine at medical institutions for infectious diseases in China, and our hospital was not a specific infectious disease hospital.

One strength of the study is the ITS analysis, which is a valuable design for evaluating the effectiveness of interventions over a clearly defined time period and targeted population-level health outcomes. Segmented regression analysis, a defined impact model, is typically used to analyse the ITS study. Our study had several limitations. First, our study was a retrospective observational study and lacked a simultaneous control group. Furthermore, our study was conducted only in a tertiary hospital in Shanghai, which could not represent the general situation in other places in China. However, we provide a practical case to illustrate that antibiotic abuse could be effectively controlled by multiple administrative measures in this study. Finally, although our study provided strong evidence regarding the association between interventions and reduced antibiotic use, the potential impact of other policies could not be completely excluded.

Conclusions

Our study showed the potent positive effect of MAMS on decreasing antibiotic use in a tertiary teaching hospital. The MAMS consisted of strategies such as the quality premium and regular interventions, which led to a significant reduction in antibiotic consumption and APP, and this effect was not influenced by COVID-19. Thereby, we suggest that future hospital management and antimicrobial stewardship programmes should include proper interventions to promote antibiotic prescription quality.

Acknowledgments

We sincerely thank Dr. Jayashree for the language editing of manuscript.

Funding: This work was supported by the Artificial Intelligence Assisted Drug Research and Development Projects of Shanghai Health and Health Development Research Center (No.29), Policy Research Project of Health and Family Planning Commission of Minhang District (2017MWZ14), Hospital Management Research Fund of Shanghai Hospital Association (X2020033), the Research Funds of Shanghai Health and Family Planning commission (20184Y0022, 20194Y0007), Shanghai “Rising Stars of Medical Talent” Youth Development Program - Youth Medical Talents - Clinical Pharmacist Program [SHWJRS (2019)_072], WU JIEPING Medical Foundation (320.6750.2020-04-31), and China International Medical Foundation (Z-2018-35-2003).

Footnote

Reporting Checklist: The authors have completed the MDAR reporting checklist. Available at https://dx.doi.org/10.21037/apm-21-700

Data Sharing Statement: Available at https://dx.doi.org/10.21037/apm-21-700

Peer Review File: Available at https://dx.doi.org/10.21037/apm-21-700

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://dx.doi.org/10.21037/apm-21-700). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). No specific patients were involved in this study; thus, the ethical approval and consent were waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Du Y, Li J, Wang X, et al. Impact of a Multifaceted Pharmacist-Led Intervention on Antimicrobial Stewardship in a Gastroenterology Ward: A Segmented Regression Analysis. Front Pharmacol 2020;11:442. [Crossref] [PubMed]
2. Zhang J, Qian X, Zhang L, et al. Evaluation of the Effectiveness of Clinical Pharmacists' Consultation in the Treatment of Infectious Diseases: A Single-Arm, Prospective Cohort Study. Front Pharmacol 2019;10:187. [Crossref] [PubMed]
3. Hu F, Zhu D, Wang F, et al. Current Status and Trends of Antibacterial Resistance in China. Clin Infect Dis 2018;67:S128-34. [Crossref] [PubMed]
4. Wang H, Wang H, Yu X, et al. Impact of antimicrobial stewardship managed by clinical pharmacists on antibiotic use and drug resistance in a Chinese hospital, 2010-2016: a retrospective observational study. BMJ Open 2019;9:e026072 [Crossref] [PubMed]
5. Nathwani D, Varghese D, Stephens J. Value of hospital antimicrobial stewardship programs [ASPs]: a systematic review. Antimicrob Resist Infect Control 2019;8:35. [Crossref] [PubMed]
6. Schweitzer VA, van Heijl I, van Werkhoven CH, et al. The quality of studies evaluating antimicrobial stewardship interventions: a systematic review. Clin Microbiol Infect 2019;25:555-61. [Crossref] [PubMed]
7. Rocha-Pereira N, Figueiredo Dias P, Correia S, et al. Persuasive antimicrobial stewardship intervention in the context of a KPC outbreak: a controlled interrupted time series analysis. Antimicrob Resist Infect Control 2020;9:55. [Crossref] [PubMed]
8. Howard P, Pulcini C, Levy Hara G, et al. An international cross-sectional survey of antimicrobial stewardship programmes in hospitals. J Antimicrob Chemother 2015;70:1245-55. [PubMed]
9. Charani E, Smith I, Skodvin B, et al. Investigating the cultural and contextual determinants of antimicrobial stewardship programmes across low-, middle- and high-income countries—A qualitative study. PLoS One 2019;14:e0209847 [Crossref] [PubMed]
10. Palavecino EL, Williamson JC, Ohl CA. Collaborative Antimicrobial Stewardship: Working with Microbiology. Infect Dis Clin North Am 2020;34:51-65. [Crossref] [PubMed]
11. Peñalva G, Fernández-Urrusuno R, Turmo JM, et al. Long-term impact of an educational antimicrobial stewardship programme in primary care on infections caused by extended-spectrum β-lactamase-producing Escherichia coli in the community: an interrupted time-series analysis. Lancet Infect Dis 2020;20:199-207. [Crossref] [PubMed]
12. Wang N, Athans V, Neuner E, et al. A pharmacist-driven antimicrobial stewardship intervention targeting cytomegalovirus viremia in ambulatory solid organ transplant recipients. Transpl Infect Dis 2018;20:e12991 [Crossref] [PubMed]
13. Chen H, Liu C, Liu J, et al. Mixed effects of restriction strategies in antimicrobial stewardship programs on antimicrobial use in 121 tertiary hospitals in China, 2013-2017. Expert Rev Pharmacoecon Outcomes Res 2019;19:483-9. [Crossref] [PubMed]
14. Dunn K, O’Reilly A, Silke B, et al. Implementing a pharmacist-led sequential antimicrobial therapy strategy: a controlled before-and-after study. International Journal of Clinical Pharmacy 2011;33:208-14. [Crossref] [PubMed]
15. National Health and Family Planning Commission of the People’s Republic of China. Available online: https://www.gov.cn/xinwen/2015-08/27/content_2920789.htm
16. NHFPC. National Health and Family Planning Commission of the People’s Republic of China. Notice regarding National Special Measure Scheme on Clinical Use of Antibiotics. 2011. Available online: https://www.gov.cn/gzdt/2012-03/06/content_2084862.htm
17. The anatomical therapeutic chemical classification system with defined daily doses (ATC/DDD). Available online: https://www.who.int/classifications/atcddd/en/
18. Pascarella G, Strumia A, Piliego C, et al. COVID-19 diagnosis and management: a comprehensive review. J Intern Med 2020;288:192-206. [Crossref] [PubMed]
19. Wu Z, McGoogan JM. Characteristics of and Important Lessons From the Coronavirus Disease 2019 (COVID-19) Outbreak in China: Summary of a Report of 72314 Cases From the Chinese Center for Disease Control and Prevention. JAMA 2020;323:1239-42. [Crossref] [PubMed]
20. Yip W, Powell-Jackson T, Chen W, et al. Capitation combined with pay-for-performance improves antibiotic prescribing practices in rural China. Health Aff (Millwood) 2014;33:502-10. [Crossref] [PubMed]
21. Giangreco A, Carugati A, Sebastiano A, et al. War outside, ceasefire inside: An analysis of the performance appraisal system of a public hospital in a zone of conflict. Eval Program Plann 2012;35:161-70. [Crossref] [PubMed]
22. Balinskaite V, Johnson AP, Holmes A, et al. The Impact of a National Antimicrobial Stewardship Program on Antibiotic Prescribing in Primary Care: An Interrupted Time Series Analysis. Clin Infect Dis 2019;69:227-32. [Crossref] [PubMed]
23. Ellegård LM, Dietrichson J, Anell A. Can pay-for-performance to primary care providers stimulate appropriate use of antibiotics? Health Econ 2018;27:e39-54. [Crossref] [PubMed]
24. van der Velden AW, Pijpers EJ, Kuyvenhoven MM, et al. Effectiveness of physician-targeted interventions to improve antibiotic use for respiratory tract infections. Br J Gen Pract 2012;62:e801-7. [Crossref] [PubMed]
25. Quality Premium: 2015/16 guidance for CCGs. NHS England. Available online: https://www.england.nhs.uk/wp-content/uploads/2013/12/qual-prem-guid.pdf
26. Serafim RB, Povoa P, Souza-Dantas V, et al. Clinical course and outcomes of critically ill patients with COVID-19 infection: a systematic review. Clin Microbiol Infect 2021;27:47-54. [Crossref] [PubMed]