The present study complies with Helsinki's Declaration and Local Bioethics Committee approved it. We followed the suggestions of a Panel of Experts to conduct adequate reporting.4

This CEA compared two different modalities of ASP. We used a previous cohort study that evaluated how Conventional or Bundled ASP differed in terms of mortality and antibiotic doses consumption.11

Conventional Strategy was defined as a simplified stewardship program, which included a clinical pharmacist screening for antimicrobial drug-related problem (ADRP), case discussions with infectious disease physicians (ID-MD) and telephone-based interventions.

On the other hand, the Bundled ASP had a more active design, which included: prospective auditing and local education/feedback about antimicrobial therapy prescription; microbiological data discussion with laboratory personnel to guide empirical or preemptive therapy; and face-to-face interventions to improve antimicrobial drug therapy.

The perspective of this study was a Southern Brazilian University Hospital, which is a 550-bed public and clinical reference institution with an average of 55–65% occupation rate. The Brazilian Health-System, namely Sistema Único de Saúde (SUS), is a primary care-centered system with universal access to all Brazilian citizens. More information regarding the aforementioned health system should be consulted in excellent reviews published elsewhere.12

All costs were collected and analyzed as local currency (R$, Brazilian Reais) and converted to United States Dollars (US$). Exchange values were collected at and they were expressed as mean value from February to September 2013, so R$ 1.00 was equivalent to US$ 0.47.

There were four relevant costs included in this CEA, namely: (I) hospital length of stay/patient-day, (II) cost of defined daily doses (DDD)/patient, (III) resources to provide microbiological and imaging diagnosis of infections, and (IV) human resources workload per day. These variables were collected through institutional databases, such as medication purchasing receipts and data from hospital's Human Resources and Planning Department. Table 1 summarizes all costs accounted for in this study and supplementary material provides detailed information.

Hospital length of stay was defined as the average cost per patient/day admitted to intensive care units or general wards, which included water consumption, human resources, medical material costs, and other relevant costs, except cost-related to ADT (see supplementary material).

DDD is a validated tool to standardize the number of doses consumed from each medication, allowing comparison of drug consumption between different health settings. Therefore, DDD was collected according to the original method developed by the World Health Organization.13 DDD was calculated based on pharmacy dispensation registries. Each unit of DDD was multiplied by the cost of drug, so antimicrobial therapy was expressed as “cost-DDD per patient”.13

Regarding the costs related to bacterial infections diagnosis, we defined all diagnostic criteria according to international guidelines.14–21 Prevalence of infections and their respective topographies were collected from the same previous cohort study,9 while costs of antibiograms, cultures, and other laboratory and imaging methods were calculated by means of microcosting bottom-up method.22

At last, cost of human workload per day was calculated by estimating the amount of time spent by each health care staff involved with ASP, whereby Bundled ASP accounted for a full-dedication clinical pharmacist resident and two partially dedicated ID physicians (one preceptor and one third-year post-graduate MD).

Transition probabilities among health states were obtained through effectiveness data.11 For both strategies, we compared 30-day mortality, which was expressed as either time-to-event data or point estimate value: Absolute Risk Rate (AR), Risk Difference (RD), and Number Needed to Treat (NNT) (Table 2).

Survival probabilities through each day (out of 30-day mortality) were also obtained from the same previous cohort.11

Data were analyzed by comparing both strategies in the base case, by reporting crude costs, clinical outcomes, and efficiency indicators such as: Cost-Effectiveness Rate (CER) and Incremental Cost-Effectiveness Ratio (ICER).

Thereafter, costs and effectiveness variables were modeled by using TreeAge Pro 2009 Suite Software (TreeAge Software Williamstown, MA, USA). A health-transition model was developed to simulate the probability of patients to make transitions between two health states (alive or death). We developed a Markov process in order to make the model sensitive to changes that might occur from one day to another. The aforementioned model consisted of 30 cycles aligned to 30-day mortality outcome. Each cycle-length corresponded to one day. Fig. 1 illustrates the model adopted in this study.

Fig. 1.

Markov process.

(0.07MB).

A deterministic one-way sensitivity analysis was conducted to identify whether variables range could critically influence the final ICER. Because binary variables cannot be assessed within this method, this sensitivity analysis did not include clinical outcomes, but only costs.

Multivariate analysis was performed by 2nd order Monte Carlo technique that simulated 10,000 probabilistic iterations. This analysis tested whether large ranges and their 95% confidence intervals could disturb the final ICER.

All cost variables were analyzed as Gama distributions and hyperparameters were calibrated to a 95% probability interval. Beta distributions were applied for effectiveness probabilities.

Finally, a tornado diagram illustrated one-way analysis, while Cost-Effectiveness Scatterplot, Acceptability Curve of Cost-Effectiveness and ICE Scatterplot showed the results from probabilistic sensitivity analysis. Discounting was not applied in the model since the horizon corresponded to one month; however, half-cycle correction was applied for all probabilistic iterations.23

In a preliminary analysis, the Bundled ASP was more expensive than a conventional ASP (Table 2) and the difference of crude costs was US$ 2119.70/patient. Considering point estimate values, which are expressed as Absolute Risk, Bundled ASP was more effective and patients are more likely to survive (73.1% vs 62.1%, p<0.05).

Nevertheless, by observing health economic indicators (CER) the base case suggests that Bundled Strategy was more cost-effective, which was also seen after 10,000 virtual simulations. Finally, ICER suggests that each averted death in 30 days costed US$ 19,287.54.

ICER did not change when confronting the aforementioned base case results against critical variables. Therefore, according to one-way deterministic analysis, previous ICER was robust (Fig. 2). We observed that length of stay cost, human resources, and antimicrobial consumption were the critical variables that had the greatest ICER range.

Fig. 2.

Tornado diagram with critical variables. Notes: Vertical line represents final ICER. Critical variables are represented as horizontal bars. All variables crossed final ICER, which means that none of them were able to disturb the final result from base case. c_obs-otim_intern, length of stay cost; c_otim_equipe, Bundled Strategy human resources cost; c_obs_medct, antimicrobial cost in Conventional Strategy; c_ c_otim_medct antimicrobial cost in Bundled Strategy; c_obs_dx_inf, bacterial infections diagnostic cost in Conventional Strategy; c_otim_dx_inf, bacterial infections diagnostic cost in Bundled Strategy.

(0.11MB).

Probabilistic sensitivity analysis showed that ICER was robust. Scatter Plot graph shows that Bundled Strategy stayed clinically superior and at similar costs compared to the other group (Fig. 3).

Fig. 3.

Scatterplot with 10,000 virtual cohorts. Note: Each dot represents one virtual cohort, thus, Bundled Strategy is likely to be more effective (y axis) and at similar costs (x axis).

(0.16MB).

Finally, Cost-Effectiveness Acceptability Curve (CEAC) (Fig. 4) was used to assess whether Bundled ASP would be affordable in the perspective of the studied health institution.

Fig. 4.

Cost-Effectiveness Acceptability Curve (CEAC) between Bundled and Conventional ASP.

Component	Quadrant	Incremental effectiveness	Incremental cost	ICER	# of dots	%
C1	IV	IE>0	IC<0	Superior	3414	34.14
C2	I	IE>0	IC>0		1851	18.51
C3	III	IE<0	IC<0	>ICER	351	3.51
C4	I	IE>0	IC>0	>ICER	4013	40.13
C5	III	IE<0	IC<0		222	2.22
C6	II	IE<0	IC>0	Inferior	149	1.49

Quadrants are read as “I” (upper right, the most effective and expensive), “II” (upper left, the less effective and the most expensive), “III” (lower left, the less effective and less expensive strategy) and “IV (lower right, the best scenario with higher probabilities of effectiveness at lower costs).

Diagonal line represents Willingness to Pay Function (WTP) and all virtual cohorts under this curve were considered affordable by the studied health institution.

“Component” represents a group of representative dots in CEAC. C1 and C2 represent that Bundled strategy is, respectively, dominant and under WTP function. C4 represents the probability (40%) that Bundled Strategy is more expensive and above WTP function. C3, C5 and C6 (less than 10% of all iterations) are less representative probabilities and account for less effective and more expensive percentages.

(0.13MB).

We observed that Bundled ASP has higher probability of being the most effective (C1+C2+C4=92.7%) and the most cost-effective strategy (C1+C2 vs C4=52.6% vs 40.1%).

When ICER was set as Willingness-To-Pay function (diagonal line represents stakeholders’ cutoff), Acceptability Curve demonstrated higher probabilities of Bundled Strategy stay under stakeholders’ willingness-to-pay. These results corroborated with previous base case and deterministic analysis.

Due to our close relation to Planning Department, it was possible to collect specific data and assemble them in our economic modeling. Microcosting bottom-up method required time and may not be feasible to other institutions, but still is a reliable way to collect costs.

Previous CEA that assessed the efficiency of one ASP9 used similar modeling and data analysis. The difference between this and past study is that previous publication included multiple setting clinical probabilities and costs. We also observed that some data derived from 1999, but manuscript was published in 2009.

Costs from other settings and inclusion of clinical outcomes from different institutions constitute the main drawback from previous publications. To avoid such bias, we believe that the present study has overcome these problems by considering expenditures and clinical data from a single institution.11

Different ASP interventions were never compared regarding their efficiency, and international literature lacks direct comparison between ASP strategies.

One could notice that Bundled ASP resembles other published and practiced stewardships with multiple types of interventions11 and, therefore, at this viewpoint, external validity is one of the positive aspects of our clinical effectiveness probabilities.

Instead of choosing a Decision Model Tree, which is commonly used to acute care conditions, Markov Model provides more accuracy for each of the 30-day probabilities. Markov Model could precisely estimate the probability of one patient to die or to survive each day: costs quantification is dependent on accumulative probabilities of surviving or dying. In other words, if one patient survives, there would be an accumulative cost in the next day; likewise, if one patient dies, it would probabilistically reduce cost-effectiveness ratio. Previous publications9 included a decision tree analysis, and due to the aforementioned reasons, they were poorly planned and could have methodological bias.

With respect to internal validity, patients should be clinically compared and primary site of infection may be insufficient to determine if groups are homogeneous.9 In a previous study, multivariate analysis determined that Charlson Comorbidity Index could independently predict mortality in patients assisted by ASP.11 The known association between Charlson Comorbidity Index and Mortality impacts previous study's list of assumptions9, where it was stated that “patients who never received an active antimicrobial drug therapy would have twice as poor outcomes (die twice) as those receiving an active therapy”. In other words, when defining an economic model, we should attempt to choose evidence-based probabilities of mortality to accurate predict real cost-effectiveness of ASP.

CEAC analysis was never performed before in other ASP research. It is a comprehensive way to help those who are not familiar with economic assessments to understand CEA. Nevertheless, to conduct such analysis, we set how much stakeholders would need to pay to have one Bundled ASP, by considering its structure, clinical and economic outcomes. Herein, we suggested ICER as the Willingness to Pay value (US 19,000 dollars per patient who survived 30-day mortality outcome).

ICER was determined as cut off value, because direct or indirect ways of defining “Willingness to Pay” are not validated and are subject to criticism. In addition, clinicians and stakeholders may interpret it more properly if NNT is also incorporated.

Therefore, by considering ICER value and that Bundled ASP has a NNT=9, we suggest the following interpretations:

• (a)

  for every nine patients treated, one will benefit (or achieve 30-day outcome);

• (b)

  for every US19,000 dollars invested, one will experience the benefit;

• (c)

  as NNT and ICER conceptions are similar, every nine patients treated will cost US19,000 dollars and one will have positive clinical outcome.

Our study is not free of limitations. The present study used effectiveness probabilities from a historical cohort, which means that retrospective data collection should always be cautiously interpreted due to incomplete registries, censoring, and research blinding problems.

Moreover, as two different ASP strategies were rarely compared in literature, more studies should be performed to assess if previous epidemiological findings (base case) are reproducible. When interpreting our results, one should consider the population included in the previous study, which was based on general wards and intensive care units inpatients.11 Other health settings should adapt this information before extrapolating our results to different practice scenarios.

Although sensitivity analysis corroborated with base case, cost-effectiveness should always consider one perspective. Readers should be aware of our study's external validity, especially because it was conducted in a Latin American health service, and differences are notorious between health systems and clinical settings.

Finally, although we believe that relevant costs were included in our economic modeling, ICER and Willingness-to Pay are not interchangeable concepts. In the present study, we did so because base case was a retrospective study, so all expenditures were already paid by hospital's financial department. Therefore, we considered that all values under Willingness-to Pay function would be affordable.