Abstract

Objectives

To determine the existence of concentration–toxicity relationships for common β-lactam antibiotic adverse effects and define thresholds above which toxicity is more likely.

Patients and methods

Retrospective review of consecutive patients treated with piperacillin, meropenem or flucloxacillin who underwent therapeutic drug monitoring (TDM) at St Vincent’s Hospital (Sydney, Australia) between January 2013 and December 2015. Adverse events investigated included neurotoxicity, nephrotoxicity, hepatotoxicity and opportunistic Clostridium difficile infection. Toxicity was measured using observational grading criteria, clinical assessment and relevant serum biomarkers. These findings were correlated with trough TDM measurements at the time of toxicity presentation.

Results

TDM results from 378 patients (piperacillin = 223, meropenem = 94 and flucloxacillin = 61) were investigated. There was no difference in baseline patient characteristics across antibiotic groups. A statistically significant elevation in mean serum trough concentrations (Cmin) was found in patients diagnosed with neurotoxicity (piperacillin, P <0.01; meropenem, P =0.04; flucloxacillin, P =0.01) and those who developed nephrotoxicity whilst being treated with piperacillin (P <0.01) or meropenem (P <0.01). Incidence of hepatotoxicity and C. difficile was not related to Cmin. Threshold concentrations for which there is 50% risk of developing a neurotoxicity event (piperacillin, Cmin >361.4 mg/L; meropenem, Cmin >64.2 mg/L; flucloxacillin, Cmin >125.1 mg/L) or nephrotoxicity (piperacillin, Cmin >452.65 mg/L; meropenem, Cmin >44.45 mg/L) varied across antibiotics.

Conclusions

Our data reveal an association between toxic concentrations for a number of β-lactam agents and neurotoxic/nephrotoxic effects. We have defined threshold concentrations above which these toxicities become more likely. Clinicians should balance concerns for therapeutic efficacy with potential toxicity when considering aggressive therapy.

Introduction

The dosing regimens for β-lactams are predominantly determined from pharmacokinetic (PK) data obtained from healthy individuals. Consequently, these standardized regimens fail to accommodate for the profoundly altered pharmacokinetics amongst particular patient groups such as burns victims, patients with febrile neutropenia or septic shock, the obese, and those with renal impairment or enhanced renal clearance.1 These patients demonstrate significant variability in protein binding, volume of distribution and drug clearance, resulting in unpredictable and fluctuating drug concentrations.2

The potential for β-lactams to precipitate antibiotic-induced toxicity is increasingly apparent, but likely remains underestimated in clinical practice.3 Toxicity may manifest in the form of neurological deterioration, renal complications or hepatic injury. Of these, only neurotoxicity has been previously correlated with increasing β-lactam serum concentrations.4–6 These adverse events could conceivably not be attributed to β-lactams by treating physicians due to an insidious onset and the complex overall clinical picture of hospitalized patients.7 Such antibiotic-induced organ dysfunction could potentially increase morbidity and mortality, whilst increasing the cost of diagnostic tests and ongoing healthcare.

The β-lactam therapeutic drug monitoring (TDM) programme at our institution was introduced in 2013 to optimize inpatient β-lactam prescriptions. Although TDM was readily available for all patients, the programme mainly targeted critically ill patients and those with difficult infections. However, the decision to utilize the available services was at the discretion of the treating physician.

Following the identification of several cases of concentration-related neurotoxicity, we postulated that other ‘idiosyncratic’ adverse effects of β-lactam antibiotics might in fact be concentration dependent. We systematically reviewed our data for the commonly prescribed intravenous penicillin antimicrobial agents piperacillin and flucloxacillin, as well as a carbapenem class drug, meropenem, to determine whether this was the case and whether we could identify threshold concentrations above which toxicity became more likely.

Patients and methods

Ethics

This study protocol was reviewed and approved by the St Vincent’s Hospital Sydney Human Research Ethics Committee (Reference Number: LNR/15/SVH/297, SSA: LNRSSA/15/SVH/314).

Study selection and data source

A retrospective cohort review of consecutive patients who underwent piperacillin, meropenem and flucloxacillin TDM at St Vincent’s Hospital (Sydney, Australia) was conducted. TDM results were extracted from the laboratory information system and matched to electronic patient records to set up an initial database. The investigated antibiotics were administered in the form of an intermittent bolus or infusion. The hospital’s standard dosing schedule was 8–16 g/24 h at 6–12 h intervals for piperacillin, 1–6 g/24 h at 8–12 h intervals for meropenem and 4–12 g/24 h at 4–6 h intervals for flucloxacillin. The choice of dose and method of administration were made by the treating physician. Trough concentrations (Cmin) were defined as those collected <2 h prior to the subsequent dose. In the event of multiple Cmin being available on a single patient, the highest concentration was used. This database was subsequently used to review selected medical records to ascertain the presence of the following adverse events: (i) neurotoxicity, (ii) nephrotoxicity, (iii) hepatotoxicity and (iv) Clostridium difficile infection (CDI).

Selection criteria

Inclusion

For patient data to be considered for analysis the following criteria needed to be satisfied:

  • The patient must have been treated with intravenous piperacillin, meropenem or flucloxacillin at St Vincent’s Hospital between January 2013 and December 2015.

  • At least one TDM assay was collected at Cmin levels as previously defined.

  • The patient did not meet any exclusion criteria as summarized below.

Exclusion

The exclusion criteria were developed specifically for each individual toxicity parameter considered and exclusion from one parameter did not preclude the use of the same patient in evaluating other toxicity outcomes (Table 1). The need to exclude data was a recognition that the proper diagnosis of β-lactam-induced toxicity in certain subjects would be unreliable due to their overall clinical status.

Table 1.

Patient exclusion criteria

ToxicityExclusion criteria
Neurotoxicity
  • Premorbid neurological impairment (e.g. severe dementia).

  • Extracorporeal membrane oxygenation at baseline or TDM.

  • Alternative clinical diagnosis (e.g. mechanical ventilation and copious sedatives rendering cognitive status undeterminable).

Nephrotoxicity
  • Chronic kidney disease (eGFR <20 mL/min/1.73 m2) at baseline.

  • Patient on continuous renal replacement therapy at baseline or TDM.

  • Alternative clinical diagnosis (e.g. severe acute blood loss).

Hepatotoxicity
  • End-stage liver cirrhosis [Model for End-stage Liver Disease (MELD) score ≥21] at baseline.

  • Severe liver enzyme abnormality where toxicity definitions are met at baseline.

  • Alternative clinical diagnosis (e.g. active viral hepatitis).

C. difficile
  • Established CDI prior to antibiotic initiation.

  • Faecal sample testing was not performed.

  • Alternative clinical diagnosis (e.g. recent bowel surgery).

ToxicityExclusion criteria
Neurotoxicity
  • Premorbid neurological impairment (e.g. severe dementia).

  • Extracorporeal membrane oxygenation at baseline or TDM.

  • Alternative clinical diagnosis (e.g. mechanical ventilation and copious sedatives rendering cognitive status undeterminable).

Nephrotoxicity
  • Chronic kidney disease (eGFR <20 mL/min/1.73 m2) at baseline.

  • Patient on continuous renal replacement therapy at baseline or TDM.

  • Alternative clinical diagnosis (e.g. severe acute blood loss).

Hepatotoxicity
  • End-stage liver cirrhosis [Model for End-stage Liver Disease (MELD) score ≥21] at baseline.

  • Severe liver enzyme abnormality where toxicity definitions are met at baseline.

  • Alternative clinical diagnosis (e.g. active viral hepatitis).

C. difficile
  • Established CDI prior to antibiotic initiation.

  • Faecal sample testing was not performed.

  • Alternative clinical diagnosis (e.g. recent bowel surgery).

Table 1.

Patient exclusion criteria

ToxicityExclusion criteria
Neurotoxicity
  • Premorbid neurological impairment (e.g. severe dementia).

  • Extracorporeal membrane oxygenation at baseline or TDM.

  • Alternative clinical diagnosis (e.g. mechanical ventilation and copious sedatives rendering cognitive status undeterminable).

Nephrotoxicity
  • Chronic kidney disease (eGFR <20 mL/min/1.73 m2) at baseline.

  • Patient on continuous renal replacement therapy at baseline or TDM.

  • Alternative clinical diagnosis (e.g. severe acute blood loss).

Hepatotoxicity
  • End-stage liver cirrhosis [Model for End-stage Liver Disease (MELD) score ≥21] at baseline.

  • Severe liver enzyme abnormality where toxicity definitions are met at baseline.

  • Alternative clinical diagnosis (e.g. active viral hepatitis).

C. difficile
  • Established CDI prior to antibiotic initiation.

  • Faecal sample testing was not performed.

  • Alternative clinical diagnosis (e.g. recent bowel surgery).

ToxicityExclusion criteria
Neurotoxicity
  • Premorbid neurological impairment (e.g. severe dementia).

  • Extracorporeal membrane oxygenation at baseline or TDM.

  • Alternative clinical diagnosis (e.g. mechanical ventilation and copious sedatives rendering cognitive status undeterminable).

Nephrotoxicity
  • Chronic kidney disease (eGFR <20 mL/min/1.73 m2) at baseline.

  • Patient on continuous renal replacement therapy at baseline or TDM.

  • Alternative clinical diagnosis (e.g. severe acute blood loss).

Hepatotoxicity
  • End-stage liver cirrhosis [Model for End-stage Liver Disease (MELD) score ≥21] at baseline.

  • Severe liver enzyme abnormality where toxicity definitions are met at baseline.

  • Alternative clinical diagnosis (e.g. active viral hepatitis).

C. difficile
  • Established CDI prior to antibiotic initiation.

  • Faecal sample testing was not performed.

  • Alternative clinical diagnosis (e.g. recent bowel surgery).

Evaluation of toxicity

Investigators blinded to TDM recorded Cmin at the time of evaluation. Toxicity events were diagnosed based on changes in predefined clinical grading criteria and relevant serum biomarkers recorded within 24 h prior to drug prescription (baseline) and ± 24 h of TDM measurement. The development of toxicity was correlated with TDM measurements at the time of presentation, thereby establishing a concentration-dependent response. In order also to consider each patient’s overall clinical picture, two scoring systems were utilized to determine general illness severity both at baseline and at the time of TDM.

The SOFA score provided a measure of change in overall health status by evaluating six different organ systems separately.8 Baseline SOFA scores were compared with SOFA scores at TDM in an attempt to elicit whether suspected β-lactam-induced toxicity corresponded to a deterioration in overall patient health. The systemic inflammatory response syndrome (SIRS) criterion9 was used as a surrogate index for sepsis and was calculated at baseline and TDM, with the scores then compared. Sepsis can precipitate complications such as organ failure and septic shock10 and monitoring SIRS scores aided evaluation of β-lactam toxicity parameters such as neurotoxicity, which may present with symptoms similar to sepsis-induced hypoperfusion.11

Toxicity definitions

Neurotoxicity

Patients were deemed to have symptomatic neurotoxicity if one or more of the following clinical features were present:

  • Deteriorating consciousness as per the Glasgow coma scale (GCS). Patient GCS scores were retrieved from medical files and were determined based on standard criteria and scored between 3 and 15.12 Where medical files indicate that the patient was intubated, a verbal response score was estimated retrospectively using a linear regression model.13 Cognitive impairment was assessed using patients’ neurological SOFA (nSOFA) sub-score adopted from Beumier et al.,4 and defined in Table 2. To assess neurotoxicity, baseline neurological function based on nSOFA scores (nSOFABaseline) was compared with the patient’s cognitive status at the time of TDM (nSOFATDM; ± 24 h). Neurological worsening status (NWS) was defined as ΔnSOFA (nSOFATDM – nSOFABaseline) ≥ 1 for an nSOFABaseline of 0–2. Patient ΔnSOFA were tabulated with the corresponding antibiotic serum concentration at the time of TDM to determine correlation.

  • Abnormal electroencephalograph (EEG). In instances where patients underwent an EEG within ± 24 h of TDM, reports were evaluated to verify a diagnosis of diffuse drug-induced encephalopathy. In the absence of an alternative diagnosis, patients with EEG reports indicating the presence of slowing and generalized semiperiodic discharges with triphasic morphology ranging from 0.5–2 Hz were deemed to have experienced antibiotic-induced neurotoxicity.14 This EEG pattern has been described as characteristic of β-lactam-induced encephalopathy, notably with cephalosporins.15–17

  • Neurotoxicity symptoms recorded in the clinical notes. A method of evaluating antibiotic-induced neurotoxicity reported by Zhang et al.18 was implemented. Patient medical charts were reviewed at baseline and at ± 24 h of TDM for descriptive indicators of drug-induced neurotoxicity. Symptoms indicative of severe neurotoxicity included status epilepticus, non-convulsive status epilepticus (NCSE), and seizure or commencement of anticonvulsant therapy. Other neurotoxic symptoms presenting acutely were considered mild to moderate. These included tremor, bizarre behaviour, delirium, slurred or incoherent speech, drowsiness/sleepiness, retrograde amnesia, myoclonus, hallucination, confusion, ataxia/loss of coordination and disorientation.

Table 2.

nSOFA score and corresponding GCS score

nSOFA scoreGCS score
015
113–14
210–12
36–9
4<6
nSOFA scoreGCS score
015
113–14
210–12
36–9
4<6
Table 2.

nSOFA score and corresponding GCS score

nSOFA scoreGCS score
015
113–14
210–12
36–9
4<6
nSOFA scoreGCS score
015
113–14
210–12
36–9
4<6

In the absence of an underlying brain disease or a contributing medical condition, associated neurological symptoms were attributed to neurotoxicity precipitated by CNS exposure to elevated concentrations of the prescribed β-lactam antibiotic. TDM results at which the neurotoxic complications became apparent were analysed to determine a concentration-dependent presentation. The temporal association of the symptom onset, drug administration and symptom relief after drug discontinuation were used to evaluate the adverse drug reaction.

Nephrotoxicity

Nephrotoxicity was defined in terms of acute kidney injury (AKI). This was determined by evaluating increases in serum creatinine (SCr) (μmol/L) at TDM over a preceding measurement within 48 h as per the Acute Kidney Injury Network criteria.19 This definition is somewhat conservative in regards to time sequence between elevated antibiotic concentrations and onset of nephrotoxicity, but was chosen to minimize the possibility of confounding due to other unmeasured events.

The severity of renal function impairment was classified into three stages:

  • Stage 1: ≥26.4 μmol/L increase or ≥50% increase in SCr.

  • Stage 2: ≥100% increase in SCr.

  • Stage 3: ≥200% increase or 44 μmol/L increase in SCr to at least 354 μmol/L.

Hepatotoxicity

In the absence of alternative aetiology, β-lactam-induced hepatotoxicity was diagnosed based on derangement in enzyme biomarkers at TDM compared with baseline using the drug-induced liver injury (DILI) criteria defined by Aithal et al.20 Serum ALT and alkaline phosphatase (ALP) activity measured were expressed as a ratio with respect to upper limits of normal (ULN) (ALT≤45 IU/L and ALP≤120 IU/L).

The pattern of DILI was determined using an R value, which was defined as ALT/ULN divided by ALP/ULN. Hepatotoxicity was classified biochemically as being hepatocellular, cholestatic or mixed (hepatocellular, R ≥5; cholestatic, R ≤2; mixed, R >2 and <5).20 Patient medical charts were also reviewed for documentation of associated signs and symptoms (e.g. jaundice, pruritus) and supporting diagnostic tests (e.g. relevant imaging).

CDI

The incidence of CDI following antibiotic exposure is significantly increased for up to 11 days following antibiotic initiation21 and this time interval was adopted by our study. The presence of opportunistic CDI in patients was confirmed by a positive C. difficile faecal toxin assay, culture or PCR amplification result within 11 days of TDM.

Confounding factors

In attributing the indicators of β-lactam toxicity to concentration-induced exposure, several confounders were considered. Given the retrospective nature of the study these could not be controlled prior to data collection and hence were adjusted for statistically in our analysis. They included the patient’s: (i) age and gender; (ii) baseline cognitive, renal and hepatic physiological function; (iii) use of other antimicrobial agents which may have contributed to the development of CDI; (iv) concurrent use of contributory neurotoxic, nephrotoxic or hepatotoxic agents within 24 h prior to presenting toxicity.

The pharmaceuticals considered as the most likely confounders included benzodiazepines and other anaesthetics, anticonvulsants, antipsychotics, opioids, angiotensin-converting-enzyme (ACE) inhibitors, angiotensin II receptor (AR II) antagonists, diuretics, non-steroidal anti-inflammatory drugs (NSAIDs), alternative antibiotics, paracetamol and statins, due to their well-established additive toxicity and prevalence of use. Less likely potential confounding agents were determined and accounted for on an individual basis (e.g. cyclosporine, radio imaging contrast).

Statistics

Statistical analysis was performed using Stata14 (StataCorp LP, College Station, TX, USA). Using Fisher’s exact test, differences in categorical variables between groups (toxicity and non-toxicity) were assessed across all antibiotics. An investigation of antibiotic toxicity was conducted using t-test (two-tailed) analysis with equal variances. Cmin was expressed as mean±SD. A multivariable model was utilized to identify confounding factors associated with antibiotic toxicity, namely the concurrent use of pharmaceuticals in the 24 h preceding toxicity assessment, and to test the strength of association between Cmin and toxicity outcomes. Logistic regression, independent of confounders, was performed to identify threshold antibiotic concentrations that predisposed patients to toxicity development. A P<0.05 was considered as statistically significant.

Results

There were no differences in baseline characteristics across all three antibiotic groups (Table 3). Toxicity events rates per antibiotic are summarized in Table 4. Of all TDM results analysed, 73% were obtained from patients in the ICU. There were no differences in the number of cases of neurotoxicity (P =0.2), nephrotoxicity (P =1.0), hepatotoxicity (P =0.8) and C. difficile (P =0.4) between ICU and non-ICU patients, as determined by a Fisher’s exact test. Furthermore, this finding was unchanged when split by antibiotic treatment (data not shown). The mean time between the initiation of antibiotic therapy and the TDM measurements analysed was 4.23 ± 4.99 days.

Table 3.

Baseline characteristics in study population

Piperacillin (N = 223)Meropenem (N = 94)Flucloxacillin (N = 61)
Male, n (%)157 (70)71 (76)43 (70)
Age (years), mean±SD53.9 ± 16.851.5 ± 16.156.3 ± 18.6
Weight (kg), mean±SD77.4 ± 20.576.2 ± 19.479.2 ± 21.1
BMI (kg/m2), mean±SD25.9 ± 5.625.7 ± 5.726.9 ± 5.8
SOFA score, mean±SD5.4 ± 3.95.7 ± 4.33.3 ± 3.5
SIRS score, mean±SD1.4 ± 0.91.5 ± 11.1 ± 1
Continuous renal replacement therapy, n (%)25 (11)21 (22)4 (7)
eGFR ≥90 mL/min/1.73 m2, n (%)87 (39)27 (29)20 (33)
eGFR >25 and <50 mL/min/1.73 m2, n (%)46 (21)26 (28)16 (26)
Cardiovascular disease, n (%)79 (35)43 (46)18 (30)
Diabetes, n (%)48 (22)22 (23)14 (23)
COPD/asthma, n (%)36 (16)11 (12)4 (7)
Cancer, n (%)37 (17)13 (14)1 (2)
Solid organ transplant, n (%)36 (16)25 (27)7 (11)
Stem cell transplant, n (%)19 (9)3 (3)1 (2)
Piperacillin (N = 223)Meropenem (N = 94)Flucloxacillin (N = 61)
Male, n (%)157 (70)71 (76)43 (70)
Age (years), mean±SD53.9 ± 16.851.5 ± 16.156.3 ± 18.6
Weight (kg), mean±SD77.4 ± 20.576.2 ± 19.479.2 ± 21.1
BMI (kg/m2), mean±SD25.9 ± 5.625.7 ± 5.726.9 ± 5.8
SOFA score, mean±SD5.4 ± 3.95.7 ± 4.33.3 ± 3.5
SIRS score, mean±SD1.4 ± 0.91.5 ± 11.1 ± 1
Continuous renal replacement therapy, n (%)25 (11)21 (22)4 (7)
eGFR ≥90 mL/min/1.73 m2, n (%)87 (39)27 (29)20 (33)
eGFR >25 and <50 mL/min/1.73 m2, n (%)46 (21)26 (28)16 (26)
Cardiovascular disease, n (%)79 (35)43 (46)18 (30)
Diabetes, n (%)48 (22)22 (23)14 (23)
COPD/asthma, n (%)36 (16)11 (12)4 (7)
Cancer, n (%)37 (17)13 (14)1 (2)
Solid organ transplant, n (%)36 (16)25 (27)7 (11)
Stem cell transplant, n (%)19 (9)3 (3)1 (2)
Table 3.

Baseline characteristics in study population

Piperacillin (N = 223)Meropenem (N = 94)Flucloxacillin (N = 61)
Male, n (%)157 (70)71 (76)43 (70)
Age (years), mean±SD53.9 ± 16.851.5 ± 16.156.3 ± 18.6
Weight (kg), mean±SD77.4 ± 20.576.2 ± 19.479.2 ± 21.1
BMI (kg/m2), mean±SD25.9 ± 5.625.7 ± 5.726.9 ± 5.8
SOFA score, mean±SD5.4 ± 3.95.7 ± 4.33.3 ± 3.5
SIRS score, mean±SD1.4 ± 0.91.5 ± 11.1 ± 1
Continuous renal replacement therapy, n (%)25 (11)21 (22)4 (7)
eGFR ≥90 mL/min/1.73 m2, n (%)87 (39)27 (29)20 (33)
eGFR >25 and <50 mL/min/1.73 m2, n (%)46 (21)26 (28)16 (26)
Cardiovascular disease, n (%)79 (35)43 (46)18 (30)
Diabetes, n (%)48 (22)22 (23)14 (23)
COPD/asthma, n (%)36 (16)11 (12)4 (7)
Cancer, n (%)37 (17)13 (14)1 (2)
Solid organ transplant, n (%)36 (16)25 (27)7 (11)
Stem cell transplant, n (%)19 (9)3 (3)1 (2)
Piperacillin (N = 223)Meropenem (N = 94)Flucloxacillin (N = 61)
Male, n (%)157 (70)71 (76)43 (70)
Age (years), mean±SD53.9 ± 16.851.5 ± 16.156.3 ± 18.6
Weight (kg), mean±SD77.4 ± 20.576.2 ± 19.479.2 ± 21.1
BMI (kg/m2), mean±SD25.9 ± 5.625.7 ± 5.726.9 ± 5.8
SOFA score, mean±SD5.4 ± 3.95.7 ± 4.33.3 ± 3.5
SIRS score, mean±SD1.4 ± 0.91.5 ± 11.1 ± 1
Continuous renal replacement therapy, n (%)25 (11)21 (22)4 (7)
eGFR ≥90 mL/min/1.73 m2, n (%)87 (39)27 (29)20 (33)
eGFR >25 and <50 mL/min/1.73 m2, n (%)46 (21)26 (28)16 (26)
Cardiovascular disease, n (%)79 (35)43 (46)18 (30)
Diabetes, n (%)48 (22)22 (23)14 (23)
COPD/asthma, n (%)36 (16)11 (12)4 (7)
Cancer, n (%)37 (17)13 (14)1 (2)
Solid organ transplant, n (%)36 (16)25 (27)7 (11)
Stem cell transplant, n (%)19 (9)3 (3)1 (2)
Table 4.

Toxicity event rates in study population following exclusion criteria

PiperacillinMeropenemFlucloxacillin
Neurotoxicity23/202 (11.4%)13/82 (15.9%)6/57 (10.5%)
Nephrotoxicity16/189 (8.5%)5/72 (6.9%)4/54 (7.4%)
 stage 18 (50%)3 (60%)2 (50%)
 stage 26 (37.5%)2 (40%)2 (50%)
 stage 32 (12.5%)0 (0%)0 (0%)
Hepatotoxicity12/180 (6.7%)6/72 (8.3%)4/46 (8.7%)
 hepatocellular7 (58.3%)2 (33.3%)0 (0%)
 cholestatic4 (33.3%)4 (66.6%)3 (75%)
 mixed1 (8.3%)0 (0%)1 (25%)
C. difficile7/89 (7.9%)1/38 (2.6%)0/15 (0%)
PiperacillinMeropenemFlucloxacillin
Neurotoxicity23/202 (11.4%)13/82 (15.9%)6/57 (10.5%)
Nephrotoxicity16/189 (8.5%)5/72 (6.9%)4/54 (7.4%)
 stage 18 (50%)3 (60%)2 (50%)
 stage 26 (37.5%)2 (40%)2 (50%)
 stage 32 (12.5%)0 (0%)0 (0%)
Hepatotoxicity12/180 (6.7%)6/72 (8.3%)4/46 (8.7%)
 hepatocellular7 (58.3%)2 (33.3%)0 (0%)
 cholestatic4 (33.3%)4 (66.6%)3 (75%)
 mixed1 (8.3%)0 (0%)1 (25%)
C. difficile7/89 (7.9%)1/38 (2.6%)0/15 (0%)
Table 4.

Toxicity event rates in study population following exclusion criteria

PiperacillinMeropenemFlucloxacillin
Neurotoxicity23/202 (11.4%)13/82 (15.9%)6/57 (10.5%)
Nephrotoxicity16/189 (8.5%)5/72 (6.9%)4/54 (7.4%)
 stage 18 (50%)3 (60%)2 (50%)
 stage 26 (37.5%)2 (40%)2 (50%)
 stage 32 (12.5%)0 (0%)0 (0%)
Hepatotoxicity12/180 (6.7%)6/72 (8.3%)4/46 (8.7%)
 hepatocellular7 (58.3%)2 (33.3%)0 (0%)
 cholestatic4 (33.3%)4 (66.6%)3 (75%)
 mixed1 (8.3%)0 (0%)1 (25%)
C. difficile7/89 (7.9%)1/38 (2.6%)0/15 (0%)
PiperacillinMeropenemFlucloxacillin
Neurotoxicity23/202 (11.4%)13/82 (15.9%)6/57 (10.5%)
Nephrotoxicity16/189 (8.5%)5/72 (6.9%)4/54 (7.4%)
 stage 18 (50%)3 (60%)2 (50%)
 stage 26 (37.5%)2 (40%)2 (50%)
 stage 32 (12.5%)0 (0%)0 (0%)
Hepatotoxicity12/180 (6.7%)6/72 (8.3%)4/46 (8.7%)
 hepatocellular7 (58.3%)2 (33.3%)0 (0%)
 cholestatic4 (33.3%)4 (66.6%)3 (75%)
 mixed1 (8.3%)0 (0%)1 (25%)
C. difficile7/89 (7.9%)1/38 (2.6%)0/15 (0%)

The corresponding β-lactam mean (±SD) trough TDM concentrations (mg/L) were determined for each toxicity parameter (Figure 1). Mean Cmin in patients with neurotoxicity was significantly higher when compared with patients without neurotoxicity across all β-lactam antibiotics investigated (Figure 1a). 52% of all neurotoxicity cases were diagnosed with both NWS and presence of neurotoxic symptoms, 29% exhibited only neurotoxicity symptoms, 14% only NWS, 2% with only diagnostic features on EEG, and 2% displayed the combination of NWS, EEG irregularity and neurotoxic symptoms.

Mean Cmin in patients with and without (a) neurotoxicity, (b) nephrotoxicity and (c) hepatotoxicity. PIP, piperacillin; MEM, meropenem; FLU, flucloxacillin.
Figure 1.

Mean Cmin in patients with and without (a) neurotoxicity, (b) nephrotoxicity and (c) hepatotoxicity. PIP, piperacillin; MEM, meropenem; FLU, flucloxacillin.

Elevated piperacillin and meropenem mean Cmin were associated with the development of nephrotoxicity (P <0.01; Figure 1b). We found no significant correlation between increasing antibiotic Cmin and the incidence of hepatotoxicity (Figure 1c). CDI was diagnosed infrequently and therefore could only be assessed in patients administered with piperacillin for which we found no concentration–toxicity relationship (P =0.79).

The use of a benzodiazepine was protective against neurotoxicity for patients administered piperacillin. The use of a diuretic agent was associated with a significantly increased risk of piperacillin nephrotoxicity. There were no differences in the occurrence of neurotoxicity and nephrotoxicity for other pharmaceuticals including iodinated radio contrast agents (Tables 5 and 6). Confounders were not analysed for toxicity outcomes found to be independent of Cmin. These included hepatotoxicity and CDI, as well as flucloxacillin nephrotoxicity. Some cofounders were omitted from analysis due to low event rates.

Table 5.

Multivariable logistic regression model of factors associated with neurotoxicity

Piperacillin
Meropenem
Flucloxacillin
Drug typeOR95% CIPOR95% CIPOR95% CIP
Benzodiazepine0.260.07–0.960.040.720.16–3.150.660.880.08–9.280.92
Opioid1.050.34–3.250.940.390.09–1.680.211.130.17–7.740.90
Intravenous anaesthetic2.250.87–5.800.091.290.27–6.080.271omittedNA
Psychotropic0.850.23–3.240.821.450.33–6.490.336.380.32–12.640.22
Anticonvulsant2.840.65–12.361.160.620.07–5.870.071omittedNA
Piperacillin
Meropenem
Flucloxacillin
Drug typeOR95% CIPOR95% CIPOR95% CIP
Benzodiazepine0.260.07–0.960.040.720.16–3.150.660.880.08–9.280.92
Opioid1.050.34–3.250.940.390.09–1.680.211.130.17–7.740.90
Intravenous anaesthetic2.250.87–5.800.091.290.27–6.080.271omittedNA
Psychotropic0.850.23–3.240.821.450.33–6.490.336.380.32–12.640.22
Anticonvulsant2.840.65–12.361.160.620.07–5.870.071omittedNA

NA, not applicable. Values displaying significant differences (P < 0.05) are shown in bold.

Table 5.

Multivariable logistic regression model of factors associated with neurotoxicity

Piperacillin
Meropenem
Flucloxacillin
Drug typeOR95% CIPOR95% CIPOR95% CIP
Benzodiazepine0.260.07–0.960.040.720.16–3.150.660.880.08–9.280.92
Opioid1.050.34–3.250.940.390.09–1.680.211.130.17–7.740.90
Intravenous anaesthetic2.250.87–5.800.091.290.27–6.080.271omittedNA
Psychotropic0.850.23–3.240.821.450.33–6.490.336.380.32–12.640.22
Anticonvulsant2.840.65–12.361.160.620.07–5.870.071omittedNA
Piperacillin
Meropenem
Flucloxacillin
Drug typeOR95% CIPOR95% CIPOR95% CIP
Benzodiazepine0.260.07–0.960.040.720.16–3.150.660.880.08–9.280.92
Opioid1.050.34–3.250.940.390.09–1.680.211.130.17–7.740.90
Intravenous anaesthetic2.250.87–5.800.091.290.27–6.080.271omittedNA
Psychotropic0.850.23–3.240.821.450.33–6.490.336.380.32–12.640.22
Anticonvulsant2.840.65–12.361.160.620.07–5.870.071omittedNA

NA, not applicable. Values displaying significant differences (P < 0.05) are shown in bold.

Table 6.

Multivariable logistic regression model of factors associated with nephrotoxicity

Piperacillin
Meropenem
OR95% CIPOR95% CIP
NSAIDs2.010.44–9.210.371omittedNA
ACE inhibitor0.900.09–8.730.941omittedNA
AR II antagonist1omittedNA1omittedNA
Diuretics31.323.33–294.70<0.011.150.16–8.090.88
Calcineurin inhibitor1.530.40–5.780.541.710.23–12.340.59
Alternate antibiotic1.550.50–4.840.452.460.25–24.500.77
Radio contrast16.780.71–398.000.081.840.15–21.220.63
Piperacillin
Meropenem
OR95% CIPOR95% CIP
NSAIDs2.010.44–9.210.371omittedNA
ACE inhibitor0.900.09–8.730.941omittedNA
AR II antagonist1omittedNA1omittedNA
Diuretics31.323.33–294.70<0.011.150.16–8.090.88
Calcineurin inhibitor1.530.40–5.780.541.710.23–12.340.59
Alternate antibiotic1.550.50–4.840.452.460.25–24.500.77
Radio contrast16.780.71–398.000.081.840.15–21.220.63

NA, not applicable. Values displaying significant differences (P < 0.05) are shown in bold.

Table 6.

Multivariable logistic regression model of factors associated with nephrotoxicity

Piperacillin
Meropenem
OR95% CIPOR95% CIP
NSAIDs2.010.44–9.210.371omittedNA
ACE inhibitor0.900.09–8.730.941omittedNA
AR II antagonist1omittedNA1omittedNA
Diuretics31.323.33–294.70<0.011.150.16–8.090.88
Calcineurin inhibitor1.530.40–5.780.541.710.23–12.340.59
Alternate antibiotic1.550.50–4.840.452.460.25–24.500.77
Radio contrast16.780.71–398.000.081.840.15–21.220.63
Piperacillin
Meropenem
OR95% CIPOR95% CIP
NSAIDs2.010.44–9.210.371omittedNA
ACE inhibitor0.900.09–8.730.941omittedNA
AR II antagonist1omittedNA1omittedNA
Diuretics31.323.33–294.70<0.011.150.16–8.090.88
Calcineurin inhibitor1.530.40–5.780.541.710.23–12.340.59
Alternate antibiotic1.550.50–4.840.452.460.25–24.500.77
Radio contrast16.780.71–398.000.081.840.15–21.220.63

NA, not applicable. Values displaying significant differences (P < 0.05) are shown in bold.

The threshold concentrations for which there is 50% risk of developing a neurotoxicity event (piperacillin, Cmin >361.4 mg/L; meropenem, Cmin >64.2 mg/L; flucloxacillin, Cmin >125.1 mg/L) or nephrotoxicity (piperacillin, Cmin >452.65 mg/L; meropenem, Cmin >44.45 mg/L) varied across antibiotics.

Discussion

This retrospective study investigated the presence of a concentration–toxicity relationship for several commonly prescribed β-lactam antibiotics and defined a number of toxicity threshold concentrations. McDonald et al.22 previously failed to find an association between high-dose piperacillin and meropenem therapy with a number of toxicity outcomes. However, our study differs from theirs in several important aspects. Notably, our use of serum concentration is a superior measure of antibiotic exposure than dosage alone. In addition, our application of strict selection criteria with consideration of baseline organ function and our larger sample size further strengthen our results.

We found a positive association between elevated serum Cmin and acute neurotoxicity for all three β-lactams investigated. This is consistent with previous studies that have similarly reported neurotoxic complications related to β-lactam use, especially in the setting of excessive dosing and impaired renal clearance.23 It has been theorized that the pathogenesis of β-lactam neurotoxicity is related to interference or inhibition of gamma-aminobutyric acid (GABA) binding to GABAA receptors.24 The seizure activity of these agents has been linked to the β-lactam ring as it shares a structural similarity with the GABA neurotransmitter.23,25 Piperacillin has previously been implicated with neuropsychiatric events, especially in critically ill patients, owing to significantly altered PK factors.26–28 A study by Beumier et al.4 had previously correlated an increased incidence of neurological deterioration with high Cmin:MIC ratio for piperacillin and meropenem. However, this study assessed neurotoxicity using only NWS and the precise threshold concentrations for toxicity remained undefined.

Furthermore, our results suggest that exposure to increased concentrations of piperacillin and meropenem also exert a nephrotoxic effect. In past studies, the incidence of renal injury was estimated to be 10% with β-lactam monotherapy29 and thought to result from acute proximal tubular necrosis.30 Although nephrotoxicity from penicillins is believed to be infrequent, piperacillin may be an exception, with both this study and prior publications identifying it as a causal agent in instances of acute renal injury and noting the toxic effect on renal function and delayed renal recovery.31,32

All β-lactam antibiotics may be associated with a transient increase in liver enzymes.33 Toxicity is thought to be allergic in nature and usually mild and asymptomatic, with the elevation in liver enzymes ordinarily reversible following discontinuation of therapy. Flucloxacillin-associated cholestasis is perhaps the best documented although the mechanisms by which it develops remains poorly understood.34 Recently, pharmacogenomic studies have noted an association between flucloxacillin DILI and the HLA-B*5701 gene.35 In line with this, our study failed to demonstrate a statistically significant antibiotic concentration–toxicity relationship for precipitation of hepatotoxicity.

The primary effect of antibiotics on developing CDI is alteration of intestinal flora, particularly destruction of protective anaerobic bacteria.36 Nearly all classes of β-lactams have been associated with CDI, with carbapenems considered to pose a greater risk than penicillins.37 Though CDI rates are strongly correlated with increasing antibiotic exposure, both duration and dosage,21 antibiotic serum concentrations have not been investigated. In our study, CDI was diagnosed infrequently and when analysed for piperacillin did not demonstrate any association with high plasma concentrations.

The favourable safety profile of β-lactams, rising prevalence of antibiotic resistance and vulnerability of hospitalized patients to infection have been compelling reasons for clinicians to favour more aggressive antibiotic therapy. Although ensuring that therapeutically effective physiological concentrations are attained can improve patient outcomes, such benefit has definite toxicity trade-offs. By defining threshold concentrations above which a number of these toxicity events can be anticipated, the findings of this study could prove significant in guiding β-lactam therapy within a hospital setting and further recognize TDM as an important tool for effective β-lactam prescription in the critically ill.

This study does have a number of limitations. First, the retrospective design hinders our ability to establish causation of toxicity as directly attributable to β-lactam therapy. The effects of this were mitigated by adhering to strict toxicity definitions, recognizing potential confounders and consistently monitoring overall patient health as an alternative explanation for apparent toxicity. SOFA and SIRS were selected as measures of overall illness severity due to the relative ease with which they could be calculated retrospectively, with relevant clinical and laboratory data routinely available in patient files and online databases. However, a limitation of both SOFA and SIRS is that they are inherently non-specific indicators for sepsis. Second, the use of subjective observational parameters to diagnose neurotoxicity is firmly reliant on physical examinations and subject to differing interpretations among individual clinicians. This also renders it difficult for researchers to appreciate subtle clinical changes during retrospective data collection. Third, Cmin samples were determined on the basis of the hospital’s electronic time-stamps. As such, it was not possible to monitor for any time-delays between collection and serum analysis, though in our local experience delays beyond 15 min rarely occur. Furthermore, for a small number of patients (n =6) administered flucloxacillin at a 4 h dosing interval, our definition of Cmin may not have reflected a true trough, albeit this is less likely considering that for all six patients TDM samples were in fact collected < 1 h prior to a subsequent dose. Fourth, although data were collected in relation to the simultaneous use of other pharmaceuticals and controlled for during analysis, we did not collect data regarding exact doses at which these potential confounders were administered. This is important in relation to high-dose sedatives and anaesthetics and could explain the apparent protective effect of benzodiazepine use on neurotoxicity development. These agents may have increased the thresholds for seizure development, masking neurotoxic events, or alternatively reduced GCS and thereby overestimated the neurotoxic effects of the antibiotics. However, both effects should have been present independent of TDM levels. Fifth, our use of SCr as a biomarker for diagnosing AKI is less than ideal (sensitivity = 80% and specificity = 90%).38 Increase in SCr does not directly reflect renal injury and an absence of change in SCr does not ensure absence of tubular insult.38 SCr itself is dependent on many non-renal factors independent of renal function, such as muscle mass and nutritional status, for which we were unable to control. As regular TDM sample collection was not performed on each patient we were not aware of antibiotic concentrations leading up to the time of nephrotoxicity. Furthermore, although we excluded patients with established chronic kidney disease from toxicity evaluation, for those with mild or moderate renal impairment at baseline we were unable to ascertain unequivocally whether elevated serum antibiotic concentrations had led to AKI or whether the patient’s baseline impairment itself had resulted in elevated antibiotic concentration. Considering all patients diagnosed with piperacillin and meropenem nephrotoxicity, only 6.25% and 20%, respectively, had baseline renal impairment warranting dose adjustment [estimated glomerular filtration rate (eGFR) <50 mL/min/1.73 m2]. As such, it is unlikely that clearance of either β-lactam agent would have been significantly compromised by baseline function in the majority of patients eventually diagnosed with nephrotoxicity. Sixth, our study also lacked information on post-discharge CDI diagnosis. As such, the apparent lack of association of CDI with high serum Cmin may be most generalizable to acute hospital onset. Finally, as empirical antibiotic initiation and pre-emptive treatment were more common than definitive therapy and diagnosis of infection was not diligently recorded utilizing our hospital’s diagnosis-related group (DRG) codes, we were unable to perform subgroup analysis based on each patient’s initial diagnosis.

Conclusions

Currently the incidence of β-lactam-induced toxicity is clinically underestimated. Although a future prospective trial with a larger sample size is required to validate the findings of this study, our results suggest that a concentration–toxicity relationship does in fact exist for β-lactam antibiotics and the threshold concentrations we have defined could inform the future performance and interpretation of TDM. Based on the ongoing challenges of bacterial resistance, by utilizing TDM early in the course of β-lactam antibiotic treatment, clinicians should aim to achieve the highest serum concentrations that can be safely attained, whilst ensuring that these toxicity thresholds are not surpassed. This should be feasible as the proposed thresholds are well in excess of those likely to be necessary for the treatment of susceptible bacteria. Additionally, TDM may be useful to diagnose β-lactam antibiotic toxicity, thereby avoiding other unnecessary investigations.

Acknowledgements

The preliminary results from this study were presented at the Australasian Society for Infectious Diseases (ASID) Annual Scientific Meeting 2016 (Abstract 90). The completed study results were presented at the ASID Annual Scientific Meeting 2017 (Abstract 101).

We would like to thank the Department of Clinical Pharmacology and the Department of Intensive Care Medicine at St Vincent’s Hospital (Sydney, Australia) for assisting with specimen analysis and collection, respectively.

Funding

This work was supported by the University of Notre Dame Australia at the School of Medicine Sydney (NO041189 to S. I.).

Transparency declarations

None to declare.

References

1

Roberts
JA
,
Norris
R
,
Paterson
DL
et al.
Therapeutic drug monitoring of antimicrobials
.
Br J Clin Pharmacol
2012
;
73
:
27
36
.

2

Gonçalves-Pereira
J
,
Póvoa
P.
Antibiotics in critically ill patients: a systematic review of the pharmacokinetics of β-lactams
.
Crit Care
2011
;
15
:
R206.

3

Grill
MF
,
Maganti
RK.
Neurotoxic effects associated with antibiotic use: management considerations
.
Br J Clin Pharmacol
2011
;
72
:
381
93
.

4

Beumier
M
,
Casu
GS
,
Hites
M
et al.
Elevated β-lactam concentrations are associated with neurological deterioration in ICU septic patients
.
Minerva Anestesiol
2015
;
81
:
497
506
.

5

Fugate
JE
,
Kalimullah
EA
,
Hocker
SE
et al.
Cefepime neurotoxicity in the ICU: a cause of severe, underappreciated encephalopathy
.
Crit Care
2013
;
17
:
R264.

6

Slama
TG.
Clinical review: balancing the therapeutic, safety, and economic issues underlying effective antipseudomonal carbapenem use
.
Crit Care
2008
;
12
:
233.

7

Lagacé-Wiens
P
,
Rubinstein
E.
Adverse reactions to β-lactam antimicrobials
.
Expert Opin Drug Saf
2012
;
11
:
381
99
.

8

Vincent
JL
,
Moreno
R
,
Takala
J
et al.
The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure
.
Intensive Care Med
1996
;
22
:
707
10
.

9

American College of Chest Physicians/Society of Critical Care Medicine Consensus Conference
.
Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis
.
Crit Care Med
1992
;
20
:
864
74
.

10

Alberti
C
,
Brun-Buisson
C
,
Chevret
S
et al.
Systemic inflammatory response and progression to severe sepsis in critically ill infected patients
.
Am J Respir Crit Care Med
2005
;
171
:
461
8
.

11

Adam
N
,
Kandelman
S
,
Mantz
J
et al.
Sepsis-induced brain dysfunction
.
Expert Rev Anti Infect Ther
2013
;
11
:
211
21
.

12

Teasdale
G
,
Jennett
B.
Assessment and prognosis of coma after head injury
.
Acta Neurochir (Wien)
1976
;
34
:
45
55
.

13

Meredith
W
,
Rutledge
R
,
Fakhry
SM
et al.
The conundrum of the Glasgow Coma Scale in intubated patients: a linear regression prediction of the Glasgow verbal score from the Glasgow eye and motor scores
.
J Trauma Acute Care Surg
1998
;
44
:
839
45
.

14

Bhattacharyya
S
,
Darby
RR
,
Raibagkar
P
et al.
Antibiotic-associated encephalopathy
.
Neurology
2016
;
86
:
963
71
.

15

Roncon-Albuquerque
R
Jr,
Pires
I
,
Martins
R
et al.
Ceftriaxone-induced acute reversible encephalopathy in a patient treated for a urinary tract infection
.
Neth J Med
2009
;
67
:
72
5
.

16

Grill
MF
,
Maganti
R.
Cephalosporin-induced neurotoxicity: clinical manifestations, potential pathogenic mechanisms, and the role of electroencephalographic monitoring
.
Ann Pharmacother
2008
;
42
:
1843
50
.

17

Chow
KM
,
Szeto
CC
,
Hui
AC
et al.
Retrospective review of neurotoxicity induced by cefepime and ceftazidime
.
Pharmacotherapy
2003
;
23
:
369
73
.

18

Zhang
J
,
Huang
C
,
Li
H
et al.
Antibiotic-induced neurotoxicity in dialysis patients: a retrospective study
.
Ren Fail
2013
;
35
:
901
5
.

19

Mehta
RL
,
Kellum
JA
,
Shah
SV
et al.
Acute Kidney Injury Network: report of an initiative to improve outcomes in acute kidney injury
.
Crit Care
2007
;
1
:
R31.

20

Aithal
GP
,
Watkins
PB
,
Andrade
RJ
et al.
Case definition and phenotype standardization in drug‐induced liver injury
.
Clin Pharmacol Ther
2011
;
89
:
806
15
.

21

Brown
KA
,
Fisman
DN
,
Moineddin
R
et al.
The magnitude and duration of Clostridium difficile infection risk associated with antibiotic therapy: a hospital cohort study
.
PLoS One
2014
;
9
:
e105454
.

22

McDonald
C
,
Cotta
MO
,
Little
PJ
et al.
Is high-dose β-lactam therapy associated with excessive drug toxicity in critically ill patients?
Minerva Anestesiol
2016
;
82
:
957
65
.

23

Chow
KM
,
Hui
AC
,
Szeto
CC.
Neurotoxicity induced by β-lactam antibiotics: from bench to bedside
.
Eur J Clin Microbiol Infect Dis
2005
;
24
:
649
53
.

24

De Sarro
A
,
De Sarro
GB
,
Ascioti
C
et al.
Epileptogenic activity of some β-lactam derivatives: structure-activity relationship
.
Neuropharmacology
1989
;
28
:
359
65
.

25

Grøndahl
,
Langmoen
IA.
Epileptogenic effect of antibiotic drugs
.
J Neurosurg
1993
;
78
:
938
43
.

26

Lin
CS
,
Cheng
CJ
,
Chou
CH
et al.
Piperacillin/tazobactam-induced seizure rapidly reversed by high flux hemodialysis in a patient on peritoneal dialysis
.
Am J Med Sci
2007
;
333
:
181
4
.

27

Tong
MKH
,
Siu
YP
,
Yung
CY
et al.
Piperacillin/tazobactam-induced acute delirium in a peritoneal dialysis patient
.
Nephrol Dial Transplant
2004
;
19
:
1341.

28

Huang
WT
,
Hsu
YJ
,
Chu
PL
et al.
Neurotoxicity associated with standard doses of piperacillin in an elderly patient with renal failure
.
Infection
2009
;
37
:
374
6
.

29

Tamma
PD
,
Turnbull
AE
,
Harris
AD
et al.
Less is more: combination antibiotic therapy for the treatment of gram-negative bacteremia in pediatric patients
.
JAMA Pediatr
2013
;
167
:
903
10
.

30

Tune
BM.
Nephrotoxicity of β-lactam antibiotics: mechanisms and strategies for prevention
.
Pediatr Nephrol
1997
;
11
:
768
72
.

31

Pratt
JA
,
Stricherz
MK
,
Verghese
PS
et al.
Suspected piperacillin‐tazobactam induced nephrotoxicity in the pediatric oncology population
.
Pediatr Blood Cancer
2014
;
61
:
366
8
.

32

Jensen
JS
,
Hein
L
,
Lundgren
B
et al.
Kidney failure related to broad-spectrum antibiotics in critically ill patients: secondary end point results from a 1200 patient randomised trial
.
BMJ Open
2012
;
2
:
e000635.

33

Leitner
JM
,
Graninger
W
,
Thalhammer
F.
Hepatotoxicity of antibacterials: pathomechanisms and clinical data
.
Infection
2010
;
38
:
3
11
.

34

Andrews
E
,
Daly
AK.
Flucloxacillin-induced liver injury
.
Toxicology
2008
;
254
:
158
63
.

35

Daly
AK
,
Donaldson
PT
,
Bhatnagar
P
et al.
HLA-B* 5701 genotype is a major determinant of drug-induced liver injury due to flucloxacillin
.
Nat Genet
2009
;
41
:
816
9
.

36

Britton
RA
,
Young
VB.
Interaction between the intestinal microbiota and host in Clostridium difficile colonization resistance
.
Trends Microbiol
2012
;
20
:
313
9
.

37

Brown
KA
,
Khanafer
N
,
Daneman
N
et al.
Meta-analysis of antibiotics and the risk of community-associated Clostridium difficile infection
.
Antimicrob Agents Chemother
2013
;
57
:
2326
32
.

38

Waikar
SS
,
Betensky
RA
,
Emerson
SC
et al.
Imperfect gold standards for kidney injury biomarker evaluation
.
J Am Soc Nephrol
2012
;
23
:
13
21
.