## Abstract

The authors evaluated the association between ambient ozone levels and cardiac mortality in California's South Coast Air Basin during the period 1983–2000 and compared inferences from several types of marginal structural model (MSM) estimators. The authors undertook an ecologic study during the high-ozone seasons among persons over age 55 years. In contrast to conditional regression analysis and MSMs based on G-computation and simple inverse probability-of-treatment weighting (IPTW), an MSM that protected against violation of the experimental treatment assignment (ETA) assumption and considered only those areas that could have experienced both high and low ozone concentrations during 1983–2000 found no consistent evidence that reductions in quarterly 1-hour maximum ozone concentrations from levels above any of the regulatory standards to levels below those standards led to decreases in cardiac mortality; however, it did find evidence of decreases related to a decrease in 8-hour maximum concentrations. The G-computation estimator and simple IPTW estimators were biased because of serious violation of the ETA assumption. These analyses highlight the importance of nonviolation of the ETA assumption for valid inference and the failure of conditional regression to provide marginal estimates in the presence of interactions. Noncausal models also consistently inferred larger associations, which may have been due to bias violation of the ETA assumption on which these models rely.

Associations between daily variation in ambient ozone concentrations and mortality have been reported since the early 1990s (1–8). The current rule for ozone in the National Ambient Air Quality Standards (9) and a recent report from the National Research Council (10) have recognized the association between ozone and mortality, based primarily on the results of 3 meta-analyses (5, 11, 12). Three more recent analyses (2 time series, 1 cohort) have raised the possibility that these mortality associations may be confounded, at least in part, by particulate matter and its constituents (13–15).

Despite this accumulated evidence for associations between ambient ozone concentrations and mortality, there is a need for more investigation, especially single studies of multiple areas (11). Historically, Southern California's South Coast Air Basin has had, and still has, some of the highest ozone concentrations in the United States, with many areas failing to achieve the current maximum 8-hour standard of 75 ppb (16).

In the present study, we evaluated the association between high (versus low) ambient ozone concentrations and cardiac mortality among persons over age 55 years in the South Coast Air Basin during the period 1983–2000. We applied an adaptation of marginal structural models that, in previous studies, has provided estimates of population exposure effects that account for the fact that not all areas and their populations have a positive probability of experiencing all levels of ozone concentrations and has demonstrated that failure to address violations of the so-called “experimental treatment assignment” (ETA) assumption (or “positivity” assumption) can lead to incorrect inference.

## MATERIALS AND METHODS

### Study area

The study area is shown in Figure 1. This Southern California location contained many areas that consistently exceeded National Ambient Air Quality Standards for ozone between 1983 and 2000 but did experience marked reductions in 1-hour and 8-hour maximum ozone concentrations during that period.

Figure 1.

Study domain grid system with locations of pollutant monitors (•) and population densities, South Coast Air Basin, California, 1983–2000. The 20,000-km2 area extends from 34.6°N latitude at its most northern end to 33.2°N latitude at its most southern end. It is bounded on the west by 118.9°W longitude and the Pacific Ocean and extends to 116.8°W longitude at its most eastern end. (AQ, air quality).

Figure 1.

Study domain grid system with locations of pollutant monitors (•) and population densities, South Coast Air Basin, California, 1983–2000. The 20,000-km2 area extends from 34.6°N latitude at its most northern end to 33.2°N latitude at its most southern end. It is bounded on the west by 118.9°W longitude and the Pacific Ocean and extends to 116.8°W longitude at its most eastern end. (AQ, air quality).

### Ambient pollutant data and exposure methods

The methods used have been described in detail elsewhere (17). The population's exposure was estimated from ambient air quality measurements obtained from a network of stations that began monitoring air quality before 1980 (Figure 1). Quarterly average concentrations of 1-hour daily maximum ozone and 24-hour average nitrogen dioxide, sulfur dioxide, and carbon monoxide were compiled from hourly measurements. Quarterly average concentrations of the 24-hour average concentrations of particulate matter ≤10 μm in diameter (PM10) were compiled from monthly averages of every-sixth-day PM10 and daily, every-third-day, and 2-week average measurements of particulate matter ≤2.5 μm in diameter (PM2.5) (see the Web Appendix, which is posted on the Journal’s Web site (http://aje.oxfordjournals.org/)).

The domain was divided into 200 10-km × 10-km grids that covered the populated portion of the South Coast Air Basin; 195 grids were used. Population, demographic, and health outcome data were aggregated into the grids (Web Figure 1). Air quality and meteorologic data were interpolated spatially (inverse distance-squared weighting) from the monitoring stations to the grid centroids (Web Appendix).

We first focused on 1-hour maximum ozone, because the 1-hour maximum standard was in place throughout most of the study period; the 8-hour daily maximum standard was adopted in 1996 (18). We confined our analyses to the months April–June (quarter 2) and July–September (quarter 3) (17) (Web Table 1).

### Mortality and demographic data

Mortality data were obtained for 1983–2000 from public-use data files for underlying and multiple causes of death (19, 20). For the immediate causes of deaths occurring prior to 1999, we used International Classification of Diseases, Ninth Revision, codes 410, 411, 427, and 428 (acute myocardial infarction, intermediate coronary syndrome, arrhythmia, and congestive heart failure). For 1999–2000, we used the International Classification of Diseases, Tenth Revision, codes (I21, I24, I50, I47, I48, I49, and I110).

Spatial allocation of the mortality data was based on information in the single-cause-of-death and multiple-cause-of-death data files. The single-cause-of-death files for 1989–2000 contained the 5-digit zip code of the deceased person's residence, as well as the “city name” for large cities or the “balance of the county” for small cities and rural locations. Only the city name or “balance of county” location variable was available in the public-use, multiple-cause mortality data. The site-specific single-cause deaths were used to establish spatial allocations by grid for each year from 1989 to 2000, adjusted for county-wide, nonaccidental death rates for infants (age <1 year), children and adults (ages 1–59 years), and elderly adults (ages >59 years) (Web Appendix). For the years 1983–1988, no zip code information was available; therefore, the 1989–1991 average spatial allocation factors were applied (Web Appendix).

We obtained socioeconomic and demographic data from the US Census Bureau's decadal surveys (1980, 1990, and 2000) and selected 57 variables (17) (Web Table 2). Spatial allocation of demographic data to exposure grids was based on the smallest geographic unit for which census data were available. GIS software (ArcGIS9; ESRI, Redlands, California) was used to map the demographic data to grids (Web Appendix). Population demographic parameters were estimated for intracensus years by linear interpolation of the gridded data for 1980, 1990, and 2000.

### Data analysis

#### Data structure.

The data consisted of 195 geographic grids with quarterly measurements from 1983–2000 that included 14,040 observations and 72 quarters for persons over age 55 years. The proportion of deaths due to cardiac disease was calculated as the number of such deaths divided by the total population of each sex over age 55 years (most heart problems occur after age 55 years) in the corresponding grid and quarter. We carried out sex-specific analyses to allow for sex differences in underlying susceptibility both to heart disease and to air pollution health effects.

The data for quarters 2 and 3 consisted of 7,020 observations each for males and females. We removed 31 outcome outliers from grids with small numbers of persons; removal did not affect our inferences.

For a detailed description of the data structure, see Moore et al. (17). In short, we denote the observed data structure by where 1) the history of ozone is denoted by where A(t) represents the ozone concentration measured at time t; 2) the history of cardiac mortality as a percentage of the total area-specific population is denoted by and Y(t) represents the mortality proportion measured at time t; and 3) the history of potential time-dependent confounders of the effect of ozone on cardiac mortality is denoted by where W(t) is a multivariate vector of potential confounders (demographic variables, copollutants, and meteorologic variables) measured at time t (Web Table 2). Since we are considering the effect of ozone on cardiac mortality during quarters 2 and 3, we have 36 outcomes of interest.

The exposure variable, denoted by $A*(t)$, is defined as an indicator for whether the continuous ozone concentration variable A(t) falls above a particular cutpoint (i.e., regulatory standard). Thus, for any given grid, $A*(t)$ = 1 indicates a high ozone concentration and $A*(t)=0$ indicates a low ozone concentration.

### Statistical models

The goal of the analysis was to estimate the causal effect of lowering ozone concentrations from levels above various ozone standards that have been considered in the regulatory process to levels below those standards on the proportion of deaths from cardiac disease—that is, the effect of $A*(t)$ on Y(t). We were interested in investigating whether this effect was modified by time. We investigated this effect separately for each of 3 exposure variables that were defined on the basis of 3 cutpoints delineating “high” ozone concentrations versus “low” ozone concentrations: 70 ppb, 75 ppb, and 80 ppb, based on various standards that were considered for maximum 1-hour ozone and maximum 8-hour ozone. Note that even though outcome at time t (Y(t)) and exposure at time t − 1 $(A*(t))$ actually are measured during the same quarter, we assume that $A*(t)$ precedes Y(t), which is required for any causal interpretation of the effect (17).

The primary analysis was based on 2 approaches: 1) history-restricted marginal structural models (HRMSM) (21) and 2) history-restricted causal models for realistic individualized exposure rules (HRCMIER) (22).

We did not assume that each spatial/geographic unit was sampled from 1 common distribution P but rather that each unit was sampled from n distinct distributions, Pi, that might have been similar, particularly for those units that were spatially close. Under this assumption, it follows that mutual independence between the random variables Oi, conditional on the exposure regimen, is a reasonable approximation. (See the supplemental material in Moore et al. (17).)

### History-restricted causal models for realistic individualized exposure rules

The HRMSM approach defines causal effects based on the concept of counterfactual outcomes. These are the outcomes that would have been observed if the grids, possibly contrary to fact, had had a particular exposure history of interest—for example, a hypothesized (counterfactual) decrease of X ppb below the observed quarterly average. An HRMSM can be used to study the counterfactual proportion of cardiac mortality in a given quarter t under the hypothetical scenario in which all grids are exposed to a given ozone level a*(t − 1), where a*(t − 1) can be either “high” (i.e., above a standard) or “low” (i.e., below a standard). HRMSM is a model for the distribution of the counterfactual outcomes over a restricted period of time, denoted $Ya*(t−1)(t)$, as opposed to a standard marginal structural model, which is a model for the distribution of the counterfactual outcomes over the entire time history, . Thus, HRMSM are applied in studies in which only part of the exposure history, rather than the entire history, is relevant (23).

Causal inference relies on 2 primary identifiability assumptions that guarantee that there is enough information contained in the observed data to infer the HRMSM parameter. The first assumption is the sequential randomization assumption, which corresponds to the assumption of no unmeasured confounding. To address this untestable assumption, we evaluated a very large number of variables that have been considered to be confounders in many such studies. The second assumption is the ETA assumption (also called the positivity assumption), which requires that each grid in the target population has a positive probability, regardless of its covariate history, of receiving all of the possible static interventions (i.e., the ozone concentration in each grid has a positive probability of being above or below the standard) at each point in time. The estimators of HRMSM parameters can suffer from severe bias in the presence of theoretical or practical ETA violations (24). Practical violations of this assumption can be tested by checking for very low estimated probabilities of high or low ozone concentrations based on the exposure mechanism model—that is, the conditional probability of exposure, given measured confounders. To mitigate the ETA violation that almost certainly occurs with continuous exposure variables, we consider the discrete exposure variable indicating high ozone concentrations versus low concentrations based on the standards noted previously. Despite discretization, violations of the ETA assumption commonly occur.

Ad hoc methods that have been proposed for addressing violations of the ETA assumption do not address the problem directly (25). One appropriate approach is to investigate the population-level effect of realistic interventions that could be introduced (22). “Realistic” individualized exposure rules are set such that static exposures are assigned only to those grids for which static exposure was possible (or realistic), given their covariate history—that is, there is no ETA violation for the dynamic treatment interventions defined by such rules. In regulatory settings, this may be particularly useful, since interest is in the population-level effect of intervention on a target population that is likely to experience the intervention. Inclusion of those subjects (grids) for whom (which) the intervention is not possible may lead to biased estimation of the total population impact.

Consider the binary exposure variable, denoted $A*(t)$, which is equal to 1 if the ozone concentration falls above a particular cutpoint (e.g., regulatory standard) and 0 if it is equal to or below that cutpoint. Now consider the rule that intervenes on $A*(t)$: If the conditional probability that the ozone concentration falls above the cutpoint, given covariate(s), $(P(A*(t)=1|W-(t−1)))$, is less than α, then assign the exposure to 0 (i.e., below the cutpoint); otherwise, assign it to 1. If the conditional probability that $P(A*(t)=0|W-(t−1))$ is less than α, then assign the exposure to 1; otherwise, assign it to 0. Such a rule, $d(a*(t−1)(W))$, is a mapping of the covariate(s) W into a realistic treatment (Appendix, section A).

HRCMIER are models for the marginal distribution of the counterfactuals $Yd(a*(t−1))(W)$ that correspond to the realistic rule. Unlike HRMSM, which allow estimation of causal effects of static interventions (i.e., those interventions in which the entire population experiences the same level of exposure), the parameters of HRCMIER are fully identifiable from the data, even when the ETA assumption for static treatment interventions is violated. HRCMIER also generalize HRMSM; that is, if there are no ETA violations based on the given α level, or similarly if α = 0, then the realistic intervention corresponds with the static intervention (Appendix, section B).

We consider 2 HRCMIER:

(1)

(2)
Model 1 (equation 1) imposes a constant effect of ozone on cardiac mortality over the study period; model 2 (equation 2) allows for the effect of ozone to vary over the study period—that is, in model 1, the effects of counterfactual contrasts are constant over time; in model 2, a given counterfactual contrast has a different effect at different times. Note that time t is represented by quarter number, which refers to the quarters from 1983 to 2000, which we label 0 through 71. The models were fitted with the inverse probability-of-treatment weighting (IPTW) procedure (26) (Appendix, section C). Selection between these 2 HRCMIER was based on the minimum cross-validated empirical risk based on the IPTW loss function (27). The HRMSM—that is, the models for $E(Ya*(t−1))$—also were fitted with the IPTW estimation procedure (Appendix, section D). Although they are not directly comparable to the binary exposure effect estimates, results from the traditional regression approach and the G-computation-based HRMSM—that is, models for $E(Ya(t−1))$, treating ozone as a continuous variable $(A(t))$—are provided in Web Table 3.

Since the consistency of the IPTW estimate relies on consistent estimation of the parameters of the model for the exposure mechanism, a data-adaptive model selection algorithm, namely the deletion/substitution/addition algorithm, was applied (28). This is a data-adaptive model selection procedure based on cross-validation that searches through a large space of polynomial models and was applied by Moore et al. (17). (Data adaptive estimation techniques based on cross-validation are the most flexible approaches to model selection and have been demonstrated theoretically to be superior to other, less aggressive approaches (29).) The variables associated with both ozone and cardiac mortality (i.e., putative confounders; Web Table 4) were candidates for the algorithm for both sexes. We applied the bootstrap procedure (10,000 iterations) to obtain P values for the coefficients in the models.

## RESULTS

Selected characteristics of the population for quarters 2 and 3 are presented in Table 1 (see Web Table 5 for the complete list). Ozone concentrations declined steadily throughout the study period (Web Figure 2). Median 1-hour and 8-hour daily maximum ozone levels (which were highly correlated: r = 0.99) for quarters 2 and 3 declined across all grids (Web Table 1). Substantial declines were seen for the other pollutants as well (supplemental material in Moore et al. (17)). Nonetheless, for quarters 2 and 3 and the years 1983–2000, 47.5% and 13.2% of the quarterly, average 1-hour maximum ozone concentrations exceeded the levels of the California and federal daily 1-hour standards, respectively. From 1980 to 2000, 1-hour maximum ozone concentrations showed moderate correlation with PM10 (r = 0.52) and little correlation with the other pollutants (all r’s < 0.16) (17).

Table 1.

Selected Demographic Characteristics of Participants Aged 56 Years or More, South Coast Air Basin, California, 1983–2000a

 Variable Spatial Grid Values Median (Interquartile Rangeb) Range Total population, no. Males (46.8%, 38.1–100.0)c 989,000 (938,000–1,049,000) 895,400–1,102,000 Females (53.2%, 0.0–61.9) 1,285,000 (1,229,000–1,350,000) 1,184,000–1,407,000 Race/ethnicity, % Hispanic 15.3 (9.6–24.7) 0–78.9 White 74.6 (60.3–83.2) 1.9–100 African-American 2.2 (1–5.1) 0–49.8 Asian 3.1 (1.1–6.6) 0–31.6 Other 1.6 (0.9–2.8) 0–33.4 Born in California, % 51.4 (45.6–57) 22.4–76 US citizen born in different state, % 34.3 (28.9–40.1) 6.5–60.8 Foreign-born, % 11.2 (7.4–18.9) 0–55.6 Residence in same house 5 years prior, % 43.8 (37.3–49.9) 9.2–71.6 Residence in other state 5 years prior, % 43.8 (37.3–49.9) 9.2–71.6 Graduated from high school, % 24.6 (20.0–28.6) 6–43.2 Income below poverty level, % 9.5 (6.0–13.2) 0–38.1
 Variable Spatial Grid Values Median (Interquartile Rangeb) Range Total population, no. Males (46.8%, 38.1–100.0)c 989,000 (938,000–1,049,000) 895,400–1,102,000 Females (53.2%, 0.0–61.9) 1,285,000 (1,229,000–1,350,000) 1,184,000–1,407,000 Race/ethnicity, % Hispanic 15.3 (9.6–24.7) 0–78.9 White 74.6 (60.3–83.2) 1.9–100 African-American 2.2 (1–5.1) 0–49.8 Asian 3.1 (1.1–6.6) 0–31.6 Other 1.6 (0.9–2.8) 0–33.4 Born in California, % 51.4 (45.6–57) 22.4–76 US citizen born in different state, % 34.3 (28.9–40.1) 6.5–60.8 Foreign-born, % 11.2 (7.4–18.9) 0–55.6 Residence in same house 5 years prior, % 43.8 (37.3–49.9) 9.2–71.6 Residence in other state 5 years prior, % 43.8 (37.3–49.9) 9.2–71.6 Graduated from high school, % 24.6 (20.0–28.6) 6–43.2 Income below poverty level, % 9.5 (6.0–13.2) 0–38.1
a

See Web Appendix (http://aje.oxfordjournals.org/) for full listing.

b

25th–75th percentiles.

c

Median percentage, range.

Cardiac mortality decreased over time for both males and females (Web Figure 3). Across the study period, the median mortality proportions from acute cardiac events were 5.6 per 10,000 population (interquartile range, 1.1/10,000–1.4/10,000) and 4.8 per 10,000 population (interquartile range, 1.1/10,000–1.3/10,000) for males and females, respectively.

We estimated the exposure mechanism model with a logistic regression model, selected by means of the deletion/substitution/addition algorithm and including the following variables: current quarter's ozone concentrations for both sexes, previous quarter's 24-hour average carbon monoxide concentration for females, grid-specific average 24-hour temperature for quarter, time, grid-specific percentage of white persons in quarter, grid-specific median household income in quarter, grid-specific percentage of foreign-born persons in quarter, grid-specific percentage of persons who had graduated from high school in quarter, grid-specific percentage of persons born in a state other than California in quarter, and grid-specific percentage of Hispanic persons in quarter. Comparison of the HRMSMs based on the empirical risk showed the model that included time (model 2) to have the better fit. Therefore, all subsequent results are based on model 2.

As expected with continuous exposure variables, the plot of the predicted ozone levels versus the observed levels showed that the relation was nearly linear, indicating an ETA violation (Web Figure 4). To investigate ETA violations based on the discrete high-ozone versus low-ozone variable, we set the cutpoint for discretizing exposure to high levels of quarterly average maximum 1-hour concentrations—90 ppb and 110 ppb. As Table 2 shows, over 60% of grids with observed ozone levels above these cutpoints would be very unlikely to achieve quarterly ozone concentrations below the respective cutpoints, based on the data. We identified the specific grids that would have large numbers (over 50% of the observed quarters) of ETA violations for various exposure rules (Web Figure 4). Almost all grids which had low probabilities (α ≤ 0.1) of quarterly average 1-hour maxima above 80 ppb or below 70 ppb were on or near the coast or inland, respectively.

Table 2.

Distribution of Grids That Had a Very Low Probability of Moving From “High” Ozone Concentrations to “Low” Ozone Concentrations on the Basis of Various Quarterly Average 1-Hour Ozone Cutpoints, South Coast Air Basin, California, 1983–2000

 Proposed Cutpoint for Ozone Concentrations 90 ppb 110 ppb No. % 75th Percentile No. % 75th Percentile No. and % of grids above cutpoint (“high” ozone concentrations) 3,319 47 1,178 26 75th percentile for grids below cutpoint (“low” ozone concentrations), ppb 80.9 92.5 % of grids above cutpoint that had a probability of ozone concentrations <75th percentile of “low” grids, given covariates, less than α = 0.1 65 61
 Proposed Cutpoint for Ozone Concentrations 90 ppb 110 ppb No. % 75th Percentile No. % 75th Percentile No. and % of grids above cutpoint (“high” ozone concentrations) 3,319 47 1,178 26 75th percentile for grids below cutpoint (“low” ozone concentrations), ppb 80.9 92.5 % of grids above cutpoint that had a probability of ozone concentrations <75th percentile of “low” grids, given covariates, less than α = 0.1 65 61

The HRMSM analysis, which investigates the effect of all grids’ moving from levels above the cutpoint to levels below the cutpoint, estimates the effect based on extrapolation that may not be warranted (bias), since it assumes that all grids can experience such an intervention. We applied the HRCMIER to estimate the effect of 3 discrete ozone variables (cutpoints of 70 ppb, 75 ppb, and 80 ppb), based on 1-hour average maximum quarterly ozone concentrations. The results were strikingly different from those obtained with the HRMSM (Table 3, top). The HRCMIER results indicated that there was no consistent evidence that, if contrary to fact, reductions in ozone concentrations from levels above any of these cutpoints to levels below these cutpoints would lead to decreases in cardiac mortality (Table 3, bottom). The only significant term was the “quarter” term, which was negative and was consistent with the observed decline in cardiac deaths over time. In comparison, the HRMSM results indicated that there was a significant effect of reductions in ozone concentrations (interaction variable and main term) below the cutpoints 75 ppb and 80 ppb for females and below 80 ppb only for males (data not shown). There was no significant effect for ozone discretized at the 70-ppb cutpoint for females and no effect for the 70- and 75-ppb cutpoints for males (data not shown).

Table 3.

Coefficient Estimates for Quarterly Average 1-Hour Maximum Ozone Concentration Among Persons Aged 56 Years or More From Model 2, Obtained Using 3 Different Cutpoints, South Coast Air Basin, California, 1983–2000a

 Cutpoint 70 ppb 75 ppb 80 ppb Coefficient (SE) P Value Coefficient (SE) P Value Coefficient (SE) P Value History-restricted marginal structural models Females Intercept 3.2e-04 (5.6e-04) 4.4e-01 3.7e-04 (2.7e-04) 7.8e-02 5.6e-04 (1.5e-04) 1.9e-03 Quarter (t − 1) 1-hour maximum ozone 1.0e-03 (5.7e-04) 5.3e-02 9.9e-04 (3.0e-04) 1.1e-02 8.2e-04 (2.0e-04) 7.0e-04 Quarter no. 3.8e-06 (1.6e-05) 7.7e-01 4.3e-06 (6.5e-06) 3.3e-01 1.7e-06 (3.2e-06) 6.2e-01 Interaction between the above 2 factors −1.5e-05 (1.6e-05) 1.7e-01 −1.6e-05 (7.0e-06) 2.8e-02 −1.3e-05 (4.3e-06) 4.1e-03 Males Intercept 1.2e-03 (9.5e-04) 1.2e-01 5.5e-04 (6.0e-04) 1.5e-01 5.9e-04 (3.3e-04) 6.7e-02 Quarter (t − 1) 1-hour maximum ozone 4.4e-04 (9.7e-04) 6.3e-01 1.1e-03 (6.2e-04) 7.0e-02 1.1e-03 (3.7e-04) 1.8e-02 Quarter no. −1.4e-05 (2.1e-05) 4.7e-01 1.8e-06 (1.2e-05) 9.4e-01 1.1e-06 (6.3e-06) 8.8e-01 Interaction between the above 2 factors −1.6e-06 (2.1e-05) 9.3e-01 −1.7e-05 (1.2e-05) 9.7e-02 −1.6e-05 (7.0e-06) 3.3e-02 History-restricted marginal structural models for realistic individualized exposure rules Females Intercept 1.4e-03 (1.7e-04) 0.0e+00 1.3e-03 (1.7e-04) 0.0e+00 1.3e-03 (1.6e-04) 0.0e+00 Quarter (t − 1) 1-hour maximum ozone −3.4e-05 (1.2e-04) 7.9e-01 2.7e-05 (1.1e-04) 8.1e-01 3.3e-05 (1.1e-04) 7.6e-01 Quarter no. −1.3e-05 (2.6e-06) 5.7e-07 −1.3e-05 (2.6e-06) 5.7e-07 −1.3e-05 (2.6e-06) 5.7e-07 Interaction between the above 2 factors 2.4e-06 (2.3e-06) 2.9e-01 1.6e-06 (2.1e-06) 4.4e-01 2.3e-06 (2.1e-06) 2.7e-01 Males Intercept 1.7e-03 (2.8e-04) 0.0e+00 1.6e-03 (2.4e-04) 0.0e+00 1.6e-03 (2.2e-04) 0.0e+00 Quarter (t − 1) 1-hour maximum ozone −1.1e-04 (2.5e-04) 6.9e-01 −9.0e-07 (2.1e-04) 1.0e+00 1.3e-05 (1.9e-04) 9.5e-01 Quarter no. −1.9e-05 (4.3e-06) 1.0e-04 −1.7e-05 (3.7e-06) 1.0e-04 −1.7e-05 (3.5e-06) 1.2e-06 Interaction between the above 2 factors 3.8e-06 (4.2e-06) 3.5e-01 2.2e-06 (3.6e-06) 5.5e-01 2.6e-06 (3.4e-06) 4.6e-01
 Cutpoint 70 ppb 75 ppb 80 ppb Coefficient (SE) P Value Coefficient (SE) P Value Coefficient (SE) P Value History-restricted marginal structural models Females Intercept 3.2e-04 (5.6e-04) 4.4e-01 3.7e-04 (2.7e-04) 7.8e-02 5.6e-04 (1.5e-04) 1.9e-03 Quarter (t − 1) 1-hour maximum ozone 1.0e-03 (5.7e-04) 5.3e-02 9.9e-04 (3.0e-04) 1.1e-02 8.2e-04 (2.0e-04) 7.0e-04 Quarter no. 3.8e-06 (1.6e-05) 7.7e-01 4.3e-06 (6.5e-06) 3.3e-01 1.7e-06 (3.2e-06) 6.2e-01 Interaction between the above 2 factors −1.5e-05 (1.6e-05) 1.7e-01 −1.6e-05 (7.0e-06) 2.8e-02 −1.3e-05 (4.3e-06) 4.1e-03 Males Intercept 1.2e-03 (9.5e-04) 1.2e-01 5.5e-04 (6.0e-04) 1.5e-01 5.9e-04 (3.3e-04) 6.7e-02 Quarter (t − 1) 1-hour maximum ozone 4.4e-04 (9.7e-04) 6.3e-01 1.1e-03 (6.2e-04) 7.0e-02 1.1e-03 (3.7e-04) 1.8e-02 Quarter no. −1.4e-05 (2.1e-05) 4.7e-01 1.8e-06 (1.2e-05) 9.4e-01 1.1e-06 (6.3e-06) 8.8e-01 Interaction between the above 2 factors −1.6e-06 (2.1e-05) 9.3e-01 −1.7e-05 (1.2e-05) 9.7e-02 −1.6e-05 (7.0e-06) 3.3e-02 History-restricted marginal structural models for realistic individualized exposure rules Females Intercept 1.4e-03 (1.7e-04) 0.0e+00 1.3e-03 (1.7e-04) 0.0e+00 1.3e-03 (1.6e-04) 0.0e+00 Quarter (t − 1) 1-hour maximum ozone −3.4e-05 (1.2e-04) 7.9e-01 2.7e-05 (1.1e-04) 8.1e-01 3.3e-05 (1.1e-04) 7.6e-01 Quarter no. −1.3e-05 (2.6e-06) 5.7e-07 −1.3e-05 (2.6e-06) 5.7e-07 −1.3e-05 (2.6e-06) 5.7e-07 Interaction between the above 2 factors 2.4e-06 (2.3e-06) 2.9e-01 1.6e-06 (2.1e-06) 4.4e-01 2.3e-06 (2.1e-06) 2.7e-01 Males Intercept 1.7e-03 (2.8e-04) 0.0e+00 1.6e-03 (2.4e-04) 0.0e+00 1.6e-03 (2.2e-04) 0.0e+00 Quarter (t − 1) 1-hour maximum ozone −1.1e-04 (2.5e-04) 6.9e-01 −9.0e-07 (2.1e-04) 1.0e+00 1.3e-05 (1.9e-04) 9.5e-01 Quarter no. −1.9e-05 (4.3e-06) 1.0e-04 −1.7e-05 (3.7e-06) 1.0e-04 −1.7e-05 (3.5e-06) 1.2e-06 Interaction between the above 2 factors 3.8e-06 (4.2e-06) 3.5e-01 2.2e-06 (3.6e-06) 5.5e-01 2.6e-06 (3.4e-06) 4.6e-01

Abbreviation: SE, standard error.

a

Covariate adjustment was handled through the treatment model for quarterly ozone concentrations—that is, the model for $(P(A*(t)=1|W-(t−1))$ (see Materials and Methods).

Since the current National Ambient Air Quality Standard for ozone is based on an 8-hour maximum, we carried out a similar analysis with the 8-hour maximum used as the quarterly exposure. The 1-hour ozone standard was replaced by the 8-hour standard in 1996 (30). Therefore, we carried out an analysis based on data from 1991–2000 to estimate what effect a change to the 8-hour standard might have had on estimated ozone-associated cardiac mortality.

On the basis of the 8-hour maximum HRCMIER analysis from 1991–2000, there was evidence of a declining effect of quarterly ozone concentrations on cardiac mortality for both sexes (Table 4). For all years through quarter 2 of 1998, for all cutpoints and both sexes, the fit of the HRCMIER estimated a smaller mortality reduction than did the estimate based on the HRMSM fit (for all cutpoints and quarters, see Web Table 6 and Web Table 7). By the year 1998, the HRCMIER estimates indicated that none of the interventions to lower quarterly ozone below each cutpoint would result in reductions in cardiac mortality for either sex. In contrast, the HRMSM approach continued to show small reductions for females through 1999 for all cutpoints; for males, the only estimated reduction was for the 80-ppb cutpoint (Table 5; for complete data, see Web Tables 6 and 7).

Table 4.

Coefficient Estimates for Quarterly Average 8-Hour Maximum Ozone Concentration Among Persons Aged 56 Years or More From Model 2, Obtained Using 3 Different Cutpoints, South Coast Air Basin, 1991–2000a

 Cutpoint 70 ppb 75 ppb 80 ppb Coefficient (SE) P Value Coefficient (SE) P Value Coefficient (SE) P Value History-restricted marginal structural models Females Intercept 6.2e-04 (2.0e-04) 2.5e-03 7.1e-04 (1.5e-04) 0.0e+00 8.7e-04 (1.4e-04) 0.0e+00 Quarter (t − 1) 1-hour maximum ozone 7.7e-04 (2.4e-04) 1.5e-03 6.5e-04 (2.4e-04) 7.3e-03 4.2e-04 (2.5e-04) 9.7e-02 Quarter no. −1.3e-07 (3.4e-06) 9.7e-01 −3.4e-07 (2.6e-06) 9.0e-01 −3.2e-06 (2.3e-06) 1.6e-01 Interaction between the above 2 factors −1.0e-05 (4.3e-06) 1.6e-02 −9.1e-06 (4.7e-06) 5.3e-02 −4.7e-06 (5.0e-06) 3.4e-01 Males Intercept 8.2e-04 (1.8e-04) 0.0e+00 8.5e-04 (1.8e-04) 0.0e+00 1.0e-03 (1.7e-04) 0.0e+00 Quarter (t − 1) 1-hour maximum ozone 8.6e-04 (2.5e-04) 9.0e-04 9.7e-04 (3.2e-04) 2.8e-03 7.1e-04 (3.3e-04) 3.0e-02 Quarter no. −2.5e-06 (2.8e-06) 3.8e-01 −2.0e-06 (3.2e-06) 5.4e-01 −5.0e-06 (2.9e-06) 7.8e-02 Interaction between the above 2 factors −1.3e-05 (4.0e-06) 2.0e-03 −1.5e-05 (5.6e-06) 6.1e-03 −1.0e-05 (6.0e-06) 9.2e-02 History-restricted marginal structural models for realistic individualized exposure rules Females Intercept 8.5e-04 (1.5e-04) 0.0e+00 7.1e-04 (1.5e-04) 0.0e+00 8.5e-04 (1.4e-04) 0.0e+00 Quarter (t − 1) 1-hour maximum ozone 6.1e-04 (1.6e-04) 1.0e-04 7.7e-04 (2.0e-04) 2.0e-04 5.9e-04 (2.1e-04) 5.7e-03 Quarter no. −3.6e-06 (2.4e-06) 1.2e-01 −6.9e-07 (2.5e-06) 7.9e-01 −3.0e-06 (2.4e-06) 1.9e-01 Interaction between the above 2 factors −8.8e-06 (2.6e-06) 1.1e-03 −1.2e-05 (3.3e-06) 3.0e-04 −8.9e-06 (3.2e-06) 4.9e-03 Males Intercept 9.5e-04 (1.8e-04) 0.0e+00 8.7e-04 (1.8e-04) 0.0e+00 1.0e-03 (1.7e-04) 0.0e+00 Quarter (t − 1) 1-hour maximum ozone 8.4e-04 (2.3e-04) 4.0e-04 1.0e-03 (3.1e-04) 4.0e-04 8.3e-04 (3.0e-04) 4.9e-03 Quarter no. −4.5e-06 (2.8e-06) 1.0e-01 −2.4e-06 (3.1e-06) 4.5e-01 −4.9e-06 (2.9e-06) 8.0e-02 Interaction between the above 2 factors −1.3e-05 (3.5e-06) 4.0e-04 −1.7e-05 1.1e-03 −1.3e-05 (4.8e-06) 5.2e-03
 Cutpoint 70 ppb 75 ppb 80 ppb Coefficient (SE) P Value Coefficient (SE) P Value Coefficient (SE) P Value History-restricted marginal structural models Females Intercept 6.2e-04 (2.0e-04) 2.5e-03 7.1e-04 (1.5e-04) 0.0e+00 8.7e-04 (1.4e-04) 0.0e+00 Quarter (t − 1) 1-hour maximum ozone 7.7e-04 (2.4e-04) 1.5e-03 6.5e-04 (2.4e-04) 7.3e-03 4.2e-04 (2.5e-04) 9.7e-02 Quarter no. −1.3e-07 (3.4e-06) 9.7e-01 −3.4e-07 (2.6e-06) 9.0e-01 −3.2e-06 (2.3e-06) 1.6e-01 Interaction between the above 2 factors −1.0e-05 (4.3e-06) 1.6e-02 −9.1e-06 (4.7e-06) 5.3e-02 −4.7e-06 (5.0e-06) 3.4e-01 Males Intercept 8.2e-04 (1.8e-04) 0.0e+00 8.5e-04 (1.8e-04) 0.0e+00 1.0e-03 (1.7e-04) 0.0e+00 Quarter (t − 1) 1-hour maximum ozone 8.6e-04 (2.5e-04) 9.0e-04 9.7e-04 (3.2e-04) 2.8e-03 7.1e-04 (3.3e-04) 3.0e-02 Quarter no. −2.5e-06 (2.8e-06) 3.8e-01 −2.0e-06 (3.2e-06) 5.4e-01 −5.0e-06 (2.9e-06) 7.8e-02 Interaction between the above 2 factors −1.3e-05 (4.0e-06) 2.0e-03 −1.5e-05 (5.6e-06) 6.1e-03 −1.0e-05 (6.0e-06) 9.2e-02 History-restricted marginal structural models for realistic individualized exposure rules Females Intercept 8.5e-04 (1.5e-04) 0.0e+00 7.1e-04 (1.5e-04) 0.0e+00 8.5e-04 (1.4e-04) 0.0e+00 Quarter (t − 1) 1-hour maximum ozone 6.1e-04 (1.6e-04) 1.0e-04 7.7e-04 (2.0e-04) 2.0e-04 5.9e-04 (2.1e-04) 5.7e-03 Quarter no. −3.6e-06 (2.4e-06) 1.2e-01 −6.9e-07 (2.5e-06) 7.9e-01 −3.0e-06 (2.4e-06) 1.9e-01 Interaction between the above 2 factors −8.8e-06 (2.6e-06) 1.1e-03 −1.2e-05 (3.3e-06) 3.0e-04 −8.9e-06 (3.2e-06) 4.9e-03 Males Intercept 9.5e-04 (1.8e-04) 0.0e+00 8.7e-04 (1.8e-04) 0.0e+00 1.0e-03 (1.7e-04) 0.0e+00 Quarter (t − 1) 1-hour maximum ozone 8.4e-04 (2.3e-04) 4.0e-04 1.0e-03 (3.1e-04) 4.0e-04 8.3e-04 (3.0e-04) 4.9e-03 Quarter no. −4.5e-06 (2.8e-06) 1.0e-01 −2.4e-06 (3.1e-06) 4.5e-01 −4.9e-06 (2.9e-06) 8.0e-02 Interaction between the above 2 factors −1.3e-05 (3.5e-06) 4.0e-04 −1.7e-05 1.1e-03 −1.3e-05 (4.8e-06) 5.2e-03

Abbreviation: SE, standard error.

a

Covariate adjustment was handled through the treatment model for quarterly ozone concentrations—that is, the model for $(P(A*(t)=1|W-(t−1))$ (see Materials and Methods).

Although the results are not directly comparable, since the HRCMIER results are based on a discrete ozone variable, the results are different from those of the continuous ozone HRMSM analysis, which showed a significant effect of ozone on cardiac-related mortality for both the quarterly average 1-hour and 8-hour maximum concentrations (Web Table 3). We caution against interpreting the results based on continuous ozone because of the severe violation of the ETA assumption.

## DISCUSSION

We observed statistically significant, population-level “causal” associations between 1- and 8-hour maximum high ozone concentrations versus low concentrations (i.e., above standard-based cutpoints vs. below those cutpoints) and cardiac mortality in the South Coast Air Basin over the years 1983–2000; moreover, these associations decreased over time. However, the results were model- and metric-dependent and were sensitive to violations of the ETA assumption. In contrast to the HRMSM approach, application of the HRCMIER that covered the period 1983–2000 failed to show any reduction in cardiac mortality related to decreasing the ozone concentrations from above the cutpoints to below the cutpoints for either the 1-hour maximum quarterly ozone concentrations or the 8-hour concentrations. When the HRCMIER was applied to the quarterly 8-hour maximum from 1991–2000, the period during which the 8-hour maximum ozone standard was in effect, a significant decrease in cardiac mortality was observed, but only through 1997. It then declined to 0% for the remaining 3 years (Web Table 6 and Web Table 8). These latter estimates were based on the expected reductions in mortality over all grids on the basis of intervening only in those grids for which the intervention was possible. These are more conservative estimates than were obtained with the HRMSM or were obtained with a conventional regression model (analysis not shown), both of which assume that all grids could experience ozone concentrations above and below the standard. In fact, approximately two-thirds of grids with observed quarterly 1-hour maximum ozone concentrations greater than 90 ppb or 110 ppb had very low predicted probabilities of experiencing ozone concentrations below the cutpoints (Table 2)—that is, they violated the ETA assumption consequent to meteorologic and pollutant transport phenomena that produce high ozone concentrations.

The HRMSM (estimated by IPTW) and HRCMIER estimates were very similar for the years 1991–2000. However, when all years were included, the HRMSM and HRCMIER results differed. The ETA violations in the years 1991–2000 can be attributed largely to the fact that, for many grids, the conditional probability of experiencing ozone concentrations above the cutpoints was very low, since in these later years the ozone concentrations had declined. In earlier years, there were also ETA violations due to the fact that for some grids, the conditional probability of experiencing ozone concentrations below the cutpoint was low. These violations in earlier years affected the HRCMIER parameter estimates, because those grids with low probabilities of ozone levels falling below the cutpoint had higher cardiac mortality, whereas those grids with low probabilities of ozone levels going above the cutpoint did not. Thus, these 2 approaches give similar inferences when there are few ETA violations, but they differ in the face of the complex ETA violations observed. However, the cost of nonviolation of the ETA assumption for both approaches is that we were not able to analyze ozone as a continuous variable, which would have been desirable in terms of estimation of population exposure-response relations.

Our results are difficult to compare with those of meta-analyses that form the core of recent evidence that increases in ozone concentrations, particularly during the summer months, are associated with increases in mortality (4, 5, 11, 12). All of these studies used short-term lagged effects; 3 of them (4, 11, 12) used time-series data with “cardiovascular disease” as the outcome and 1 (5) used “mortality” as the outcome. Support for a longer averaging time, such as that used in the present study, can be found in the study by Schwartz and Dockery (31), who found that increases in daily ischemic heart disease mortality were associated with increases in PM2.5, with percentages increasing monotonically with smoothing windows of 15, 30, 45, and 60 days.

A second difference relates to the choice of model-fitting approaches for the exposure mechanism model on which the IPTW estimates are based. We used a very flexible model-fitting algorithm that explored a large number of demographic, temporal, meteorologic, and pollution (24-hour concentrations of nitrogen dioxide, carbon monoxide, and PM10) putative confounders over a very large model space, based on the minimum cross-validated empirical risk (28), and allowed for models that were highly nonlinear in the variables and interaction terms.

Another important difference is that we applied a number of different modeling approaches, time periods, and quarterly ozone metrics with the same data structure. Given our interest in population-level (marginal) inferences, it turns out that conventional statistical methods proved inadequate. We considered discrete exposure variables indicating high ozone concentrations versus low concentrations because there were clear violations of the assumption that the conditional probability of experiencing all ozone levels was positive in all 195 spatial units over all 36 quarters of data included in this analysis. The data in Table 2 and Web Figure 4 indicated that this assumption was not warranted and probably has not been warranted in most studies that have included widely divergent spatial units and relatively long time periods (5, 11, 12, 32). Violations of this assumption mean that standard approaches rely on extrapolation outside of the supporting data. HRCMIER addresses this problem through the use of exposure rules that intervene only on units that have a “reasonable” conditional probability of actually having ozone concentrations above and below some level, defining a more “realistic” parameter.

In summary, our approach anticipated the National Research Council recommendation to evaluate longer averaging times with respect to ozone-associated mortality (10). We demonstrated the need for sensitivity analyses that use different modeling approaches based on flexible model-fitting practices. We also showed that failure to consider violations of the ETA assumption can lead to biased estimates and incorrect inference.

### Abbreviations

Abbreviations
• ETA

experimental treatment assignment

• HRCMIER

history-restricted causal models for realistic individualized exposure rules

• HRMSM

history-restricted marginal structural models

• IPTW

inverse probability-of-treatment weighting

• PM2.5

particulate matter ≤2.5 μm in diameter

• PM10

particulate matter ≤10 μm in diameter

Author affiliations: Division of Biostatistics, School of Public Health, University of California, Berkeley, Berkeley, California (Kelly Moore, Romain Neugebauer); Sonoma Technology, Inc., Petaluma, California (Fred Lurmann, Siana Alcorn); Department of Economics and Institute for Economic and Environmental Studies, California State University, Fullerton, California (Jane Hall, Vic Brajer); and Division of Epidemiology, School of Public Health, University of California, Berkeley, Berkeley, California (Ira Tager).

Financial support for this work was provided by the California Air Resources Board (contract 01-346).

The authors acknowledge Dr. Mark van der Laan for his guidance on statistical theory.

Conflict of interest: none declared.

### APPENDIX

#### A. Definition of realistic individualized exposure rule

A formalization of the rule $d(a*)(W-(t−1))$ for the binary exposure variable $A*(t)$ is given by

where α is set to a user-defined value; here we set α = 0.1. For a discussion of the selection of α, see Moore et al. (17). Realistic rules can also be applied to categorical exposure variables (see Bembom et al. (33)). Definition of the rule relies on selection of α. The general idea is to select an α small enough to approach the original static intervention but large enough to avoid extremely large weights for a given grid. On the basis of simulation results, we recommend selecting an α value in the range of 0.05–0.1.

#### B. Causal models for realistic individualized exposure rules

History-restricted causal models for realistic individualized exposure rules (HRCMIER) are models for the marginal distribution of the counterfactuals $Yd(a(t−1))(W)$ which correspond to each realistic rule considered. The effect of intervening on units with rule d(a(t − 1))(W) on the counterfactual outcomes $Yd(a(t−1))(W)$, adjusted for $V⊂W$, $E(Yd(a(t−1))(W)|V)$, can be modeled with the pooled HRCMIER over time:

β0 is the true realistic causal effect of interest.

Consider the HRCMIER (model 1) and history-restricted marginal structural models (HRMSM),

The parameter $β1′$ represents the difference in mean proportions $E(Y1)$ and $E(Y0)$ under the static interventions, where $E(Y1)$ is the mean proportion of the proportion of cardiac-related mortality had all grids been exposed to ozone concentrations above the cutpoint and $E(Y0)$ is the mean proportion of the proportion of cardiac-related mortalities had all grids been exposed to ozone concentrations below the cutpoint. In contrast, the parameter $β1$ represents the difference in mean proportions of cardiac-related mortalities under the dynamic interventions: $E(Yd(1)(W))$ and $E(Yd(0)(W))$.

Like HRMSM parameters, the parameters of the HRCMIER remain interpretable at the population level. Unlike the case with marginal structural models, however, HRCMIER parameters remain fully identifiable from the observed data even when the experimental treatment assignment (ETA) assumption is violated for the static treatment interventions, if the dynamic interventions are realistic such as the one proposed above.

HRCMIER generalize HRMSM: If both $g(1|W-(t−1))>α$ and $g(0|W-(t−1))>α$ for all grids, then the realistic and static interventions would be equivalent and thus the HRMSM and HRCMIER parameters would represent the same effect. In addition, as an increasing number of grids violate the ETA assumption, $β1$ converges to zero because it is increasingly difficult to identify an effect, since so few grids can experience the intervention.

#### C. Exposure mechanism for a binary exposure variable based on a continuous exposure

For inverse probability-of-treatment weighting (IPTW) estimation, for both HRMSM and HRCMIER parameters, an estimate of the exposure mechanism is required. For details on IPTW estimation with HRCMIER, see van der Laan and Petersen (22).

The conditional probability distribution of exposure at time t, given covariates (i.e., the exposure mechanism at time t, assuming that the sequential randomization assumption holds), is denoted by $g(A(t)|W-(t−1))$. The conditional probability distribution of A(t) > θ given covariates, where θ is the cutpoint (e.g., ozone standard), can be obtained by integration of $g(A(t)|W-(t−1))$ over values of A(t) ranging from θ to ∞. Conversely, the conditional probability for A(t) ≤ θ can be obtained by integration over all values of A(t) ranging from −∞ to θ. Since the binary treatment variable $A*(t)$ is set to 1 when A(t) > θ and 0 when A(t) ≤ θ, the treatment mechanism for $A*(t)$ denoted by $gn(A*(t)|W-(t−1))$ can be obtained from the following 2 relations:

To estimate $g(A*(t)|W-(t−1))$, we first estimated $g(A(t)|W-(t−1) )$ using the deletion/substitution/addition model selection algorithm. The above equalities were then applied to obtain estimates for $g(A*(t)=1|W-(t−1))$ and $g(A*(t)=0|W-(t−1))$.

#### D. History-restricted marginal structural models

For the continuous ozone exposure variable (A(t)), the following 2 HRMSM were considered:

(1)

(2)
For the continuous ozone exposure variable $(A*(t))$, the following 2 HRMSM were considered:
(1)

(2)

#### E. Population intervention model

To estimate the proportion of cardiac deaths that could be attributed to ozone, we used the “plug-in” estimator of the population intervention model of Hubbard and van der Laan (34), based on the results from the HRCMIER. The latter estimate is subtracted from the observed, crude mean ($E(Y|t)−E(Ya*(t−1)=0|t)$) and is interpreted as the estimated decrease in population cardiac mortality at time t, if, contrary to fact, the population had been exposed to ozone levels below the specific cutpoint.

## References

1.
Hoek
G
Schwartz
JD
Groot
B
, et al.  .
Effects of ambient particulate matter and ozone on daily mortality in Rotterdam, the Netherlands
Arch Environ Health
,
1997
, vol.
52

6
(pg.
455
-
463
)
2.
Kelsall
JE
Samet
JM
Zeger
SL
, et al.  .
Air pollution and mortality in Philadelphia, 1974–1988
Am J Epidemiol
,
1997
, vol.
146

9
(pg.
750
-
762
)
3.
Kinney
PL
Ozkaynak
H
Associations of daily mortality and air pollution in Los Angeles County
Environ Res.
,
1991
, vol.
54

2
(pg.
99
-
120
)
4.
Bell
ML
McDermott
A
Zeger
SL
, et al.  .
Ozone and short-term mortality in 95 US urban communities, 1987–2000
JAMA
,
2004
, vol.
292

19
(pg.
2372
-
2378
)
5.
Levy
JI
Chemerynski
SM
Sarnat
JA
Ozone exposure and mortality: an empiric Bayes metaregression analysis
Epidemiology
,
2005
, vol.
16

4
(pg.
458
-
468
)
6.
Goldberg
MS
Burnett
RT
Brook
J
, et al.  .
Associations between daily cause-specific mortality and concentrations of ground-level ozone in Montreal, Quebec
Am J Epidemiol
,
2001
, vol.
154

9
(pg.
817
-
826
)
7.
Gryparis
A
Forsberg
B
Katsouyanni
K
, et al.  .
Acute effects of ozone on mortality from the “Air Pollution and Health: a European Approach” project
Am J Respir Crit Care Med
,
2004
, vol.
170

10
(pg.
1080
-
1087
)
8.
Bell
ML
Ebisu
K
Peng
RD
, et al.  .
Seasonal and regional short-term effects of fine particles on hospital admissions in 202 US counties, 1999–2005
Am J Epidemiol
,
2008
, vol.
168

11
(pg.
1301
-
1310
)
9.
Environmental Protection Agency
40 CRF Part 50 National Ambient Air Quality Standards for Ozone, Proposed Rule
Fed Regist
,
2007
, vol.
72

132
(pg.
37818
-
37919
)
10.
Committee on Estimating Mortality Risk Reduction From Decreasing Tropospheric Ozone Exposure, National Research Council
Estimating Mortality Risk Reduction and Economic Benefits from Controlling Ozone Air Pollution
,
2008
Washington, DC
11.
Bell
ML
Dominici
F
Samet
JM
A meta-analysis of time-series studies of ozone and mortality with comparison to the National Morbidity, Mortality, and Air Pollution Study
Epidemiology
,
2005
, vol.
16

4
(pg.
436
-
445
)
12.
Ito
K
De Leon
SF
Lippmann
M
Associations between ozone and daily mortality: analysis and meta-analysis
Epidemiology
,
2005
, vol.
16

4
(pg.
446
-
457
)
13.
Bell
ML
Kim
JY
Dominici
F
Potential confounding of particulate matter on the short-term association between ozone and mortality in multisite time-series studies
Environ Health Perspect
,
2007
, vol.
115

11
(pg.
1591
-
1595
)
14.
Franklin
M
Schwartz
J
The impact of secondary particles on the association between ambient ozone and mortality
Environ Health Perspect
,
2008
, vol.
116

4
(pg.
453
-
458
)
15.
Jerrett
M
Burnett
RT
Pope
CA
III
, et al.  .
Long-term ozone exposure and mortality
N Engl J Med
,
2009
, vol.
360

11
(pg.
1085
-
1095
)
16.
Office of Air Quality Planning and Standards, Environmental Protection Agency
National Air Quality Trends: Status and Trends Through 2007. (Publication no. EPA-454/R-08–006)
,
2008
Research Triangle Park, NC
Environmental Protection Agency
17.
Moore
K
Neugebauer
R
Lurmann
F
, et al.  .
Ambient ozone concentrations cause increased hospitalizations for asthma in children: an 18-year study in Southern California
Environ Health Perspect
,
2008
, vol.
116

8
(pg.
1063
-
1070
)
18.
Environmental Protection Agency
National Ambient Air Quality Standard for Particulate Matter. 40 CFR Part 50
,
1996
Washington, DC
National Archives and Records Administration
19.
National Center for Health Statistics
Public-Use Data Tape Documentation for Multiple Causes of Death for ICD 9 1983–1998 Data [16 volumes]
,
1985
Hyattsville, MD
National Center for Health Statistics
20.
National Center for Health Statistics
Public-Use Data Tape Documentation for Multiple-Cause of Death for ICD-10 1999–2000 Data [2 volumes]
,
2001
Hyattsville, MD
National Center for Health Statistics
21.
Neugebauer
R
van der Laan
M
Nonparametric causal effects based on marginal structural models
J Stat Plan Inference
,
2007
, vol.
137

2
(pg.
419
-
434
)
22.
van der Laan
MJ
Petersen
ML
Causal effect models for realistic treatment and intention to treat rules [electronic article]
Int J Biostat
,
2007
, vol.
3

1
pg.
3

23.
Neugebauer
R
Joffe
MM
Tager
IB
, et al.  .
Causal inference in longitudinal studies with history-restricted marginal structural models [electronic article]
Electron J Stat
,
2007
, vol.
1
(pg.
119
-
154
)
24.
Neugebauer
R
van der Laan
MJ
Why prefer double robust estimators in causal inference?
J Stat Plan Inference
,
2005
, vol.
129

1-2
(pg.
405
-
426
)
25.
Moore
KL
Neugebauer
RS
van der Laan
MJ
, et al.  .
Causal Inference in Epidemiological Studies with Strong Confounding. (U.C. Berkeley Division of Biostatistics Working Paper Series, Working Paper 255)
,
2009
Berkeley, CA
University of California, Berkeley
26.
Cole
SR
Hernán
MA
Constructing inverse probability weights for marginal structural models
Am J Epidemiol
,
2008
, vol.
168

6
(pg.
656
-
664
)
27.
Wang
Y
Bembom
O
van der Laan
M
Data-adaptive estimation of the treatment-specific mean
J Stat Plan Inference
,
2007
, vol.
137

6
(pg.
1871
-
1887
)
28.
Sinisi
SE
van der Laan
MJ
Deletion/substitution/addition algorithm in learning with applications in genomics [electronic article]
Stat Appl Genet Mol Biol.
,
2004
, vol.
3

Article 18
29.
van der Laan
MJ
Dudoit
S
Unified Cross-Validation Methodology for Selection Among Estimators and a General Cross-Validated Adaptive Epsilon-Net Estimator: Finite Sample Oracle Inequalities and Examples. (U.C. Berkeley Division of Biostatistics Working Paper Series, Working Paper 130)
,
2003
Berkeley, CA
University of California, Berkeley
30.
Environmental Protection Agency
Air Quality Criteria for Ozone and Related Photochemical Oxidants. (Publication no. EPA/600/P-93/004cF)
,
1996
Research Triangle Park, NC
Environmental Protection Agency
31.
Schwartz
J
Dockery
DW
Increased mortality in Philadelphia associated with daily air pollution concentrations
Am Rev Respir Dis
,
1992
, vol.
145

3
(pg.
600
-
604
)
32.
Zanobetti
A
Schwartz
J
Mortality displacement in the association of ozone with mortality: an analysis of 48 cities in the United States
Am J Respir Crit Care Med
,
2008
, vol.
177

2
(pg.
184
-
189
)
33.
Bembom
O
van der Laan
MJ
Analyzing sequentially randomized trials based on causal effect models for realistic individualized treatment rules
Stat Med.
,
2008
, vol.
27

19
(pg.
3689
-
3716
)
34.
Hubbard
AE
van der Laan
MJ
Population intervention models in causal inference
Biometrika.
,
2008
, vol.
95

1
(pg.
35
-
47
)