Background. Pillbox organizers are inexpensive and easily used; however, their effect on adherence to antiretroviral medications is unknown.
Methods. Data were obtained from an observational cohort of 245 human immunodeficiency virus (HIV)–infected subjects who were observed from 1996 through 2000 in San Francisco, California. Adherence was the primary outcome and was measured using unannounced monthly pill counts. Plasma HIV RNA level was considered as a secondary outcome. Marginal structural models were used to estimate the effect of pillbox organizer use on adherence and viral suppression, adjusting for confounding by CD4+ T cell count, viral load, prior adherence, recreational drug use, demographic characteristics, and current and past treatment.
Results. Pillbox organizer use was estimated to improve adherence by 4.1%–4.5% and was associated with a decrease in viral load of 0.34–0.37 log10 copies/mL and a 14.2%–15.7% higher probability of achieving a viral load ⩽400 copies/mL (odds ratio, 1.8–1.9). All effect estimates were statistically significant.
Conclusion. Pillbox organizers appear to significantly improve adherence to antiretroviral therapy and to improve virologic suppression. We estimate that pillbox organizers may be associated with a cost of ∼$19,000 per quality-adjusted life-year. Pillbox organizers should be a standard intervention to improve adherence to antiretroviral therapy.
Incomplete adherence is the most common cause of suboptimal response to medical therapy . Adherence to antiretroviral treatment among HIV-infected individuals is closely associated with viral suppression [2,3–4], and incomplete adherence is associated with the development of drug-resistant infection, disease progression, and death [5,6–7]. Pillbox organizers (or medisets) are the least expensive and most widely used adherence aid for medical therapy of chronic diseases. However, the extent to which this simple, inexpensive, and widely used aid impacts adherence to antiretroviral therapy is unknown.
We examined the relationship between pillbox organizer use and adherence to HIV antiretroviral therapy in a population of HIV-positive, urban poor individuals in San Francisco. We studied individuals recruited from single-room occupancy hotels, homeless shelters, and free-meal programs because of the common barriers to adherence in this population, such as mental illness and active substance use. Adherence was measured using unannounced pill counts at the participant's usual place of residence, as described elsewhere . This method has a close association with concurrent viral load , electronic pill cap adherence assessment , and progression to AIDS . Unlike electronic pill cap assessment, however, unannounced pill count does not interfere with the use of pillbox organizers and does not require multiple devices to measure adherence to all antiretroviral medications. Furthermore, unannounced pill count results in a more complete count of pills in the participant's possession than does clinic-based pill count, and the unannounced nature of the visit makes it more difficult for participants to empty their pill bottles (or “pill dump”) prior to the assessment.
Because pillbox organizers are a routine part of standard medical care, a randomized, controlled trial would raise ethical challenges. Instead, we relied on observational data to estimate the effect of pillbox organizers on adherence, employing marginal structural models to mimic the results of a randomized trial. Marginal structural models are a well-established statistical tool for estimating causal effects on the basis of observational data [9, 10]. Three distinct marginal structural model estimators are available: G-computation, inverse probability-of-treatment weighted, and double robust. Marginal structural models are based on the statistical estimating function methodology, which maximizes robustness to model misspecification. Concordance in effect size and direction of the 3 estimators adds further to the robustness of conclusions.
Research on Access to Care in the Homeless (REACH) cohort. Participants were identified from the REACH cohort, a systematic sample of HIV-positive adults recruited from San Francisco homeless shelters, free meal programs, and low-income, single-room occupancy hotels [11, 12]. Five hundred and eighty-nine HIV-positive subjects were recruited into the REACH cohort from July 1996 through April 2000. The University of California at San Francisco Committee on Human Subjects Research approved all procedures.
Adherence monitoring. Participants taking ⩾3 antiretroviral medications were invited to participate in adherence monitoring. Over a 12-month period, pill counts were conducted on all antiretroviral medications every 3–6 weeks at the subject's usual place of residence. Pill counts occurred on an unannounced day, as described elsewhere . The pill count adherence measure at each assessment was the difference between the current and previous pill counts divided by the prescribed number of doses for the same period. Thus, pill count adherence estimated the percentage of doses prescribed that were actually consumed and did not reflect variations in adherence to the prescribed timing of doses. Pillbox organizer use was assessed at the first visit each month. Individuals not using pillbox organizers were given electronic medication monitors for the antiretroviral medication with the most frequent doses and the highest pill burden.
Confounding variable assessment. Thirty-day drug use (route, substance, and frequency of use), 30-day alcohol use (quantity and frequency), and homelessness (number of nights living on the street in the past 90 days) were assessed by self-report in quarterly interviews. Confounders considered included prior adherence, CD4+ T cell count, plasma HIV RNA level, demographic data (sex, ethnicity, and age), sexual orientation, past and current antiretroviral treatment characteristics, recreational drug and alcohol use, calendar time, and homelessness. Ethnicity classifications were defined and assigned by participants.
Specimen collection. Blood samples were collected monthly. Plasma was processed and stored at -40°C within 6 h after collection. HIV RNA levels were determined monthly. CD4+ T cell count was determined at the baseline adherence-monitoring visit each month. HIV load was determined using the HIV-1 Amplicor Monitor Version 1.0 Ultra-Sensitive Assay (Roche Molecular Systems).
Statistical analyses. Marginal structural models were used to estimate the difference in mean adherence for a given month that would have been observed between the treatment group (those individuals that used pillbox organizers) and the control group (those individuals that did not use pillbox organizers) if pillbox organizer use had been assigned randomly. Three distinct estimators—inverse probability of treatment weighted (IPTW), G-computation, and double robust—were used to estimate this parameter [13, 14]. The 3 approaches rely on estimation of distinct parts of the likelihood.
G-computation (in the current application) relies on standard multivariable regression of adherence on pillbox organizer use and confounders. IPTW estimation, in contrast, relies on multivariable logistic regression of pillbox organizer use on confounders (the treatment mechanism). This logistic regression model is used to assign each subject a weight inversely proportional to the subject's probability of receiving the treatment (pillbox organizer use or not) that he in fact received. The weighting procedure corrects for nonrandom pillbox organizer use. The double robust estimator makes use of (1) the multivariable regression of adherence on pillbox organizer use and confounders used in G-computation and (2) the multivariable logistic regression of pillbox organizer use on confounders (treatment mechanism) used in IPTW estimation. The double robust estimator, which is based on estimating function methodology , is maximally robust to model misspecification: the estimator remains consistent if either of the regression models are correctly specified.
Time-lagged confounder measurements were used to ensure that confounders occurred before (and, therefore, could not be influenced by) pillbox organizer use. Missing confounder values were imputed by carrying the most recent observation forward. Missing values for pillbox organizer use, adherence, and viral load outcomes were not imputed. All multivariable regressions were fit data-adaptively using the deletion/substitution/addition algorithm  with 5-fold cross validation. Bias in the IPTW estimator because of violation of the experimental treatment assignment assumption  was quantified by drawing parametric bootstrap samples based on the 2 fitted models outlined above. All standard errors were estimated using 200 nonparametric bootstrap samples. Because a single patient could contribute multiple data points (1 data point for each person-month for which pillbox organizer use and adherence were measured), bootstrap sampling was based on each patient, rather than on each data point. Equivalent analyses were also performed to examine 2 alternative outcomes: viral load, and the probability of a viral load ⩽400 copies/mL.
Participant characteristics. Of the 589 subjects observed in the REACH cohort, 372 (63%) received at least 1 month of 3-drug combination antiretroviral therapy and, therefore, contributed to the target population of individuals receiving antiretroviral therapy. Of these subjects, 269 (72%) observed from March 1998 through September 2005 received 3170 total unannounced adherence-monitoring visits at their usual place of residence.
The majority of study participants (58%) were of nonwhite ethnicity, with a high proportion of current injection drug users (32%). At the start of follow-up, most individuals (62%) were receiving protease inhibitor–based therapy, and many subjects (42%) had a history of mono- or dual-nucleoside reverse-transcriptase exposure prior to the initial 3-drug antiretroviral regimen. Over the course of follow-up, 163 (61%) of the subjects used pillbox organizers for at least 1 month; pillbox organizers were used for 43% of the observed person-time (table 1).
Two hundred thirty-seven individuals (observed for a total of 2628 person-months) had measurements of all potential confounders and pillbox organizer use. Very few person-months were missing data on pillbox organizer use (0.4%) or adherence (4%). A total of 2648 viral load measurements were collected at 3170 visits (84% of visits); months for which viral load data were not available were excluded from the analyses examining the effect of pillbox organizers on viral load outcome. Three deaths occurred during follow-up (1 in the pillbox user group and 2 in the non–pillbox user group). Ten subjects (<4%) were lost to follow-up (3 in the pillbox user group and 7 in the non–pillbox user group)
A logistic regression model of the treatment mechanism was fit to implement the IPTW and double robust estimators for both adherence and viral load outcomes. Weights based on the resulting model fit had a mean of 0.995, supporting its correct specification (which implies a mean of 1.0). In this multivariable logistic regression model, pillbox organizer use was more likely to occur among individuals with a higher rate of prior adherence and among women and was more likely to occur during more recent calendar time. Pillbox use was less common among the homeless and among individuals who were either antiretroviral naive or had experienced a large number of antiretroviral drugs.
Adherence and pillbox organizer use. Of the 2628 person-months of follow-up, measurements of adherence were available for 2504 (95%) of the person-months. Mean (±SD) adherence over the course of follow-up was 73% ± 30%, with a range of 0%–100%. In univariate associations, adherence was lower among individuals with higher alcohol use and lower prior adherence. Adherence was also lower among individuals with lower prior CD4+ T cell counts and higher prior viral loads, likely because these covariate values reflect lower prior adherence. Crack users and injection drug users, subjects sleeping on the street or in a shelter, women, and black individuals also had lower adherence. Hispanic ethnicity and more recent calendar date were associated with higher adherence. Prior to adjusting for confounding, pillbox organizer use was associated with a mean 8.8% higher rate of adherence in this population (95% CI, 4.0%–13.6%).
The G-computation estimator was implemented on the basis of data-adaptive multivariable regression of adherence on pillbox organizer use and confounders (table 2). After adjusting for confounding using marginal structural models, pillbox organizer use was estimated to increase adherence by 4.1%–4.5% (table 3). The estimates from the 3 marginal structural model estimators (G-computation, IPTW, and double robust) yielded results that were consistent with each other, and the increase in adherence resulting from pillbox organizer use was statistically significant for the G-computation and double robust estimators and of borderline significance for the IPTW estimator. Simulation results suggested ∼1% relative bias in the IPTW estimator because of violation of the experimental treatment assignment assumption.
Pillbox organizer use and viral suppression. Of 237 subjects (2628 person-months of follow-up), 194 (2227 person-months of follow-up; 85%) had viral load measurements available. Of these subjects, 139 (72%) achieved a mean viral load of <400 copies/mL at least once during follow-up; virological suppression was present during 58% of person-time. Mean adherence among individuals with a viral load ⩽400 copies/mL was 28% lower (95% CI, 23%–32%) than adherence among those with a viral load >400 copies/mL. Adherence was closely associated with mean viral load; a 10% higher adherence was marginally associated with a mean viral load that was 0.25 log10 copies/mL (95% CI, 0.22–0.29 log10 copies/mL) lower for the same month.
Marginal structural models were used to estimate the effect of pillbox organizer use on viral load and on the probability of having a viral load ⩽400 copies/mL. Data-adaptive multivariable regression models were fit regressing both viral load outcomes on pillbox organizer use and confounders (shown for 1 viral load outcome in table 4).
Marginal structural models estimated a 0.34–0.37 log10 copies/mL reduction in viral load because of pillbox organizer use. Pillbox organizer use increased the probability of achieving a viral load ⩽400 copies/mL by 14.2%–15.7% (OR, 1.81–1.91). All 3 estimators of the effect of pillbox organizer use on viral load and on the probability of achieving virologic suppression were statistically significant (table 3). Simulation results suggested that violation of the experimental treatment assignment assumption resulted in <0.5% relative bias in the IPTW estimator for both viral load outcomes.
Our finding that pillbox organizer use improved adherence is supported by the results of a recent trial in which elderly patients were randomized to receive their medications in time-specific blister packs, together with pharmacy-based follow-up . The study found a significant improvement in adherence among the subjects who received the combined intervention; however, it was not possible to determine to what extent the improved adherence was attributable to the use of the medication organizers, as opposed to pharmacy follow-up. The results presented here suggest that medication organizers make an important independent contribution to improved adherence.
Prior findings on the effect of medication organizer use on adherence have been summarized in a systematic review . The authors reviewed the results of 12 randomized trials and concluded that there was a trend towards improved adherence among patients who used a medication organizer. One-half of the trials included in this review showed a statistically significant effect of medication organizer use only. However, as the authors point out, the majority of the studies reviewed had small sample sizes and short periods of follow-up, and thus would not have been able to detect a small-to-moderate–sized effect. Interpretation of the results was further complicated by the diversity of outcome assessments and methodologic heterogeneity. For example, a trial by Huang et al.  that did not conclude a benefit from organizer use may have been limited in its ability to detect such an effect, because it relied on clinic-based pill counts to measure adherence and was based on a small number of volunteers who reported very high adherence levels in both treatment groups. Two additional trials, not included in the systematic review, were similarly limited in their ability to detect differences in adherence because of their small sample size [21, 22] and the inherent imprecision of self-reported adherence  or clinic-based pill count adherence measures . Finally, a single observational study also found a trend towards better adherence among individuals who used adherence aids, such as reminder devices or pillbox organizers .
Our observations suggest that pillbox organizer use is associated with a 4%–5% higher adherence in a predominantly urban poor, HIV-positive population receiving antiretroviral therapy. This estimate was reached by 3 separate statistical approaches controlling for common variables that may confound the effect of pillbox organizer use on adherence. A 4% improvement in adherence is associated with a difference of 0.12 log10 copies/mL in viral load  and an 11% reduction in the risk of progression to clinical AIDS . More-intensive adherence interventions have been associated with a 10%–15% improvement in objectively measured adherence [24,25,26,27–28]. Using the Johns Hopkins cohort, Goldie et al.  estimated that a $100 per month intervention that improved adherence by 10% was associated with a cost-effectiveness ratio of <$50,000 per quality-adjusted life-year. In their analysis, they found that a 14.5% increase in the rate of viral suppression corresponded to a reduction of ∼20% in the monthly treatment failure rate. Figure 1 shows estimates of the cost per quality-adjusted life-year for a range of adherence interventions, with differing monthly costs and differing effectiveness (in terms of an improved rate of viral suppression). To the extent that the REACH cohort and Johns Hopkins cohort are comparable urban poor, HIV-positive populations, we estimate that pillbox organizer use, at $5 per month, should be associated with a cost of ∼$19,000 per quality-adjusted life-year (figure 1).
There are a number of limitations to our study. First, we cannot exclude the possibility that intensive adherence monitoring changed adherence. However, both pillbox users and nonusers received unannounced pill counts, which, therefore, should not be a source of bias in our estimated effect of pillbox use. In addition, the control group (consisting of individuals who were not using pillbox organizers) received electronic pill cap monitoring, which may have altered adherence. Studies to date, however, have been unable to detect an effect of intensive adherence monitoring on adherence behavior . If electronic monitoring in the control group did alter adherence, it would most likely increase adherence in the control group and bias our results to an underestimate of the effect of pillbox organizers on adherence. Second, as in any observational study, we cannot exclude the possibility of unmeasured confounding. However, we controlled for major known confounders of intervention effects on adherence and viral load, using sophisticated data-adaptive regression techniques and cross-validation. Finally, most of the patients in this study received twice-daily regimens with several pills per dose. It is unclear whether pillbox organizers would have the same effect on the simpler regimens that are in use today.
Our study also had several strengths. Data were drawn from a well-studied cohort with very high retention rates and well-characterized, objective adherence measures. Use of unannounced pill counts helped protect against overestimation of adherence as a result of “pill dumping,” and the effect of pillbox organizers on viral load was also assessed. Finally, state-of-the-art data analyses and the use of alternative methods to control for confounding improved the robustness of the findings.
The urban poor and recreational drug users were highly represented in the target population of this study. Although this population has often been regarded as having difficulty with adherence, adherence problems are by no means limited to patients with low socioeconomic status. Therefore, the findings of this study may have broad applications for more-affluent patient populations.
In summary, we found that pillbox organizers are associated with improved adherence to HIV antiretroviral therapy and improved viral suppression in a diverse population with a wide distribution of adherence. These improvements are likely to be associated with favorable cost per quality-adjusted life-year. The successful treatment of HIV infection is similar to that of other chronic diseases, such as diabetes mellitus and hypertension; to prevent disease progression, individuals must maintain high levels of medication adherence to complex therapies, often in the absence of symptoms, for the duration of their lifetimes. Additional work is needed to demonstrate whether pillbox organizers similarly improve adherence to medical therapies other than antiretroviral drugs. However, given the simplicity and low cost of the intervention, clinicians should consider including pillbox organizers in their routine treatment of chronic disease.
We thank Sue J. Goldie of the Harvard School of Public Health (Boston, MA), who, on behalf of the Cost-Effectiveness of Preventing AIDS Complications team, provided both helpful comments and access to their previously published cost-effectiveness results.
Financial support. National Institute of Mental Health (grants 54907 and 63011; R01 GM071397 to M.v.d.L.), the Doris Duke Charitable Foundation (to D.R.B.), and the Howard Hughes Medical Institute (predoctoral fellowship to M.P.). Viral load kits were donated by Roche.
Potential conflicts of interest. All authors: no conflicts.