Incorporating nonparametric methods for estimating causal excursion effects in mobile health with zero-inflated count outcomes

In mobile health, tailoring interventions for real-time delivery is of paramount importance. Micro-randomized trials have emerged as the"gold-standard"methodology for developing such interventions. Analyzing data from these trials provides insights into the efficacy of interventions and the potential moderation by specific covariates. The"causal excursion effect", a novel class of causal estimand, addresses these inquiries. Yet, existing research mainly focuses on continuous or binary data, leaving count data largely unexplored. The current work is motivated by the Drink Less micro-randomized trial from the UK, which focuses on a zero-inflated proximal outcome, i.e., the number of screen views in the subsequent hour following the intervention decision point. To be specific, we revisit the concept of causal excursion effect, specifically for zero-inflated count outcomes, and introduce novel estimation approaches that incorporate nonparametric techniques. Bidirectional asymptotics are established for the proposed estimators. Simulation studies are conducted to evaluate the performance of the proposed methods. As an illustration, we also implement these methods to the Drink Less trial data.


Introduction
Mobile health (mHealth) interventions, particularly those utilizing text messages or push notifications, have been developed to promote health-related behaviors.These interventions exhibit significant potential across a wide range of health issues, from alcohol consumption reduction [Song et al., 2019] to physical activity maintenance [Lee et al., 2019].
Advancements in mobile and sensing technologies now facilitate real-time tracking of an individual's internal state and context, offering timely and personalized support [Kumar et al., 2013, Spruijt-Metz andNilsen, 2014].This has given rise to the concept of the just-in-time adaptive intervention (JITAI), a method that tailors treatment in response to the evolving needs and situations of the individual, bearing the goal of delivering the right treatment on the right occasion [Nahum-Shani et al., 2018].For instance, the evening, recognized as a high-risk window for individuals with a history of excessive alcohol consumption, might be ideal for targeted interventions [Day et al., 2014, Bell et al., 2020].
The micro-randomized trial (MRT) has emerged as the touchstone methodology for devising these interventions [Klasnja et al., 2015, Liao et al., 2016].Within the framework of an MRT, participants undergo sequential randomization, aligning them with one of the intervention options across hundreds or even thousands of decision points.Typically, data analysis from micro-randomized trials seeks to answer three critical scientific questions: (1) which interventions can impact the proximal outcome; (2) in which context should the intervention be delivered; and (3) does the treatment effect change with time?The "causal excursion effect", a novel class of causal estimand, provides solutions to these inquiries [Boruvka et al., 2018, Qian et al., 2021a].Notably, these effects can be perceived as a marginal generalization of the treatment "blips" in the structural nested mean model [Robins, 1989[Robins, , 1994]], as they are conditional on a few selected variables instead of all past observed variables.
To estimate the causal excursion effect, one might consider using standard methods such as generalized estimating equation (GEE) approaches [Liang and Zeger, 1986] or random-effects models [Laird and Ware, 1982], especially given the longitudinal nature of data from mHealth studies.Yet, these methods are not guaranteed to give consistent estimates of treatment effects when the data includes both time-varying treatments and time-varying confounders [Sullivan Pepe andAnderson, 1994, Robins andHernan, 2008].Structural nested mean models have been proposed to address this issue by modeling the causal effects of a time-varying treatment on a time-varying outcome [Robins, 1994].
Augmenting this, the G-estimation methodology offers an effective toolset for isolating treatment effects in the presence of time-varying confounding.
In the evolving landscape of MRT-related methodologies, both Boruvka et al. [2018] and Qian et al. [2021a] have innovatively expanded upon the concept, enabling the estimation of causal excursion effects through a weighted and centered least square estimator.However, these studies focus on either continuous or binary data, leaving count data largely unexplored.Our study is motivated by the Drink Less trial where the proximal outcome is the user's engagement with the mobile app, measured by the number of screen views in the subsequent hour following the notification decision point [Bell et al., 2020].Notably, this outcome has a unique characteristic of being zero-inflated as under certain contexts or intervention conditions users may rarely open the app, thereby consistently recording zero screen views.
Consequently, common distributional assumptions such as normal or binomial distribution are no longer tenable for such data.
In this paper, we revisit the concept of causal excursion effects under potentially zero-inflated count outcomes and introduce novel estimation methodologies.As highlighted by Shi and Dempsey [2023], even within MRTs, the accurate collection of randomization probabilities can be compromised due to technical errors.To this end, we propose a class of doubly robust estimating equations applicable to both MRTs and observational mHealth studies, even when treatments are not randomized.Our work yields three primary contributions: 1. We revisit causal excursion effects and the weighted and centered least square estimator, also known as the estimator of the marginal excursion effect (EMEE) from Qian et al. [2021a], focusing on zero-inflated count outcomes.
2. We advocate for nonparametric techniques to estimate nuisance functions, such as generalized additive models.
In particular, we adopt a two-part model for estimating the conditional mean of the proximal outcomes to adapt to the zero-inflation nature.
3. We propose a doubly-robust estimator to estimate the causal excursion effect.The resulting estimator is consistent and asymptotically normal provided the convergence rates of nuisance function estimators are sufficiently fast, known as rate double robustness [Yu et al., 2023].Moreover, we establish bidirectional asymptotics, which require either the sample size or the number of decision points to go to infinity.
The rest of this article is organized as follows.Section 2 provides an overview of the Drink Less trial as a motivating example.The notation and causal estimand are reviewed in Section 3. Section 4 details the incorporation of nonparametric methods for estimating the causal excursion effect.Section 5 provides the data generation procedure, settings, and results of the simulation study.Next, we apply the proposed method to analyzing the Drink Less data in Section 6.
We conclude with a discussion in Section 7.

Motivating example: Drink Less
Our motivating example is from an mHealth app named Drink Less.This app targets the general UK adults who are drinking alcohol at increasing and higher risk levels in reducing hazardous and harmful alcohol consumption [Garnett et al., 2019[Garnett et al., , 2021]].However, a decline in user engagement can notably affect the app's effectiveness.
The MRT design offers a robust experimental framework, producing quality data that can enhance the timing, content, and delivery sequence of mHealth interventions [Klasnja et al., 2015, Liao et al., 2016, Qian et al., 2022, Liu et al., 2023].Participants in MRTs are sequentially randomized to receive varying intervention options, such as receiving a push notification or not.Along with each intervention, data on proximal outcomes and covariates are also gathered, resulting in a collection of time-varying covariates, interventions, and proximal outcomes.The primary objective of MRTs is to inform the design of JITAIs embedded in behavior change apps by assessing the proximal effects of treatments and how covariates might influence those effects.
A 30-day MRT with 349 participants was conducted to improve the push notification strategy in Drink Less [Bell et al., 2020].Participants with a baseline Alcohol Use Disorders Identification Test (AUDIT) score of 8 or higher [Bohn et al., 1995, Saunders et al., 1993], who resided in the UK, were at least 18 years old, and desired to drink less, were recruited into the trial.Every day at 8 p.m., during the trial, participants were randomly given one of three intervention options: no notification, the standard notification, or a message randomly selected from a new message bank.For more details about the messages, see Bell et al. [2020].Each option was assigned according to a static randomization probability of 40%, 30%, and 30%, respectively.The depth of engagement with the app was measured by the total number of screen views between 8 p.m. and 9 p.m. Along with the interventions, data on age, AUDIT score, gender, and other covariates were also recorded.The main goal is to explore the effects of push notifications on user engagement, whether or not these effects differ according to the user's context, and how these effects change over time.
We first examine the distribution of the proximal outcome from the Drink Less trial.Figure 1 displays the histogram and Q-Q plot of the proximal outcome, which is the number of screen views between 8 p.m. and 9 p.m. following no notification, a new notification, or the standard notification.The results suggest that the proximal outcome is not normally distributed and is highly zero-inflated.3 Causal Excursion Effect: A Review

Notations
Consider a setting with longitudinal data spanning T decision points for n participants.For each participant, the treatment assignment at time t is represented by A t .We simplify by considering only binary treatments A t ∈ {0, 1}, where 1 indicates the administration of treatment, and 0 indicates its absence.Let X t denote individual and contextual information or covariates collected after time t − 1 and up to time t.This includes prior treatments, proximal outcomes, and the participant's availability status I t ∈ {0, 1}.The value 1 implies availability for treatment at t, and 0 denotes the opposite.
After providing the treatment at time t, we observe a proximal outcome Y t,∆ .This outcome is usually considered a deterministic function of data collected over a span, or interval, of length ∆.To illustrate, let us consider the Drink Less example from Section 2. Here, the decision time is daily at 8 p.m., with the proximal outcome being screen views between 8 p.m. and 9 p.m., resulting in ∆ = 1.In contrast, other mHealth studies might designate each minute as a decision point, leading to potentially larger ∆ values [Battalio et al., 2021].For the majority of our discussion, we focus on situations where ∆ = 1.We also assume that Y t,1 is count data, and like in the "Drink Less" study, it might exhibit a high degree of zero-inflation.
In our notation, an overbar signifies a sequence of random variables; for instance, Āt encompasses the series The data collected until time t is represented by the history H t = ( Xt , Āt−1 ).

Causal Excursion Effect
In the following, we introduce the potential outcomes framework [Rubin, 1974, Robins, 1989] to define the causal excursion effect.Let X t (ā t−1 ) be the potential covariates that would have been collected, and A t (ā t−1 ) the treatment that would have been assigned, had the participant received the treatment sequence āt−1 ∈ {0, 1} t−1 .Additionally, denote by Y t,1 (ā t ) the potential proximal outcome that would have been observed had that participant received the treatment sequence āt ∈ {0, 1} t .Here, treatments A t and covariates X t are also expressed as potential outcomes of previous treatment to mimic mHealth settings where covariates and treatment assignments can depend on previous treatments.
The potential history at time t is represented by As in Boruvka et al. [2018] and Qian et al. [2021a], we are interested in estimating the causal excursion effect of treatment A t on Y t,1 : Here, S t Āt−1 ∈ R p denotes a vector of potential moderators formed from H t ( Āt−1 ).Specifically, the effect contrasts two excursions from the current treatment protocol at time t and characterizes the treatment effect in the short term.By conditioning on S t Āt−1 and I t Āt−1 = 1, the effect is defined for only individuals available for treatment at time t and is marginalized over variables in H t ( Āt−1 ) that are not in S t ( Āt−1 ).To accommodate possibly zero-inflated count outcomes, we opt for the logarithm of the ratio of, rather than the difference in, the expected outcomes.
When choosing S t Āt−1 = H t Āt−1 , we get the fully conditional version of the causal excursion effect, which can be expressed as The fully conditional effect closely parallels the treatment blips in the structural nested mean model [Robins, 1994].
However, our focus is solely on the immediate effect of a time-varying treatment, not the cumulative effect of all previous treatments.

Identification
To estimate the causal excursion effect from the observed data, we state three fundamental assumptions in causal inference.
Assumption 3.1 (Consistency.).The observed data is equal to the potential outcome under the observed treatment sequence, i.e., for each t ≤ T , Assumption 3.1 connects the potential outcomes with the observed data and states that there is no interference between the observations.Both Assumptions 3.2 and 3.3 are inherently satisfied in MRTs due to the sequential randomization of treatments based on known probabilities.However, in observational studies, Assumption 3.3 is not inherently satisfied.
It is crucial to note that this assumption cannot be verified in purely observational datasets.To evaluate the impact of potential unmeasured confounding, one might employ sensitivity analyses, as recommended by Yang and Lok [2018].
Under these assumptions, the causal excursion effect can be expressed in terms of observed data: The derivation of the identifiability results is essentially the same as in Qian et al. [2021a].For thoroughness, we include the proof for the general case where ∆ > 1 in Appendix A.
Additionally, the causal excursion effect simplifies to when conditioning on the full history H t .In what follows, we begin by detailing the estimation procedure for the fully conditional excursion effect, subsequently extending the approach to instances where S t ̸ = H t .
4 Incorporating Nonparametric Methods for Causal Excursion Effect Estimation

Estimating the Conditional Excursion Effect
In this section, we discuss the estimation procedure for the conditional excursion effect.We start by assuming a linear model for the conditional excursion effect.Specifically, suppose that holds for some unknown p-dimension parameter vector ϕ, where f (•) is a known deterministic function.Next, we will explain the estimation and inference procedure for the unknown parameter ϕ.Throughout, we denote the true value of ϕ by ϕ 0 .
We base the estimation procedure on Robins [1994]'s G-estimation.First, one can construct a variable U t (ϕ) that mimics the potential outcome Y t,1 ( Āt−1 , 0) that would have been observed had the treatment been removed at time t.
Therefore, for t = 1, . . ., T we have where the second equality follows due to Assumption 3.3 of sequential ignorability.Let h(H denote the conditional mean of U t (ϕ) given H t and I t = 1, and let p t (H t ) = Pr(A t = 1|H t ), t = 1, . . ., T denote the treatment randomization probability.Estimation of ϕ in model (5) can thus be based on the following estimating function: for any given h and p t .Building upon argument (6), the expectation of (7) equals zero when ϕ = ϕ 0 under conditions h = h 0 or p t = p t,0 .To denote individual participants, we introduce the subscript i, resulting in n independent and identically distributed copies of the data sequence H T , represented as H 1,T , . . ., H n,T .The estimating equation for ϕ is expressed as: In MRTs with a known randomization probability, estimating the conditional mean h(H t ) remains the primary task.To address this, we express h(H t ) in terms of two estimable conditional means: Here, Remark 1.In the study by Qian et al. [2021a], an estimator for the conditional excursion effect, termed as ECE for binary outcomes, was introduced.Building on the discussion of Kim et al. [2021], this ECE estimator can be extended to count outcomes using the estimating function: , where Kt is defined as Distinctly, their estimator assumes a parametric working model for E (Y t,1 | H t , A t = 0, I t = 1).Instead, we represent the h(H t ) using two nonparametrically estimated conditional means, µ 1 (H t ) and µ 0 (H t ).Semiparametric efficiency can be achieved when the working model is correctly specified.In the simulations, we examine the performance of both the ECE and the ECE-NonP estimator with the above weight Kt .
Remark 2. The ECE-NonP estimator's consistency is contingent upon the correct specification of the treatment effect model ( 5).Yet, the full history H t is typically of high dimensionality, which complicates the accurate specification of excursion effects in the model.Presently, there is limited guidance on how to effectively determine the treatment effect model, which remains a prominent challenge.
Remark 3. The principle of bidirectional asymptotics suggests that as either the sample size n or the number of decision points T approaches infinity, the ECE-NonP estimator achieves consistency and asymptotic normality.However, this should be used with caution.A larger T may not necessarily translate to more information for estimating ϕ when one considers user availability during a trial.Time points where I t = 0 do not contribute to (7) and the value of I t can vary with time.Notably, this is not a problem in studies such as Drink Less, where users are assumed to be always available given the nature of the intervention.

Estimating the Marginal Excursion Effect
In this section, we provide the estimation procedure for the marginal excursion effect, paralleling the methodology from the preceding section.First, we make a parametric assumption concerning the excursion effect.For 1 ≤ t ≤ T , we assume that holds.Within this context, the parameter of interest is β.Throughout, we denote the true value of β by β 0 .
Here, we construct a variable U t = Y t,1 exp −A t S ⊤ t β under model ( 9).Estimation of β can thus be based on the following estimating function: where and weight W t is added to allow for the estimation of marginal causal excursion effects conditional on S t instead of H t , a technique proposed by Boruvka et al. [2018] and Qian et al. [2021a].Here, the weight W t is defined as where the numerical probability pt (S t ) can be chosen arbitrarily as long as it only depends on S t .Intuitively, the weight W t transforms the data distribution where A t is randomized with probability p t (H t ) to a distribution where A t is randomized with probability pt (S t ).The solution to the estimating equation T t=1 m M (H i,t ; β, h, p t ) = 0 yields an estimator for β.
Similar to our previous approach, it is necessary to estimate the nuisance functions and subsequently plug these estimates into (10).We can express the nuisance function h(H t ) utilizing two conditional means: Here, we substitute p t (H t ) by pt (S t ) due to the change of probability introduced by weight W t .In MRTs, we assume that the propensity score p t (H t ) appearing in W t is known.Otherwise, this score can be estimated, either nonparametrically or parametrically, analogous to the estimations of µ 1 and µ 0 .For ease of reference, we label this estimator as the "EMEE-NonP" approach.
Proposition 4.2 provides the bidirectional asymptotics of the EMEE-NonP estimator.The proof is deferred to Appendix C.
Suppose model (9), Assumptions 3.1 -3.3, Assumption 4.1 and some regularity conditions hold.Let ṁβ (H Remark 4. The EMEE method, as proposed in Qian et al. [2021a] for a binary outcome, provides a comprehensive framework to concurrently determine both α and β parameters.The estimating equation is expressed as: Consistency and asymptotic normality of the EMEE estimator were also established.One important property of the estimator is that the consistency is robust to the misspecification of the working model g(H t ) ⊤ α, which is difficult to model due to the high-dimensional nature of H t .In our method, this essential requirement for robustness is embodied in Assumption 4.1.
Remark 5.The EMEE-NonP estimator lacks the double robustness property, which we will demonstrate.Consider the expectation of (10): However, under identifiability assumptions 3.1-3.3,this cannot be ensured.A similar issue arises for the EMEE estimator.As highlighted in Qian et al. [2021b], even if exp g t β in the estimating equation (11) does not generally have conditional expectation zero given H t .This arises because while the causal excursion effect exists as a marginal model, the fully conditional effect, dependent on H t , is not necessarily S ⊤ t β.
Remark 6.For the estimator of β to be consistent, it is imperative that the treatment effect model as defined in (9) is correctly specified.Given that S t consists of summary variables chosen from H t by the researcher, this is an easier task compared with modeling the conditional excursion effect.Yet, by conditioning on only part of the history, i.e., S t , the marginal excursion effect depends on the randomization probability of past treatment assignments Āt−1 [Guo et al., 2021, Zhang and Laber, 2021, Qian et al., 2021b].Therefore, any interpretation of the causal excursion effect must be contextualized within the current treatment protocol.

A Doubly-Robust EMEE-NonP Estimator
The preceding EMEE-NonP estimator lacks double robustness, which means that it solely relies on known or correctly specified randomization probabilities.This makes it difficult to extend to observational mHealth studies.In this section we provide a doubly robust estimator, referred to as the DR-EMEE-NonP estimator.
The modified estimating function is expressed as where Below we make a new assumption about the convergence rate of nuisance functions and establish the bidirectional asymptotics for DR-EMEE-NonP in Proposition 4.3.The proof is deferred to Appendix D.
The idea of forming a doubly robust estimator originated with the research of Scharfstein et al. outcomes.This work distinguishes itself in two primary ways.Firstly, our approach centers on estimating the causal excursion effect for zero-inflated count outcomes-an issue previously unexamined.Secondly, we establish bidirectional asymptotics for the DR-EMEE-NonP estimator, necessitating that either the sample size or the number of decision points approaches infinity.

Extension to Multi-Category Treatment
In the previous sections, our discussions predominantly focused on binary treatments.Notably, the proposed methods can be readily adapted to scenarios with multiple treatment categories, a case exemplified by the Drink Less study.
Consider a scenario with K treatment options.To denote each specific treatment option at time t, one can employ dummy variables, denoted as ).Specifically, the configuration where A t,k = 1 (and all other elements are 0) represents the selection of the kth treatment.Conversely, a value of A t = 0 indicates the absence of any treatment.
Illustratively, the model for the causal excursion effect can be expressed as: Within this equation, (β 1 , • • • , β K ) represents the set of parameter vectors yet to be determined, each denoting the causal excursion effect of the corresponding treatment.
When transitioning to multi-treatment settings, certain modifications to the estimating equation are imperative.The modification of DR-EMEE-NonP and EMEE-NonP involves representing the nuisance parameter h(H t ) with K conditional means, which is where In addition, the weight W t in (10) should be modified as Here, pt,k denotes the numerical probability for each treatment k, and p t,k denotes the randomization probability specified by the treatment protocol corresponding to treatment k.Other estimators can be similarly modified, we thus omit the details.It is crucial to note that reliable estimation of these parameters requires sufficient data, either a large sample size or a sufficiently long follow-up time.

Simulation Studies
In this section, we conduct extensive simulation experiments to assess the finite-sample performance of the proposed estimators across several scenarios.Specifically, we consider (1) A two-arm MRT with p t (H t ) = expit (−0.5A t−1 + 0.5Z t ), where Z t is a time-varying covariate.
(3) A two-arm MRT with p t (H t ) given by a Thompson sampling (TS) algorithm [Russo et al., 2018] (4) A three-arm MRT with p t,1 In our simulations, the first two scenarios compare seven estimators, including ECE, ECE-NonP, EMEE, EMEE-NonP, DR-EMEE-NonP, and the GEE approach with either an independence working correlation matrix (GEE.IND) or an exchangeable working correlation matrix (GEE.EXCH).Other scenarios focus on the comparison of EMEE, EMEE-NonP, and DR-EMEE-NonP with GEE methods.
Our simulation study builds upon the setup established by Qian et al. [2021a].We adopt a simple setting with ∆ = 1, and all participants are available at all decision points, i.e., I t = 1, t = 1, . . ., T .The time-varying covariate Z t can take three values, 0, 1, and 2, each with an equal probability.We generate the outcome Y t,1 using a zero-inflated negative binomial (NB) model as follows: where r denotes the dispersion parameter.We set r = 1 across all settings.
In Scenarios ( 1) and (3), we set π t,1 = exp(−0.4(Zt + 0.1) + 0.1Z t A t ) and µ t,1 = {2.21Zt=0 + 2.51 Zt=1 +2.41 Zt=2 } exp{A t (0.1 + 0.3Z t )}.Hence, the true conditional causal excursion effect is Here, Z t interacts with the treatment A t , a moderator that impacts the conditional causal excursion effect.We also consider the fully marginal excursion effect, which is We omit the details of the TS algorithm here and defer these to Appendix E.
In this case, the true marginal excursion effect becomes In Scenario (4), we have two types of treatments and an absence of treatment.
Here, we set π t+1 = exp (−0.4(Z t + 0.1) + 0.1Z t A t,1 + 0.1Z t A t,2 ) and µ t+1 = {2.21Zt=0 + 2.51 Zt=1 + 2.41 Zt=2 } e At,1(0.1+0.3Zt)+At,2(0.1+0.1Zt) .As a result, the true conditional causal excursion effect is Further, the fully marginal excursion effect becomes A working model of the form g(H t ) ⊤ α = α 0 + α 1 Z t is used for the logarithm of the expected outcome under no treatment when implementing ECE, EMEE and the GEE methods.For methods ECE-NonP, EMEE-NonP, and DR-EMEE-NonP, the preliminary step involves estimating the nuisance parameters These estimates are subsequently incorporated into the estimating equation ( 7).These parameters are estimated via nonparametric regressions, specifically through the generalized additive model.Given the zero-inflation in the data, we model the conditional mean in two parts.The first part models the probability of attaining value 0, while the second part models the non-zero counts, which is a hurdle model [Hu et al., 2011].Note that, in Scenario (2)-which explores an observational data framework -the objective is to examine the rate double robustness of the proposed DR-EMEE-NonP estimator.Here, p t (H t ) also requires estimation from the data.For all methods, we use the sample proportion for estimating p t (H t ).
The performance measures include estimation bias (Bias), mean estimated standard error (SE), standard deviation (SD), root mean squared error (RMSE), and coverage probability of 95% confidence interval (CP) across 1000 replicates.In the simulation experiments, we set the number of decision points to T = 30, 100, 150 and the sample size to n = 100.
In   11 and 12 in Appendix F, we introduce two types of treatments in addition to an absence of treatment.Nonetheless, the proposed methods outperform the standard GEE methods.

Application to Drink Less Data
The Drink Less MRT engaged 349 participants exhibiting excessive alcohol use over a 30-day duration.The primary objective centered on enhancing user interaction with the mobile application among those identified with hazardous alcohol consumption patterns.Our study evaluates the impact of delivering push notifications on the degree of app engagement, quantified by the count of screen views between 8 p.m. to 9 pm.Due to the form of intervention, participants were available for the intervention at all times, i.e., I t = 1.For each participant, baseline and time-varying covariates including age, gender, employment type, AUDIT score, days since download, and the number of screen views yesterday were collected.These covariates are subsequently employed as control variables in g(H t ).
For the EMEE approach, we adopt a working model of the form g(H t ) ⊤ α = α 0 + α 1 Z t for the logarithm of the expected outcome under no treatment.In EMEE-NonP and DR-EMEE-NonP methods, we leverage two-part generalized additive models to estimate the conditional mean of proximal outcomes.

Primary Analysis
In our primary analysis, we first estimate the marginal causal excursion effect of push notifications exerted on the number of screen views.Adopting an analysis model where S t = 1, the relationship is articulated as: The results are presented in Table 5. Notably, all employed estimators elucidate that the effect is statistically distinct from zero, with EMEE-NonP and DR-EMEE-NonP yielding marginally reduced standard errors.Subsequently, we compare the marginal excursion effect between the standard notification and the notification from a new message bank.
The analysis model has the form Here, A t,1 = 1 denotes the administration of the standard notification, and A t,2 = 1 denotes the administration of the new notification.The findings from this analysis, presented in Table 6, affirm the efficacy of both types of notifications.
Yet, the standard notification has a higher treatment effect.

Secondary Analysis
For our secondary analysis, we limit attention to the effects of providing push notifications on user engagement.
Particularly, we examine the effect moderation by setting S t to variables such as "days since download" and "the number of screen views yesterday".The analytical model is expressed as: The findings from our analysis are summarized in Tables 6 and 7. A standout observation from the EMEE-NonP and DR-EMEE-NonP estimators is the identification of "days since download" as a significant moderator, a phenomenon overlooked by the EMEE estimator.Furthermore, all estimators reveal that "the number of screen views yesterday" does not significantly influence the treatment effect.
Based on our analytical findings, there are several insights.First, the push notifications yielded an improvement in user engagement with the Drink Less app, with the standard notification exhibiting a higher effect.Additionally, over prolonged use, users seem to get habituated to the app, resulting in diminishing treatment effects.This highlights the potential utility of periodically refreshing intervention strategies to maintain user engagement.Lastly, our analysis indicates that the treatment effects remain uninfluenced by the number of screen views from the previous day.

Discussion
The current paper revisits the concept of the causal excursion effect, a contrast between two excursions from the current treatment protocol into the future, particularly focusing on zero-inflated count proximal outcomes.Building upon the multiplicative structural nested mean model, the proposed method provides a coarse description of the excursion effect instead of using a two-part model to separately model the treatment effect on the zero part and the count part.While our work extends the ECE and EMEE methods from Qian et al. [2021a]-originally developed for binary outcomes-to our framework, we also introduce novel estimators that harness nonparametric techniques for estimating nuisance functions.
Notably, within the domain of mHealth, the exploration of zero-inflated count outcomes remains relatively less, with only a few discussions hinting at the extension to count outcomes, but lacking in-depth theoretical and empirical investigation.
We demonstrate the rate double robustness of the ECE-NonP and the DR-EMEE-NonP estimator, which is useful for analyzing observational mHealth data.Beyond this, we establish the consistency and asymptotic normality for ECE-NonP, EMEE-NonP, and DR-EMEE-NonP, requiring either the sample size or the number of decision points to go to infinity, known as bidirectional asymptotics [Yu et al., 2023].
We summarize a few directions for future research.First, the current work primarily emphasizes marginal excursion effects; however, person-specific effects may be helpful for informing decision-making, which requires incorporating random effects into the causal excursion effect.Second, the identification of potential moderators during initial research phases could be useful, given that, as of now, the selection of S t from H t is predominantly governed by the researcher's own judgment.

Appendices
The Supplementary Material is organized as follows.In Section A, the identifiability results of the causal excursion effect are provided.The proofs for Propositions 1, 2, and 3 are presented in Sections B, C, and D, respectively.Section E contains the Thomspon sampling algorithm used in the simulation study.We also provide additional simulation results in Section F.

A Derivation of identifiability result
This section provides the identifiability result of the causal excursion effect under Assumptions 3.1-3.3for ∆ > 1.
Proof.We first assume that the following statement is true.
Notice that where the first equality follows from the law of iterated expectation, the second equality follows from consistency (Assumption 3.1), the third equality follows from sequential ignorability (Assumption 3.3), and the last one follows from statement 13.
Next, we prove the preceding statement by induction.For k = 1, statement 13 holds as we define In what follows we consider the case ∆ ≥ 2. Suppose that statement 13 holds for k which completes the proof.

B Proof of Proposition 1
In this section, we derive the bidirectional asymptotic properties of ECE-NonP.We consider a general observational data framework where treatments A t are not randomly assigned.Building on the three identification assumptions outlined in Section 3, such an observational study can be interpreted as a sequentially randomized experiment.The distinction, however, is in the randomization probabilities p t (H t ), which remain unknown and necessitate data-driven estimation, as highlighted by [Guo et al., 2021, Robins, 1986].In observational studies, the nuisance functions of ECE-NonP include three components: 1.The conditional mean 3. The randomization probability p t = p t (H t ).
For simplicity in notation, the triplet function is defined as η = (µ 1 , µ 0 , p t ), with its true value being η 0 = (µ 1,0 , µ 0,0 , p t,0 ).The estimator of η is denoted by η = (μ 1 , μ0 , pt ).In what follows, we will show that, given the convergence rates specified for the estimators of the nuisance functions under Assumption 4.1, both the consistency and asymptotic normality of the estimator φ can be established.
where g represents an arbitrary function or operator of m C .Similarly, let The Cartesian product of this set is denoted as To establish Proposition 1, we require the following regularity conditions.
We further introduce the subsequent two lemmas for our analysis.
Lemma B.1.Given that Conditions B.2 -B.3 are satisfied, as nT → ∞, the following holds: where ṁϕ (H t ; ϕ, η) = ∂m C (Ht;ϕ,η) where the symbol "⇝" signifies the weak convergence of a stochastic process, and l ∞ (G η0 ) denotes the set of all bounded functions f : process, with its sample paths belonging to the set The proof of Lemma B.1 and B.2 can be found in Yu et al. [2023] and is directly applicable to our context.Below we begin the proof of Proposition 1.
Proof of Proposition 1. Consistency.To establish the consistency of the estimator φ, we aim to demonstrate that By leveraging the triangle inequality and acknowledging that P n m C (H t ; φ, η) = 0, we obtain the following: Now, our goal is to show that the two terms in Equation ( 14) are negligible.Specifically, we must establish Applying Taylor's expansion, we find where the inequality arises from the Cauchy-Schwarz inequality and the fact that Substituting this into the first term of Equation ( 14) yields . Next, we bound the term Pm C (H t ; ϕ 0 , η).Expanding this expression, we get: Utilizing Jensen's inequality and the Cauchy-Schwarz inequality, we can express: where the notation ⪯ indicates that the expression on the right serves as an upper bound for the one on the left, and the last inequality follows from Assumption 4.1.From the preceding derivations, we obtain the following convergence result: In addition, based on Lemma B.2, the scaled difference between the empirical measure and the actual probability measure converges weakly as: Given Assumption 4.1, we know that η P → η 0 in the semimetric space G η0 relative to the metric ∥ • ∥ 2,P .As a result, we establish the joint convergence √ nT We define a mapping s : l ∞ (G η0 ) × G η0 → R p as s(z, η) = z(η) − z (η 0 ).Note that s is continuous at every point (z, η) where η → z(η) is continuous.Given Lemma B.2, we observe that almost all sample paths of Z are continuous on G η0 .
This further implies that s is continuous almost everywhere for the points (Z, η 0 ).Utilizing the continuous-mapping theorem, we have √ nT Taking into account Equations ( 18) and ( 19), we further have Next, we aim to establish that √ nT (P n − P) m C (H t ; ϕ 0 , η 0 ) For a given integer satisfying 1 ≤ g ≤ nT , define i(g) as the quotient of g + T divided by T , and t(g) as the integer satisfying g = {i(g) − 1}T + t(g) and 1 ≤ t(g) ≤ T .With this setup, proving (20) is equivalent to proving nT g=1 Let F 0 = {L 1,1 }.Iteratively, we define the sequence {F g } 1 ≤ g ≤ nT as Recall that E m C H i(g),t(g) ; ϕ 0 , η 0 | F g−1 = 0. Consequently, the left-hand side of (21) comprises a martingale difference sequence with respect to the filtration σ(F g ) g≥1 , where σ(F g ) denotes the σ-algebra generated by F g .
To verify Condition 1, notice that According to Assumption B.3, we have max 1≤g≤nT (nT Hence, Condition 1 holds.
To verify Condition 2, observe that It suffices to show The sequence {M g } g≥1 forms a martingale difference sequence with respect to the filtration {σ (F g )} g≥1 .Given that is bounded for all g, we have (nT This leads us to This proves Condition 2. Hence, Proposition 1 is established.

C Proof of Proposition 2
In the context of the marginal excursion effect, the necessary conditions and lemmas parallel those previously mentioned, According to the law of iterated expectation, we have where the last equality follows from Model (4.9).

D Proof of Proposition 3
In this section, we provide the proof for Proposition 3, i.e., the bidirectional asymptotics for DR-EMEE-NonP.Again, we omit the proof for consistency.For deriving asymptotic normality, we need only to bound the term Pm D (H t ; β 0 , η), and the remaining proof is similar to that of Proposition 1.
where the third equality follows from E[Y t,1 |H t , A t , I t = 1] = µ At (H t ), the sixth equality follows from Model (4.9).

E The Thompson sampling algorithm
This section presents the Thompson sampling algorithm used in the simulation scenario (3).Get I t as a byproduct of step 3.

F Additional simulation results
This section presents additional simulation results under Scenario (3) and Scenario (4).Across all settings, the proposed methods outperform the standard GEE methods in terms of diminished bias and enhanced coverage probability of 95% confidence intervals.

Figure 1 :
Figure 1: Graphical inspection of the distribution and normality check of the number of screen views from 8 p.m. to 9 p.m. after each type of notification.
Table1, we present the simulation results of marginal excursion effects under Scenario (1).A clear trend emerges from these results: as the sample size grows, both standard deviations and root mean squared errors of all methods decline.Notably, the DR-EMEE-NonP, EMEE-NonP, and EMEE techniques consistently display negligible bias across all settings, with the empirical coverage probability aligning closely to the nominal level.In contrast, both ECE-NonP and ECE underperform compared to DR-EMEE-NonP, EMEE-NonP, and EMEE, displaying greater biases.Their empirical coverage probabilities for 95% confidence intervals significantly deviate from the nominal level, primarily due to the misspecification of the treatment effect model when estimating the fully marginal excursion effect.Moreover, Table2displays the simulation results of conditional excursion effects.We can see that methods ECE and ECE-NonP parallel the performances of EMEE, EMEE-NonP, and DR-EMEE-NonP, exhibiting minimal bias and great empirical coverage probabilities.This is attributed to the correct specification of the treatment effect model by including the moderator Z t .Furthermore, GEE methods yield biased results across all settings, as shown in Tables1 -2, with their empirical coverage probabilities diverging notably from expected values.

Table 3
summarizes the simulation results of marginal excursion effects under Scenario (2).Here, only DR-EMEE-NonP performs well, exhibiting minimal bias and impressive empirical coverage probabilities, a virtue of its rate double robustness property, which other methods lack.Yet, in Table4, all methods exhibit excellent performances, attributed to the correct specification of the outcome model and the treatment effect model.For Scenario (2), we employed a Thompson sampling algorithm to determine p t (H t ), and the results are deferred to Tables9 and 10in Appendix F. Such algorithms have gained popularity in MRTs, facilitating timely and tailored interventions as evidenced in the DIAMANTE study [Aguilera et al., 2020].The results closely mirror those of the first scenario.Both the EMEE-NonP and EMEE methods yield comparable results, outpacing the GEE methods in performance.Transitioning to Scenario (3), which is presented in Tables

Table 1 :
Comparison of seven estimators for the fully marginal excursion effect under Scenario (1)

Table 2 :
Comparison of seven estimators of the treatment effect moderation Z t under Scenario (1)

Table 3 :
Comparison of seven estimators for the fully marginal excursion effect under Scenario (2)

Table 4 :
Comparison of seven estimators of the treatment effect moderation Z t under Scenario (2)

Table 5 :
Marginal causal excursion effects in the Drink Less micro-randomized trial

Table 6 :
Marginal excursion effects of providing standard notifications and new notifications in the Drink Less

Table 7 :
Effect moderation of days since download in the Drink Less micro-randomized trial

Table 9 :
Comparison of five estimators of the fully marginal excursion effect under Scenario (3)

Table 10 :
Comparison of five estimators of treatment effect moderation Z t under Scenario (3)

Table 11 :
Comparison of five estimators of the fully marginal excursion effects under Scenario (4)

Table 12 :
Comparison of five estimators of treatment effect moderation Z t under Scenario (4)