Temporal Learning Among Prefrontal and Striatal Ensembles

Abstract Behavioral flexibility requires the prefrontal cortex and striatum, but it is unclear if these structures play similar or distinct roles in adapting to novel circumstances. Here, we investigate neuronal ensembles in the medial frontal cortex (MFC) and the dorsomedial striatum (DMS) during one form of behavioral flexibility: learning a new temporal interval. We studied corticostriatal neuronal activity as rodents trained to respond after a 12-s fixed interval (FI12) learned to respond at a shorter 3-s fixed interval (FI3). On FI12 trials, we found that a key form of temporal processing—time-related ramping activity—decreased in the MFC but did not change in the DMS as animals learned to respond at a shorter interval. However, while MFC and DMS ramping was stable with successive days of two-interval performance, temporal decoding by DMS ensembles improved on FI3 trials. Finally, when comparing FI12 versus FI3 trials, we found that more DMS neurons than MFC neurons exhibited differential interval-related activity early in two-interval performance. These data suggest that the MFC and DMS play distinct roles during temporal learning and provide insight into corticostriatal circuits.


Introduction
Behavioral flexibility requires learning to adapt to uncertainty. Two forebrain structures critical for flexibility are the prefrontal cortex and striatum (Fuster 2008;Kehagia et al. 2010). Prefrontal cortical neurons densely innervate the striatum (Gabbott et al. 2005;Wall et al. 2013) and disruptions of either structure profoundly impact the learning of new goals, rules, and strategies (Ragozzino 2007;Hart et al. 2018). Dysfunctional corticostriatal circuits and connectivity are implicated in a range of psychiatric and neurological disorders (Deutch 1993;Shepherd 2013).
However, the relative contributions of prefrontal and striatal networks to behavioral flexibility are unclear, and clarifying their respective roles may provide novel approaches to identifying new biomarkers or treatments for human brain diseases.
One task that provides an ideal window into behavioral flexibility is interval timing, which requires participants to estimate an interval of several seconds via a motor response. Across species, interval timing requires the prefrontal cortex and striatum (Matell and Meck 2004;Coull et al. 2011;Merchant and de Lafuente 2014;Mello et al. 2015;Emmons et al. 2016Emmons et al. , 2017Dallérac et al. 2017). Work from our group and others has shown that both prefrontal and striatal neurons encode temporal information via "time-related ramping" activity-or monotonic changes in firing rate over a temporal interval (Donnelly et al. 2015;Narayanan 2016;Bakhurin et al. 2017;Emmons et al. 2017;Kim et al. 2018;Wang et al. 2018). Our past work suggested that ramping activity in neurons of the medial frontal cortex (MFC) and the dorsomedial striatum (DMS) is very similar, with onethird of neurons in each area exhibiting such activity (Emmons et al. 2017). We have also found that MFC inactivation attenuates DMS ramping (Emmons et al. 2017(Emmons et al. , 2019 and that corticostriatal stimulation is sufficient to recover decreases in DMS ramping caused by MFC inactivation (Emmons et al. 2019). These data suggest that DMS ramping is closely linked to MFC ramping and suggest the hypothesis that MFC and DMS ensembles respond similarly as animals learn new temporal intervals. By contrast, recordings from primate lateral prefrontal cortex and caudate indicate that striatal ensembles encode stimulus-response associations earlier than prefrontal ensembles, leading to the hypothesis that prefrontal and striatal ensembles play distinct roles during learning (Pasupathy and Miller 2005;Histed et al. 2009;Antzoulatos and Miller 2011).
We tested if MFC and DMS ensembles played similar or distinct roles during temporal learning. We recorded MFC and DMS activity in rodents as they learned to respond to a new temporal interval. Animals trained in the temporal context of one 12-s interval learned a new temporal context where two intervalseither 12 s or 3 s-could be presented. We report three main results. First, for 12-s intervals, MFC ramping decreased in the context of two-interval sessions compared to one-interval sessions, whereas DMS ramping did not change. Second, while MFC and DMS ramping was stable over subsequent days of two-interval performance, DMS ensembles improved temporal decoding for 3-s intervals. Finally, comparing activity on 12-s versus 3-s intervals revealed that DMS neurons were more likely to have different activity patterns compared to MFC neurons, particularly on the first day of two-interval performance. These data suggest that the MFC and DMS play distinct roles during temporal learning.

Rodents
All procedures were approved by the University of Iowa IACUC, and all methods were performed in accordance with the relevant guidelines and regulations (protocol #7072039). Seven male Long-Evans rats were trained on the 12-s fixed-interval timing task (FI12) (Emmons et al. 2016(Emmons et al. , 2017. In this task, rats press the lever in anticipation of the end of the interval. We have used fixedinterval timing to study corticostriatal circuits in detail in rodents (Narayanan et al. 2012;Parker et al. 2014;Parker, Ruggiero, et al. 2015;Emmons et al. 2016Emmons et al. , 2017Kim et al. 2017;Parker et al. 2017;Emmons et al. 2019Emmons et al. , 2019 and in humans (Parker, Chen, et al. 2015;Kelley et al. 2018). Rats were motivated by waterrestriction maintained at 85-90% of their free access weight. Food was freely available. In brief, the rats were autoshaped to press a lever for water reward using a fixed-ratio task before being trained on the 12-s fixed-interval timing task (FI12). Trials began with the presentation of a house light, and the first response made after 12 s resulted in the delivery of a water reward, a concurrent click, and termination of the house light (Fig. 1A). Responses made before the interval ended were not reinforced. Trials were separated by a randomly chosen 6-, 8-, 10-, or 12-s intertrial interval. After animals behaved consistently, the MFC and DMS were each implanted with recording electrodes ( Fig. 1B; see below). Animals were then acclimatized to the recording procedures and recordings were made during behavior in the one-interval task (FI12 trials only; Day 0). The following day, an additional 3-s interval (FI3) was added to the task and cued by a light distinct from the one used to indicate FI12. In general, sessions were 60 min. Critically, task stimuli on FI12 trials were identical between one-interval and two-interval sessions. FI12 and FI3 trials were randomly intermixed. Behavior and simultaneous neuronal activity in the MFC and DMS were recorded over the following 3 days of two-interval performance (FI12/FI3 trials on Day 1, Day 2, and Day 3). Data from subsequent two-interval recording sessions in 5/7 rodents were included in prior manuscripts (Emmons et al. 2016(Emmons et al. , 2017.

Surgical and Histological Procedures
Animals were anesthetized using ketamine (100 mg/kg IP) and xylazine (10 mg/kg IP), and a surgical level of anesthesia was maintained using ketamine supplements (10 mg/kg IP). Craniotomies were drilled above the left MFC and left DMS and 4 holes were drilled for skull screws, which were connected to electrode recording arrays via a separate ground wire. Microelectrode arrays were composed of 4 × 4 50-μm stainless steel wires (250 μm between wires and rows). These arrays were positioned in the MFC (coordinates from bregma: AP +3.2, ML ±1.2, DV -3.6 @ 12 • in the anterior plane) and the DMS (coordinates from bregma: AP +0.0, ML ±4.2, DV -3.6 @ 12 • in the posterior or lateral plane) while recording neuronal activity to verify that implantation was in correct brain area. The craniotomy was sealed with cyanoacrylate ("SloZap," Pacer Technologies, Rancho Cucamonga, CA), and the reaction was accelerated by "ZipKicker" (Pacer Technologies) and methyl methacrylate (AM Systems, Port Angeles, WA). Rats recovered for 1 week before being acclimatized to behavioral and recording procedures.
Following these experiments, the rats were anesthetized and sacrificed by injection with 100 mg/kg sodium pentobarbital and transcardially perfused with 4% formalin. Brains were postfixed in a solution of 4% formalin and 20% sucrose before being horizontally sectioned on a freezing microtome. Brain slices were mounted on Superfrost Plus microscope slides and stained for cell bodies using either DAPI or Cresyl violet. Histological reconstruction was completed using postmortem analysis of electrode placement by slide-scanning fluorescent microscopy (Olympus, Center Valley, PA).

Neurophysiological Recordings and Neuronal Analyses
Neuronal ensemble recordings were made using a multielectrode recording system (Plexon, Dallas, TX). In each animal, one electrode without single units was reserved for local referencing, yielding 15 electrodes per animal. Offline Sorter (Plexon) was used to analyze the signals after the experiments and to remove artifacts. Spike activity was analyzed for all cells that fired at rates above 0.1 Hz. Principal component analysis (PCA) and waveform shape were used for spike sorting. Single units were defined as those 1) having a consistent waveform shape, 2) being a separable cluster in PCA space, and 3) having a consistent refractory period of at least 2 ms in interspike interval histograms. As in our past work, putative cortical interneurons were excluded at initial spike-sorting because they were difficult to definitively identify (Narayanan and Laubach 2009;Emmons et al. 2017). Putative DMS medium spiny neurons (MSNs) were further separated from striatal interneurons based on waveform peak-to-trough ratio and rodents performed fixed-interval timing tasks in which a reward was given for the first lever press after a 12-s interval (FI12). Interval start was cued by a house light, motivation was a liquid reward, and presses before interval end were not reinforced. On Day 1, there was a new temporal context with the addition of a second, shorter 3-s interval (FI3) cued by a distinct light. FI3 trials were randomly intermixed with FI12 trials. Recordings were performed for 3 days of two-interval performance (Days 1-3). (B) Animals were implanted with multielectrode recording arrays targeting the MFC and DMS; horizontal sections with approximate AP coordinates shown. (C) MSNs within the DMS were identified based on waveform shape. the half-peak width ( Fig. 1C; Berke, 2011). All neuronal analyses focused on putative pyramidal neurons and MSNs. Counts of recorded neurons are in Table 1, note that the same electrodes are recorded from across days.

Statistics
All data and statistical approaches were reviewed by the Biostatistics, Epidemiology, and Research Design (BERD) Core at the Institute for Clinical and Translational Sciences at the University of Iowa. We used generalized linear-mixed effects models (GLMM; fitglme.m in MATLAB, Natick, MA) where the outcome variable was response time. For FI12 trials, the predictor variable was temporal context (i.e., one-interval sessions on Day 0 when FI12 trials were presented alone, or two-interval sessions on Days 1-3 when FI3 trials were presented along with FI12 trials). For two-interval performance, response time on FI12 or FI3 trials was the outcome variable, and Day (Days 1-3) was the predictor variable. Animal-specific variance was included as a random effect; this explicitly accounts for variability between individual animals. For two-interval trials, single-trial analyses were used to find start times and coefficients of variation for FI3 and FI12 trials (Church et al. 1994). Finally, because we cannot definitively track neurons between days, all statistical analyses assumed that each population of neurons was statistically independent. Analyses of neuronal activity and basic firing properties were carried out using NeuroExplorer (Nex Technologies, Littleton, MA) and custom routines for MATLAB, as described in detail previously (Parker et al. 2014;Emmons et al. 2017Emmons et al. , 2019. Peri-event rasters and peri-event time histograms (PETHs) were constructed around houselight and lever press. As in our past work, neuronal ensemble modulations were quantified via PCA, a data-driven set of orthogonal basis vectors that captures patterns of activity in multivariate neuronal ensembles (Chapin and Nicolelis 1999;Narayanan and Laubach 2009). PCA was calculated from average PETHs computed from kernel-density estimates (ksdensity.m; FI3: bandwidth 0.2; FI12: bandwidth 0.5) and normalized by Z-score. Notably, single neurons can either ramp up or down, and our past work has not identified reliable differences between these populations (Parker et al. 2014;Parker, Ruggiero, et al. 2015;Narayanan 2016;Emmons et al. 2017;Kim et al. 2017). Consequently, we used absolute values of PC1 scores (indicated by |score|) to compare ramping strength across areas and days as in our past work (Parker et al. 2014Emmons et al. 2017;Kim et al. 2017). For time-related ramping, we used GLMMs where PC1 |score| was used as an outcome variable; predictor variables were brain area (MFC vs. DMS) and either temporal context (FI12 on Day 0 vs. FI12/3 sessions on Days 1-3) or day of two-interval performance (Days 1-3). Animal-specific variance was included as a random effect; as with behavioral data, this explicitly accounts for variability between individual rodents. Post hoc testing was performed via estimated marginal means (emmeans in R) with Tukey's method for family-wise comparisons after linear-mixed effects modeling (lmer in R).
We also used GLMMs to analyze neuronal modulations. We calculated trial-by-trial GLMMs for all trials and neurons where the outcome was firing rate binned at 0.1 s, the predictors were area (MFC or DMS) or Day 0 or Day 1, and random effects were responses and neurons. Our past work has demonstrated that GLMMs account for the contribution of lever-press response-related activity to time-related ramping; this analysis appropriately considers variance across neurons and animals (Narayanan and Laubach 2009;Emmons et al. 2017Emmons et al. , 2019. To examine interval-related modulation for all trials and neurons, we used an equation where the outcome was firing rate, the predictors were interval (FI3 or FI12), area, or day, and random effects were lever-press responses and neurons. Poisson distributions were used for all firing-rate models.
We used a naïve Bayesian classifier to examine neuronal ensemble decoding, as we have in our past work (Emmons et al. 2017;Kim et al. 2017). We calculated kernel density estimates (bandwidth: 1.2) of trial-by-trial firing rates from MFC and DMS neurons. To prevent edge effects that might bias classifier performance, we included data from 6 s prior to trial start and 6 s after interval end. We used leave-one-out cross-validation to predict objective time from firing rate within a trial. We evaluated classifier performance by computing the R 2 of objective time and predicted time only for bins during the interval. With perfect classification, the R 2 would approach 1. Classifier performance was compared to ensembles with time-shuffled firing rates. For each area and interval, the performance was quantified via GLMMs of R 2 versus each day.

Results
We studied behavioral flexibility in the MFC and DMS by introducing a new 3-s fixed interval (FI3) to rats after they had been trained on a task with 12-s fixed intervals (FI12; Fig. 1A). On Day 0, animals were presented only with one FI12 interval, but on Days 1, 2, and 3, animals were presented with a new temporal context in which either FI12 or FI3 intervals could be presented. For FI12 trials, response times were significantly longer on Day 0 with only one interval (FI12; 10.14 ± 0.09 s; mean ± SEM) compared to Days 1-3 when the temporal context included two intervals (FI12 or FI3 could be presented, FI12 response times: Day 1: 9.61 ± 0.12 s, Day 2: 9.96 ± 0.13 s, Day 3: 9.31 ± 0.12 s; main effect of context on response time; one-way ANOVA of GLMM: F (7022) = 18.3, P = 0.00002; Fig. 2A). FI12 response times grew shorter over subsequent days of two-interval performance (main effect of Days 1-3 on FI12 response time: one-way ANOVA of GLMM: F (4796) = 7.0, P = 0.0009; FI12 trials only; Fig. 2A). On these days, we also found that FI3 response times shortened over subsequent days of twointerval performance (Day 1: 6.05 ± 0.18 s, Day 2: 5.38 ± 0.15 s, Day 3: 5.01 ± 0.14 s; main effect of Days 1-3 on FI3 response time; one-way ANOVA of GLMMs: F (1741) = 8.4, P = 0.0002; FI3 trials only; Fig. 2B). On the very first day of two-interval performance (Day 1), there was a main effect of FI12 versus FI3 on response time (two-way ANOVA of GLMM: F (2072) = 78.8, P = 2e −18 ) but no reliable effect of trial order within-session on response time (twoway ANOVA of GLMM: F (2072) = 0.9, P = 0.34) and no reliable higher interactions (F (2072) = 1.4, P = 0.23). This analysis suggested that on Day 1, response times did not consistently change over the session for either the FI3 or FI12 trial type.
We analyzed fixed-interval behavior using single-trial analysis, which was developed for peak-interval timing but can be useful to analyze start times during fixed-interval tasks (Church et al. 1994;Emmons et al. 2019). On Day 1, single-trial analysis revealed that animals had earlier start times on FI3 trials compared to FI12 trials (two-way ANOVA of GLMMs of interval and day on start times; main effect of interval: F (43) = 10.4, P = 0.002; Fig. 2C; note that one animal on Day 1 did not have enough FI3 trials for analysis). There was no effect of day or interaction. One indication that timing processes are scalar is that the coefficient of variation (CV-the ratio of the standard deviation of temporal estimates to the mean) is relatively constant at different intervals (Gibbon et al. 1984;Rakitin et al. 1998). Accordingly, we found that during fixed-interval performance, start-time CVs were similar on FI3 and FI12 trials across Days 0-3 (two-way ANOVA of GLMMs of interval and day on CV: main effect of interval: F (43) = 0.2, P = 0.62; Bayes Factor = 5.9; there were no higher effects of day or interaction). These data suggest that start times during fixedinterval timing could exhibit scalar properties (Gibbon et al. 1984) and demonstrate that during FI12 and FI3 trials animals were guiding their responses in time ( Fig. 2D and E).
We recorded neuronal ensembles simultaneously in the MFC and DMS as animals trained on a single interval learned to respond at two intervals (Table 1). As in our past work, we found that neurons in both brain regions exhibited time-dependent ramping, that is, monotonic increases or decreases in firing rate across the interval (Emmons et al., 2017;Fig. 3A and B). We turned to PCA to compare time-dependent ramping activity as animals learned to perform two-interval tasks (Chapin and Nicolelis 1999;Narayanan and Laubach 2009;Emmons et al. 2017). Consistent with our prior work, we found that principal component 1 (PC1) exhibited time-related ramping ( Fig. 3C; Emmons et al., 2017;Kim et al., 2017;Parker et al., 2015b, Parker et al. 2014Zhang et al., 2019). Note that this past work has found no reliable differences between neurons that ramp up or down; hence, we focused on the absolute value of PC1 |score| as a measure of ramping strength (Parker et al. 2014;Narayanan 2016;Emmons et al. 2017).
We compared PC1 |scores| across all areas and days (Days 0-3) via GLMMs on FI12 trials only. Critically for our hypothesis, we found a main effect of temporal context for PC1 |scores| on FI12 trials (one-interval vs. two-interval sessions, or Day 0 vs. Days 1-3; two-way ANOVA of GLMMs: F (472) = 12.2, P = 0.0005; FI12 trials only) and area (MFC vs. DMS; F (472) = 4.5, P = 0.03), as well as an interaction between context and area (F (472) = 6.5, P = 0.01; Fig. 3D and E). Note that this analysis explicitly included animals as a random effect, accounting for variance contributed by each animal. Post hoc testing revealed that MFC PC1 |scores| were stronger for one-interval compared to two-interval sessions for FI12 trials (P = 0.001 Tukey's post hoc; Fig. 3E). However, DMS PC1 |scores| were similar in all sessions (P = 0.70 Tukey's post hoc; Fig. 3F-G). Critically, there was a main effect of context on PC1 even when PCA was calculated on matched responses (one-interval vs. two-interval sessions-two-way ANOVA of GLMMs only on trials with responses between 11-13 s: F (472) = 11.4, P = 0.0008). These data imply that differences in PC1 between one-interval and two-interval sessions were not a function of differences in response rate.
Interestingly, for one-interval sessions on Day 0, PC1 |scores| were not different between MFC and DMS (P = 0.11 Tukey's post hoc). This observation was held for trial-by-trial analysis of firing rate on FI12 trials, which revealed a highly significant interaction between time-related ramping, temporal context, and brain area (three-way ANOVA of GLMMs; F (4 045 080) = 226.8, P = 3e-51; Table 2). Of note, these GLMMs accounted for a variance from each animal and linked neuronal activity to behavior on a trial-by-trial level (Emmons et al. 2017(Emmons et al. , 2019. Taken together, our data demonstrate that time-related ramping in the MFC but not the DMS decreased in two-interval sessions compared to one-interval sessions. Our results suggest that MFC but not DMS ensembles are sensitive to the temporal context (Jazayeri and Shadlen 2010;Antzoulatos and Miller 2011;Shi et al. 2013).
We turned to decoding analyses based on machine learning to capture more complex features of MFC and DMS ensembles beyond time-related ramping (Gouvea et al. 2015;Paton and Buonomano 2018). Specifically, we constructed neurondimensional arrays of smoothed trial-by-trial firing rates over the interval binned at 0.1 s (Fig. 5A). As in our past work, we decoded time in the interval from ensemble firing rates using naïve Bayesian classifiers. Classifier performance was assessed  by computing the variance explained (R 2 ) of predicted versus observed time (Emmons et al. 2017;Kim et al. 2017). For all sessions, R 2 was much less for time-shuffled ensembles-that  improved only for DMS ensembles on FI3 trials and are consistent with prior studies that show striatal decoding can be more reliable than cortical decoding (Bakhurin et al. 2017).
We directly compared neuronal activity in the MFC and DMS on FI3 and FI12 trials. First, ramping neurons can have distinct slopes of firing rate versus time on FI3 and FI12 trials (Fig. 6A-C; Table 4). We ran GLMMs where firing-rate slope versus time was the outcome variable, interval and day were predictor variables, and neuron-specific variance was a random effect. As with PC1, we were interested in the slope magnitude rather than sign, so we focused on absolute value (|slope|). Consistent with past work by our group and others, ramping neuron |slopes| were consistently steeper on FI3 than on FI12 trials for both the MFC ( Fig. 6C; two-way ANOVA of GLMMs of interval and day on MFC |slope|; main effect of interval: F (139) = 4.5, P = 0.04) and for the DMS ( Fig. 6C; main effect of interval: F (210) = 6.9, P = 0.01; Xu et al. 2014;Mello et al. 2015;Emmons et al. 2017;Wang et al. 2018). There was no effect of day or higher interactions for either MFC or DMS. Taken together, these data are consistent with drift-diffusion models of interval timing, suggesting that drift rates increase with shorter intervals (Simen et al. 2011). Next, we searched for neurons in which firing rates were affected by interval duration (Fig. 6D-E). Specifically, we used GLMMs to identify neurons with an effect of interval on firing rate (P < 0.05). Interval-modulated neurons were more common in the DMS than the MFC on Day 1 of two-interval performance ( Fig. 6E; neurons with a main effect of interval on firing rate in one-way GLMMs; MFC 10 of 47 versus DMS: 28 of 58; X 2 = 8.20; P = 0.004). Interestingly, ∼ 50% of interval-modulated neurons also had ramping activity in MFC (5 of 10) and DMS (15 of 28). Of note, the number of interval-modulated neurons was not different between MFC and DMS on Days 2 and 3 (Fig. 6E). Consistent with these analyses, trial-by-trial GLMMs revealed a significant interaction between interval modulation, brain area, and Days 1-3 (three-way ANOVA of GLMMs of area, day, and interval on firing rate: F (3018772) = 6.4, P = 0.01; Table 4).
Of note, we found in GLMMs that there was no main effect of response on firing rate (F (3018772) = 2.5, P = 0.11), and no interaction with Days 1-3 (F (3018772) = 0.4, P = 0.51). Taken together, these data indicate that while MFC ramping is sensitive to temporal context, DMS neurons increased temporal decoding for FI3 trials and had distinct interval-related modulations. These data provide insight into corticostriatal dynamics during temporal learning.

Discussion
In this manuscript, we investigated whether MFC and DMS played similar or distinct roles in temporal learning. We found three novel distinctions between MFC and DMS during temporal learning. First, MFC but not DMS ramping decreased as animals that had been trained on a one-interval task learned to respond to a new temporal context that included a shorter second interval. Second, while MFC and DMS ramping was stable over 3 days of two-interval performance, DMS temporal decoding improved on the new FI3 interval. Finally, interval-modulated neurons were more common in the DMS early in two-interval performance. Our data suggest that MFC ensembles are sensitive to the context or "rules" of the task-that is, FI12 versus FI3/FI12, while the DMS optimizes behavior, particularly on FI3 trials. These data provide insight into the respective roles of prefrontal and striatal networks during temporal learning.
These results contradicted our hypothesis that MFC and DMS time-related ramping would be similarly affected by the introduction of a new temporal interval. Our hypothesis was based on 5 lines of evidence: 1) the existence of strong projections from the MFC to the DMS (Gabbott et al. 2005;Wall et al. 2013;Han et al. 2017), 2) clear roles for both structures in interval timing (Meck 2006;Coull et al. 2011;Emmons et al. 2017;De Corte et al. 2019), 3) similarities in time-related ramping in the MFC and DMS (Emmons et al. 2017), 4) the necessity of MFC activity for DMS ramping (Emmons et al. 2017(Emmons et al. , 2019, and 5) our recent demonstration that the stimulation of axons that project from the MFC to the DMS is sufficient to recover timerelated ramping in the DMS (Emmons et al. 2019). Given these data, it is notable that the MFC and DMS play distinct roles during temporal learning, although this observation is concordant with the vastly different connectivity and synaptic organization of these two structures (Shepherd 2003). Decreases in MFC ramping after the introduction of a shorter interval suggest that MFC ramping is sensitive to the temporal context of the one-interval versus two-interval task, and they may reflect Bayesian priors of temporal probabilities (Jazayeri and Shadlen 2010;Shi et al. 2013). Our data are consistent with the theory that prefrontal ramping reflects learned "rules" (Wallis et al. 2001;Genovesio et al. 2005;Fuster 2008;Ruge et al. 2019), in this case, the temporal contexts of the one versus two-interval task. By contrast, striatal ensembles did not change on FI12 trials but improved temporal decoding on FI3 trials, suggesting a view that striatal ramping is related to optimizing performance (Pasupathy and Miller 2005).
Corticostriatal differences were anticipated based on a recent comparison of neuronal ensembles during a temporal categorization task that involved maze running ). This study indicated that ramping was more prevalent in the MFC than the dorsal striatum. In this task, as the intervals became longer, temporal decoding by the MFC was less effective than that by the striatum. Although this task was more complex than ours and a number of others (Donnelly et al. 2015;Narayanan 2016;Bakhurin et al. 2017;Wang et al. 2018), the strong temporal encoding across corticostriatal ensembles is consistent with our findings here.
Our findings are in line with drift-diffusion models of twointerval tasks, as we find that time-related ramping scales with the interval duration (Simen et al. 2011;Xu et al. 2014). Here we find that MFC ramping is sensitive to temporal context whereas DMS ramping is not, and that nonramping interval-related modulations and temporal predictions in the DMS change with twointerval performance. These results suggest that time-related ramping reflects distinct processes in MFC and DMS. Given that MFC activity influences ramping in DMS (Emmons et al. 2017(Emmons et al. , 2019, DMS ramping activity might integrate aspects of MFC ramping as well as nonramping activity. Because time-related ramping activity in MFC and DMS ensembles did not change during two-interval performance, ramping activity may be remarkably stable in both brain regions when the temporal context does not change. It is unclear how ramping might change with extended periods of behavior over several days or weeks (Barnes et al. 2005;Yin et al. 2005;Graybiel 2008). However, we did find that on FI3 trials, temporal decoding in the DMS improved even though DMS ramping was stable. In the DMS, patterns beyond ramping activity might change during two-interval performance and contribute to improved temporal decoding (Paton and Buonomano 2018). The improvement in temporal prediction despite unchanged ramping activity supports improved "population clock"-based temporal predictions during FI3 trials (Karmarkar and Buonomano 2007;Laje and Buonomano 2013). Finally, <15% of corticostriatal neurons can be modulated by responses, and this modulation can interact with ramping activity (Emmons et al. 2017). However, in the present dataset, we found that response-related activity did not change over subsequent days of two-interval performance, and that there were main effects of context even when responses were matched between Day 0 and Days 1-3. These data imply that the shifts in MFC ramping and DMS temporal decoding could not be fully accounted for by changes in responding.
We found that the DMS contained more neurons in which there was a main effect of interval compared to the MFC on the early days of two-interval performance. Notably, half of intervalmodulated neurons were not ramping. These data suggest that patterns beyond time-related ramping encode information about temporal intervals. On progressive days of two-interval performance, interval-related activity between the MFC and DMS equalized. Because our task design involved a second cue for FI3 intervals, we cannot distinguish whether this activity was related to working memory for temporal intervals, cue-related processing, or other aspects of interval timing. Future work using more advanced learning paradigms may clarify these patterns of activity.
To our knowledge, our study is one of the first to record from corticostriatal ensembles during temporal learning and to suggest that components of corticostriatal ensembles play distinct roles in temporal learning. The striatum has a wellestablished role in learning other contexts including habit formation, reversal learning, and instrumental learning (Yin and Knowlton 2006;Kimchi and Laubach 2009;Graybiel and Grafton 2015). Striatal neuronal ensembles robustly encode time, and this temporal encoding rapidly rescales as animals learn novel intervals (Mello et al. 2015). While prefrontal ensembles are also involved in learning (Baeg et al. 2007), direct comparisons with striatal ensembles during learning are rare in rodents. One exception is a recent study of prenatal alcohol exposure, which showed that the orbitofrontal cortex disengages and the dorsal striatum updates reward contingencies (Marquardt et al. 2020). These findings parallel the changes in the MFC and DMS that we report here. The observation that prenatal exposure to alcohol leads to changes in cortical activity underscores the clinical significance of this brain circuit.
During associative learning in primates, corticostriatal ensembles are highly sensitive to learning, with striatal neurons rapidly encoding new associations and the prefrontal cortex learning more slowly (Pasupathy and Miller 2005). Primate striatal neurons rapidly encoded stimulus-response associations, whereas primate prefrontal neurons encoded category abstraction (Antzoulatos and Miller 2011). In line with these results, we found that time-related ramping in prefrontal regions decreased in two-interval versus one-interval sessions, suggesting that these ensembles may be sensitive to temporal categories or context. It is important to note that these primate studies recorded from lateral prefrontal areas, which lack a clear rodent analogue (Laubach et al. 2018), and that they employed vastly different task conditions. Furthermore, more recent work has shown that prefrontal ensembles can rapidly change during learning (Schuck et al. 2015;Siniscalchi et al. 2016), and therefore, the relative changes in corticostriatal ensembles may depend on the details of each task. Prior work has consistently established that corticostriatal neuronal ensembles are modulated during interval-timing tasks (Matell et al. 2003;Bakhurin et al. 2017;Emmons et al. 2017) and shown that single neurons in the prefrontal cortex (Xu et al. 2014) or striatum (Portugal et al. 2011;Mello et al. 2015) can flexibly adapt to new temporal contingencies. Our data are congruent with this work and directly compare MFC and DMS, suggesting that MFC ramping is sensitive to temporal context while DMS ensembles increase temporal decoding on trials with a novel, shorter interval. Our work provides insight into the dynamics of rodent corticostriatal ensembles during an elementary temporal learning paradigm.
Our study has several limitations. First, we used fixed-interval timing; peak-interval timing tasks might enable more precise dissection of start and stop times (Buhusi and Meck 2005). Indeed, the effects reported here may extend beyond timing processes and could also be a result of temporal discrimination of FI3 trials, learning a new contingency on FI3 trials, or disrupting an existing contingency on FI12 trials. In addition, we did not counterbalance between cues for FI3 versus FI12 trials, which would not affect comparisons of FI12 or FI3 trials over days (Figs 3-5), but might affect comparisons between activity on FI12 and FI3 trials. However, animals' response times on both FI12 and FI3 trials were responsive to both temporal contexts over subsequent days of two-interval performance. Despite stimuli and training differences between FI3 and FI12 trials, both responses and corticostriatal neuronal activity were affected by adding the FI3 interval. Our neural findings strongly suggest dynamic roles for corticostriatal ensembles during interval timing. Second, we did not track individual neurons over days. We describe MFC changes to temporal context and DMS changes over two-interval performance at group-level analyses (i.e., PC1 and decoding) as well as by trial-by-trial analyses of firing rate (Tables 2 and 4)thus, our data provide insight into flexibility among corticostriatal ensembles rather than at the single-neuron level. Third, our techniques cannot identify the genetic or molecular identity of recorded neurons. This detail would be of particular interest in the case of the DMS, which contains D1 and D2 MSNs, which play complementary roles in movement (Kreitzer 2009). Fourth, we are unsure if the MFC and DMS neurons that we captured were synaptically connected, because of the sparsity of cortical projections and constraints of our recording techniques (Wall et al. 2013). This limitation might be overcome in future work by exploiting optogenetic tagging and retrograde viral tracing to isolate corticostriatal projections (Otis et al. 2017). Studying how MFC-DMS connectivity changes with learning might provide further insight into corticostriatal circuits. Fifth, we were unable to identify clear behavioral correlates of temporal learning during the first two-interval session on Day 1. Behavioral transitions can be faster adjusting to shorter intervals and slower adjusting to longer intervals (Higa 1996(Higa , 1997Higa and Tillou 2001). Furthermore, adapting to new intervals may require multiple adjustments ). In addition, FI3 trials involve a higher reward rate as more rewards are acquired on average per second. Our data cannot address these issues and are instead focused on a single transition-from sessions with FI12 to sessions with FI12 and FI3 trials-but future work will be required to explore how corticostriatal ramping adapts to longer intervals and to distinct reward rates. Note that two of our main points-that MFC ramping decreases on FI12 trials, and that DMS temporal decoding increases on FI3 trials-were derived from comparisons of similar trial types, and were not affected by differences in training history, reward rate, and other features that differ between FI12 and FI3 trials. Corticostriatal ensembles may rapidly learn the new FI3 interval. The striatum is essential for rapid adaptations (MacDonald et al. 2012), but capturing changes in MSN firing might require a different task design to capture trial-by-trial neuronal dynamics during rapid learning (Kimchi and Laubach 2009).
In summary, we investigated whether the MFC and DMS played similar or distinct roles in temporal learning. We recorded from corticostriatal ensembles in rodents that had been trained to perform a single fixed-interval timing task while they learned to incorporate a new interval. We discovered that for FI12 trials, time-related ramping activity in the MFC decreased following introduction of the shorter interval, whereas ramping activity in the DMS was unchanged. For FI3 trials, we found that corticostriatal ramping activity did not change, yet DMS temporal decoding improved. Finally, more DMS neurons fired differentially on each interval compared to the MFC early in two-interval performance. Taken together, our data provide novel evidence that the MFC and DMS play distinct roles in temporal learning.