Neural signatures of classical conditioning during human sleep

1 Scientific efforts to teach humans novel verbal information during sleep have been 2 largely unsuccessful. However, recent findings demonstrate that sleeping humans 3 can learn entirely new non-verbal information in a stage-dependent manner. The 4 brain activity supporting the learning of novel information presented during sleep is 5 still unknown. Here, we aimed to elucidate the brain processes enabling sleep-stage-6 dependent novel associative learning during sleep. We presented auditory-olfactory 7 conditioning during non-rapid eye movement (NREM) and rapid eye movement 8 (REM) sleep while measuring electrophysiological and behavioural responses during 9 sleep and the subsequent morning. We found that associative learning during sleep 10 modulated learning-related delta and sigma power in NREM sleep. Furthermore, 11 learning-related delta power was associated with learning-related behaviour both 12 during sleep and the subsequent morning, but notably, this delta-behaviour 13 relationship changed between learning during sleep and during retrieval the following 14 morning. Together, these findings suggest that slow waves have a functional role in 15 sleep stage-dependent associative learning in sleep. 16


Introduction
The possibility to learn during sleep has intrigued humanity for over a century.In his 1911 science fiction novel "Ralph 124C 41+", Hugo Gernsback describe the Hypnobioscope, a device that transmits words directly to the sleeping brain such that they would be fully remembered in the next morning.However, decades of scientific attempts to teach sleeping humans new information have been mostly ineffective, and led to the conclusion that humans cannot learn novel verbal information in sleep (Peigneux, Laureys, Delbeuck, & Maquet, 2001;Simon & Emmons, 1956;Tani & Yoshii, 1970;Wood, Bootzin, Kihlstrom, & Schacter, 1992).Recently, new findings demonstrate that sleeping humans can learn entirely new non-verbal information (Andrillon, Pressnitzer, Léger, & Kouider, 2017;Arzi et al., 2014Arzi et al., , 2012)).Moreover, novel information learned in sleep can later modulate behaviour in a sleep stagedependent manner.Specifically, associations presented in sleep can be retrieved during both NREM and REM sleep 2 .Yet, associations learned in NREM sleep had a stronger and longer lasting effects on behaviour upon awake than associations learned in REM sleep (Arzi et al., 2014(Arzi et al., , 2012)).The brain processes underlying sleep stage-dependent associative learning during sleep are still unknown.
To elucidate the brain activity supporting novel associative learning during sleep we measured behavioural and electrophysiological responses during partialreinforcement auditory-olfactory conditioning in NREM and REM sleep (Fig. 1).On reinforced learning trials, each tone (400Hz or 1200Hz) was paired with either a pleasant or an unpleasant (aversive) odour.On non-reinforced learning trials, a tone was presented without an ensuing odour, enabling the measure of learning without the interference of odour.Learning was evaluated by the sniff-response, a change in nasal airflow in response to an odour, where unpleasant odours drive smaller sniffs than pleasant odours.We have previously shown that sleeping participants that learned theses tone-odour associations subsequently modulated their sniffs in response to tones alone.Moreover, both during sleep and subsequent morning the tone-induced sniffs differed according to the odour valence associated with the tone during sleep indicating that sleeping humans can learn to associate a specific odour with a specific tone (Arzi et al., 2012). .We focused our investigation on the main brain sleep rhythms associated with memory (Diekelmann & Born, 2010;Stickgold, 2005).In NREM sleep, slow waves defined by delta (0.5-4 Hz) electroencephalographic (EEG) activity and spindles characterized by sigma (11-16Hz) EEG activity have been associated with and causally related to memory consolidation (Antony, Gobel, O'Hare, Reber, & Paller, 2012;Cairney, Durrant, Hulleman, & Lewis, 2014;Marshall, Helgadóttir, Mölle, & Born, 2006;Rasch, Büchel, Gais, & Born, 2007;Schabus et al., 2004).For example, memory improvement was observed following enhancement of slow-waves by sensory and transcranial stimulation (Marshall et al., 2006;Rihm, Diekelmann, Born, & Rasch, 2014) and memory performance was correlated with slow-waves and spindles activity in NREM sleep (Antony et al., 2012;Cairney et al., 2014;Schabus et al., 2004).During REM sleep, theta oscillations (4-8Hz) were found to support memory consolidation; Theta power in REM sleep correlated with emotional and declarative memory, and abolishing theta oscillations in REM sleep without disrupting sleep structure impaired memory (Boyce, Glasgow, & Williams, 2016;Fogel, Smith, & Cote, 2007;Nishida, Pearsall, Buckner, & Walker, 2009;Poe, Nitz, Mcnaughton, & Barnes, 2000;Sopp, Michael, Weeß, & Mecklinger, 2017).Thus, we hypothesised that novel associative learning during sleep would elicit learningrelated delta and sigma changes during NREM sleep, and theta modulation during REM sleep.In addition, hypothesised that the learning-induced EEG modulation will be directly linked to the learning behaviour -the sniff-response.

Results
First, we set out to identify electrophysiological brain activity mediating novel associative learning during sleep.We analysed EEG activity of 549 non-reinforced trials collected from 35 participants during NREM sleep, and 206 non-reinforced trials collected from 14 participants during REM sleep.We found higher power following a tone alone previously paired during sleep with an unpleasant odour (CSu) than following a tone alone previously paired during sleep with a pleasant odour (CSp), in delta (Wilcoxon signed-rank test z(34) = 2.51, p = 0.01, effect size r = 0.42, BF = 9.23; Figure 2a,c,e) and sigma (Wilcoxon signed-rank test z(34) = 2.2, p = 0.03, effect size r = 0.37, BF = 2.41; Fig. 2a,c,f) in NREM sleep.However, contrary to our hypothesis, we found no differences between CSu and CSp in theta power following conditioning in REM sleep (Wilcoxon signed-rank test p = 0.39, BF = 0.98 Figure 2b,d,g).These findings suggest that newly acquired associations learned in sleep shape learning-related slow waves and spindles during NREM sleep, but provide no evidence for learning-related theta modulation in REM sleep.
To address the possibility that participants showing arousal events or awake periods during learning in NREM sleep skewed these findings, we cropped the data at the first point of arousal during a reinforced trial.Repeating the analysis on the cropped data with 282 non-reinforced trials from 32 participants retained the same effect in delta (Wilcoxon signed-rank test z(30) = 3.2, p = 0.002, effect size r = 0.56, BF = 198.47;Fig. 3a, one outlier was excluded) and sigma (Wilcoxon signed-ranks z(31) = 2.9, p = 0.004, effect size r = 0.51, BF = 20.19;Fig. 3b), confirming that it is learning during sleep and not during arousals from sleep that modulated the tone-induced brain activity.
To verify that the observed higher power for CSu in comparison to CSp in delta and sigma frequency bands is driven by the learning process and not merely reflect a differential response to the pitch of the tones, we re-divided the trials according to pitch (400Hz or 1200Hz), regardless of odour quality previously paired with tones in sleep.We found reliable evidence for a lack of differences both in delta and sigma power (Delta: Wilcoxon sign-rank test z(34) = 0.33, p > 0.74, effect size r = 0.06, evidence supporting the null BF = 0.15, Fig. 3c ;Sigma: Wilcoxon sign-rank test z(34) = 0.09, p > 0.92, effect size r = 0.02, evidence supporting the null BF = 0.3, Fig. 3d), suggesting that the newly learned tone-odour associations, and not the tones' physical properties, modulate tone-induced EEG response in NREM sleep.
Altogether, these findings support the hypothesis that the plastic processes occurring during novel conditioning in sleep are associated with slow waves and spindles activity in NREM sleep.
Next, we tested whether the ability to learn new associations in sleep is linked to learning-related brain activity in sleep.The learning was measured by the sniffresponse, a tone-induced change in sniff volume indexing expectation of an odour.
We have shown before that novel tone-odour associations learned in NREM sleep but not in REM sleep were later retrieved during the subsequent morning (Arzi et al., 2012).Hence, we hypothesised that learning-related EEG activity in NREM sleep, but not in REM sleep, would predict the learning-related sniff-response during the subsequent morning.We found positive associations between tone-induced delta power in NREM sleep and sniff-response during the subsequent morning for CSu (r spearman = 0.38, p = 0.034, BF = 2.46; Fig. 4a) and CSp (r spearman = 0.54, p = 0.002 BF = 80.19; Fig. 4b).These results reflect higher the delta power greater the reduction in sniff volume.In addition, no association was found between CSu-CSp difference in delta power and CSu-CSp difference in sniff-response (r spearman = 0.16, p = 0.37, BF = 0.45), implying that delta activity mediates non-discriminatory associative learning in sleep, but not discriminatory learning.No significant associations were found between tone-induced sigma power in NREM sleep or theta power in REM sleep and learning-related sniff-response in the morning (Supplementary Fig. 2).Subsequently, we tested whether tone-induced delta power was associated with the sniff-response during the learning process in sleep.We found a negative correlation between tone-induced delta power and sniff-response in NREM sleep for CSu (r spearman = -0.48,p = 0.005, BF = 15.92;Fig. 3c), but not for CSp (r spearman = 0.18, p = 0.3 BF = 0.5; Fig. 3d), suggesting that the aversiveness of the unpleasant odour promoted a more prominent learning (Lipp, Sheridan, & Siddle, 1994;Pittino, Kliegl, & Huckauf, 2017).
Intriguingly, the direction of the delta-sniff correlation reversed between sleep and the morning.To examine the dynamics of the learning behaviour and neural activity, we tested whether the relation between the tone-induced delta and the sniffresponse differed between NREM sleep and the following morning.We found that delta-sniff correlation was significantly different between the two states for both CSu and CSp (base on bootstrapping of correlation coefficient pairs with 10,000 permutations, CSu: p = 0.0002, CSp: p = 0.0012).Notably, the behavioural responses (sniff-response in NREM sleep and morning) were not associated, (CSu: r spearman = -0.06,p = 0.73, BF = 0.37; CSp: r spearman = 0.001, p = 0.99, BF = 0.31).The significant change in relation tone-induced delta and the sniff-response between sleep and morning implies that further consolidation processes occur after the learning procedure has ended.
Last, we found the same patterns of results when repeating the analyses including data from conditioning during NREM sleep-only (Supplementary Fig. 3), confirming that effects are driven by learning acquired during NREM sleep and not by interaction between learning in different sleep stages.

Discussion
In this study, we set out to elucidate the brain processes involve the ability to learn new information during sleep.We hypothesised that novel associative learning during sleep would elicit learning-related delta and sigma changes during NREM sleep, and theta modulation during REM sleep.We found that learning entirely new auditory-olfactory associations during NREM sleep modifies learning-related delta and sigma power.These findings support our hypothesis that sleep-learning modulate slow waves and spindles, and dovetail nicely with memory consolidation (Diekelmann & Born, 2010;Marshall et al., 2006;Schabus et al., 2004;Stickgold, 2005), and memory reactivation studies showing cue-induced delta and sigma enhancement during NREM sleep (Antony et al., 2012;Cairney et al., 2014;Hauner, Howard, Zelano, & Gottfried, 2013;Rasch et al., 2007;Rihm et al., 2014).
In REM sleep however, we found no evidence of learning-related theta modulation.
The lack of learning-related theta modulation during REM sleep is inconsistence with the presence of learning-related sniff-response during REM sleep, where a tone predicting an unpleasant odour triggered a larger reduction in sniff volume in comparison to a tone predicting a pleasant odour (Arzi et al., 2012).This controversy between theta activity in REM sleep and learning processes may reflect the limited understanding of their intricate relationship (Boyce et al., 2016;Fogel et al., 2007;Lehmann, Schreiner, Seifritz, & Rasch, 2016;Poe et al., 2000;Sopp et al., 2017).
For example, attenuation of the memory-associated theta rhythm in REM sleep impaired contextual memory, while cued memory was not altered (Boyce et al., 2016), and cuing of neutral, but not of emotional memories enhanced theta activity in REM sleep (Lehmann et al., 2016).Furthermore, novel perceptual learning in REM sleep increased theta power, however the memory improvement upon awake was associated with the proportion of tonic, but not phasic REM sleep (Andrillon et al., 2017).The distinctive brain activity and responsiveness in phasic and tonic REM sleep (Ermis, Krakow, & Voss, 2010;Simor, Gombos, Blaskovich, & Bódizs, 2017) that interacts with learning processes (Andrillon et al., 2017;Simor et al., 2017) may potentially explain some of the discrepancies.
In addition, we found that learning-related delta power was directly linked to learning behaviour.During NREM sleep, learning-related delta power negatively correlated with the sniff-response for aversive memory, but not for non-aversive memory.A possible explanation for this dissociation could be that the aversiveness of the unpleasant odour promoted more prominent learning.Indeed, several studies showed that conditioning with aversive stimuli induced greater responses (Lipp et al., 1994;Pittino et al., 2017) and wider generalization (Resnik, Sobel, & Paz, 2011) than conditioning with non-aversive stimuli.In the morning, however, learning-related delta power positively correlated with the sniff-response for both aversive and nonaversive memory.The non-discriminatory delta-sniff correlations in the morning may be due to generalization processes occurring during consolidation in sleep (Stickgold & Walker, 2013).Interestingly, the direction of the learning-related delta power and the sniff-response correlation reversed between NREM sleep and the following morning.One can speculate that the change in the direction of the association between learning during NREM sleep and retrieval upon awake could be partially explained by consolidation processes occurring during sleep after the learning procedure had finished.

Materials and Methods
The data used here was collected as part of a study that examined whether humans can learn new associations during sleep and was published independently (Arzi et al., 2012).Thus, detailed information about participants, experimental design, data acquisition and behavioural results can be found in the original article.Participants were unaware of specific experimental aims and conditions.

Stimuli.
Pleasant (shampoo or deodorant) and aversive unpleasant (rotten fish or carrion) odorants were presented in a nasal mask by computer-controlled air-dilution olfactometer from an adjacent room (stimulus duration = 3 s, constant flow = 6 litres per minute).Tones (400, 800 and 1,200 Hz, duration = 1 s, at a non-arousing 40 dB) were presented by loudspeaker ~2 m from participants' heads.

Procedures.
Participants arrived at the sleep laboratory at a self-selected time, based on their usual sleep pattern, typically at 11:00 pm.After fitting of the polysomnography devices (Iber, Ancoli-Israel, & A, 2007), subjects were left alone in the darkened room to be observed from the neighbouring control room via infrared video camera.
The experimenters observed the real-time polysomnography reading and, after they determined that the subject had entered the desirable sleep stage, they initiated the experimental protocol.The conditioned and unconditioned stimuli were partially reinforced at a ratio of 2:1; on reinforced trials (two-thirds of trials), each 1-s auditory conditioned stimulus (either 1,200 Hz or 400 Hz) was triggered by inhalation and paired with a 3-s olfactory unconditioned stimulus (either pleasant or aversive unpleasant).On non-reinforced trials (one-third of trials), a tone was generated without an odorant (tone alone).Stimuli were generated in blocks of six trials (two reinforced trials with pleasant odour, two with unpleasant odour and two nonreinforced trials, one of each tone, randomized between blocks).Tone-odour contingencies were counter-balanced across participants.The conditioned response was measured by the sniff-response magnitude to tones alone.During wakefulness the sniff-response can be conditioned to a tone such that different tones can drive different sniffs (Resnik et al., 2011).Therefore, the sniff-response was chosen to be the conditioned response in this experiment.In the first experiment (28 participants), in a night without arousals/wakes within a window of 30 s from tone onset, five blocks were presented in NREM sleep, then the procedure was halted up to stable REM sleep, at which point an additional five blocks were presented.In a second experiment, the procedure was triggered during either NREM sleep only (15 participants) or REM sleep only (12 participants).If an arousal/wake was detected in the ongoing polysomnographic recording, the experiment was immediately stopped until stable sleep was resumed and then continued up to a maximum of 18 blocks.
Because the experiment was halted following arousal or wake, different subjects had different numbers of trials.About half an hour after spontaneous morning wake, conditioned response was tested in a retention procedure: three auditory stimuli, 1,200 Hz and 400 Hz that were presented during the night, and a new 800-Hz tone (eight repetitions each), were presented while nasal respiration was recorded.
Retention procedure data from three subjects (n = 1 in REM-only and n = 2 in NREM-only) was lost as a result of technical errors.

Polysomnography.
Sleep was recorded by standard polysomnography (Iber et al., 2007).EEG (obtained from C3 and C4, referenced to opposite mastoid), electro-oculogram (placed 1 cm above or below and laterally of each eye, referenced to opposite mastoid), electromyogram (located bilaterally adjacent to the submentalis muscles), and respiration were simultaneously recorded (Power-Lab 16SP and Octal Bio Amp ML138, ADInstruments) at 1 kHz.Nasal respiration was measured using a spirometer (ML141, ADInstruments) and high-sensitivity pneumotachometer (#4719, Hans Rudolph) in line with the vent ports of the nasal mask.

Nasal airflow analysis.
We normalized the nasal inhalation volume during sleep by dividing the sniff volume in each trial by the block baseline (averaged volume of 15 nasal inhalations preceding block onset).We normalized the nasal inhalation volume in the wake retention paradigm by dividing the sniff volume for each tone by the baseline nasal inhalation volume (averaged volume of 15 nasal inhalations preceding retention procedure onset).The sniff response was calculated as 1 -normalized sniff volume.
Participants' sniff-response differing by 3.5 s.d. were excluded (two participants at night and one in the morning in the CSu condition).

EEG analysis.
EEG activity was recorded from C3 and C4 electrodes.Visual inspection found that C3 was less noisy and therefore data recorded from this electrode was used for the EEG time-frequency analysis.Trials with EEG artefacts within a 10-second window before or after tone onset were excluded.EEG spectral analysis in the 0.5-40 Hz frequency range for all non-reinforced trials that met study criteria was conducted using Hilbert transform on a 10-second window before and after tone onset using customized MATLAB scripts.The power in each 20-second trial was then z-scored.
Non-reinforced trials from the same participant were then averaged to create a single time-frequency representation per participant, per condition (tone alone previously paired with an aversive unpleasant odour (CSu) and tone alone previously paired with a pleasant odour (CSp)) and per sleep stage (NREM sleep and REM sleep).
Next, a permutation-based statistical test of time-frequency data was applied using FieldTrip (Oostenveld, Fries, Maris, & Schoffelen, 2011) and customized scripts.Time-frequency representation of CSu and CSp conditions, for each sleep stage, were submitted to a cluster-based non-parametric permutation test (Maris & test r, was calculated by dividing the Z statistics of the Wilcoxon signed-rank test by the square root of the number of participants.Spearman correlation were used to estimate the degree of association between tone-induced EEG response and sniffresponse.Comparisons between correlations was estimated using a bootstrapping procedure with 10000 repetitions.Bayes Factors of the null and alternative hypothesis were calculated using Bayes factor calculator (Dienes, 2014) (http://www.lifesci.sussex.ac.uk/home/Zoltan_Dienes/inference/Bayes.htm).For the EEG power alternative hypothesis tests, Bayes Factors were calculated using a halfnormal distribution with a standard deviation of half the maximum observed power.
For the EEG power null tests Bayes Factors were calculated using a uniform distribution of between zero and the maximum-effect.For the association tests, Bayes Factors were calculated using a half-normal distribution with a standard deviation of 0.5, which was Fisher's z transform to 0.549.Correction for multiple comparison was done using false discovery rate (FDR) correction.For EEG power analysis between CSu and CSp significance threshold was corrected for the number of frequency bands that were tested (delta, sigma and theta frequency).For correlation analysis significance threshold was corrected the number of correlations tested in each frequency band.Statistical analyses were performed using Matlab and open-source statistical software JASP (JASP Team (2017), version 0.8.3.1).
and animals (De Lavilléon, Lacroix, Rondi-Reig, & Benchenane, 2015; Hennevin & Hars, 1992) can learn entirely new non-verbal information during sleep.An important step in the course of identifying what is possible to learn during sleep specifically, and unconsciously in general is the understanding of the brain mechanisms underlying encoding, consolidation and retrieval of new information presented in sleep.Here, we start to elucidate part of these mechanisms by showing that associative learning during sleep is supported by learning-related slow wave activity, and that the modulation of slow waves occurring during the process of learning in sleep may determine the fate of newly acquired memories.
Sixty-nine healthy participants (mean age = 25.2 ± 3.0 years, 24 females) gave informed consent to procedures approved by the Weizmann Institute Ethics Committee.Participant exclusion criteria included use of medication, history of sleep disorders and nasal insults, or insufficient sleeping time.Fifty-five participants (mean age = 25.2 ± 3.2 years, 20 females) met the inclusion criteria.Out of these, 28 participants were presented with the auditory-olfactory conditioning during both NREM and REM sleep (experiment 1), 15 participants during NREM sleep only (experiment 2) and 12 participants during REM sleep only (experiment 2).

Figure 1 :
Figure 1: Auditory-olfactory conditioning during sleep (a) Stimuli were generated in blocks of six trials: two reinforced trials with pleasant odour (pink), two reinforced trials with unpleasant odour (grey) and two nonreinforced trials (tone alone), one of each tone.If sleep was not disturbed, five blocks were presented during NREM and five blocks during REM sleep (Experiment 1) or ten blocks were presented during NREM sleep-only or REM sleep-only (Experiment 2) (See methods).T, tone.On reinforced trials, each auditory stimulus (1,200 Hz or 400 Hz) was paired with either a pleasant or aversive unpleasant odour.On nonreinforced trials, a tone was generated without an odour.(b) A retention paradigm during the subsequent morning with three auditory stimuli (1,200 Hz, 400 Hz and a novel 800-Hz tone, eight repetitions each), but no odours presented.

Figure 2 :
Figure 2: Learning-related electrophysiological activity in NREM and REM sleep.Time-frequency decomposition of the EEG signal recorded from C3 electrode in NREM (right) and REM (left) sleep time-locked to a tone previously paired during sleep with (a-b) an aversive unpleasant odour (CSu) and (c-d) with a pleasant odour (CSp).Black bar represents tone duration, dotted vertical line represent tone onset, and rectangle represent the window of interest in delta (0.5-4 Hz), sigma (11-16 Hz), delta power in NREM sleep (z−score) theta power in REM sleep (z−score) n = 35, p = 0.01* n = 35, p = 0.03* n = 14, p = 0.39 and theta (4-8 Hz) frequency band.(e) Tone-induced delta power in CSu (dark purple) and CSp (light purple) trials (n = 35).(f) Tone-induced sigma power in CSu (dark orange) and CSp (light orange) trials (n = 35).(g) Tone-induced theta power in CSu (dark green) and CSp (light green) trials (n = 14).Pairs of circles connected by a grey line represents a single participant, boxplots represent median and the first and third quartiles of the group.* FDR corrected statistical thresholds.

Figure 3 :
Figure 3: Tone-induced Delta and Sigma power.(a-b) Learning-related EEG power in NREM sleep up to the first arousal during conditioning.(a) Tone-induced delta power in CSu (dark purple) and CSp (light purple) trials in NREM sleep up to the first arousal during conditioning (n = 31).Note: when including one outlier the same results persisted: z(31) = 2.9, p = 0.004, n = 32 (b) Tone-induced sigma power in CSu (dark orange) and CSp (light orange) trials in NREM sleep up to the first arousal during conditioning (n = 32).(c-d) Tone-induced EEG power in NREM sleep for tone pitch.Tone-induced (c) delta and (d) sigma power for 1200Hz tone (black) and 400Hz tone (grey) trials in NREM sleep (n = 35).Pairs of circles connected by a grey line represents a single participant, boxplots represent the median and the first and third quartiles of the group.
delta power in NREM sleep (z−score) sigma power in NREM sleep (z−score) sigma power in NREM sleep (z−score)