Abstract
Visual short-term memory (VSTM) is an important measure of information processing capacity and supports many higher-order cognitive processes. We examined how sleep deprivation (SD) and maintenance duration interact to influence the number and precision of items in VSTM using an experimental design that limits the contribution of lapses at encoding.
For each trial, participants attempted to maintain the location and color of three stimuli over a delay. After a retention interval of either 1 or 10 seconds, participants reported the color of the item at the cued location by selecting it on a color wheel. The probability of reporting the probed item, the precision of report, and the probability of reporting a nonprobed item were determined using a mixture-modeling analysis. Participants were studied twice in counterbalanced order, once after a night of normal sleep and once following a night of sleep deprivation.
Sleep laboratory.
Nineteen healthy college age volunteers (seven females) with regular sleep patterns.
Approximately 24 hours of total SD.
SD selectively reduced the number of integrated representations that can be retrieved after a delay, while leaving the precision of object information in the stored representations intact. Delay interacted with SD to lower the rate of successful recall.
Visual short-term memory is compromised during sleep deprivation, an effect compounded by delay. However, when memories are retrieved, they tend to be intact.
INTRODUCTION
Reduced information processing capacity is an important contributor to the deleterious effects of sleep deprivation (SD) on cognitive performance.1,2 Even when we are well rested, our ability to attend to and maintain information in visual short-term memory (VSTM) is severely capacity-limited.3 VSTM allows representations to be actively extended over time in the absence of sensory input, making such information available to more complex cognitive operations ranging from mental arithmetic4 to problem-solving.5 If short-term memoranda fail to be maintained over brief delays, critical items that we need to manipulate or process may be irretrievable. Hence, a better understanding of how VSTM is affected by SD across delays is relevant for persons such as air traffic controllers or medical personnel who need to maintain and process considerable amounts of visual information while on overnight duty.
Early research on VSTM primarily focused on characterizing its ‘capacity limit’ in terms of the number of items that can simultaneously be stored, independent of the amount of visual detail stored for each item. This capacity limit is approximately three to four easily discriminable objects.6,7 SD might be expected to reduce VSTM capacity on account of its effects on parietal and visual extrastriate cortices, whose engagement contributes to the maintenance of short-term memory representations.8,9 Indeed, VSTM capacity evaluated in the early morning after a night of SD was reduced to approximately two items.10,11
Although the number of items is a key determinant of memory capacity, VSTM is also sensitive to the detail and precision of stored representations. For example, the number of items we can “hold in mind” varies inversely with the quantity of visual information associated with a given item.12 To account for both the number of items stored and the fidelity of the stored representations, flexible resource and two-factor models of VSTM capacity have been advanced.13–16 These models take into account how visual complexity affects the flexible allocation of processing resources to objects we wish to remember. In support of a dissociation between the number and resolution of representations in short-term memory, fluid intelligence has been found to correlate with the number of stored representations but not with the resolution of these representations.15 Furthermore, different posterior brain regions have been shown to track the number of items stored and the visual complexity of these items.9
Although SD can reduce the number of items stored in VSTM, it is not known whether it also affects the fidelity of stored representations. These representations may be less precise during SD because the neural substrate that supports VSTM is underactivated during task performance in that state.10,11 SD-related reductions in performance and neural activity can be accompanied by increased behavioral instability,17 which may further contribute to the degradation of items in VSTM. Consistent with this possibility, sleep-deprived animals engaged in goal-directed behavior show random dropouts in neuronal firing in frontal, parietal,18 and extrastriate visual cortices.19 These observations are relevant because the maintenance of short-term visual representations is thought to depend on recurrent reverberatory activity20 within such cortical regions.21,22
The effects of SD also render short-term memory vulnerable to extended delays. For example, serially presented auditory and visual items decay more quickly when participants are sleep deprived.23–25 Although delay-dependent reductions in the number of stored representations have been found for both healthy volunteers26 and patients with schizophrenia,27 these studies did not observe statistically significant reductions in fidelity across a delay. If SD leads to decreased representational fidelity, however, one might expect long retention intervals to cause further decreases in precision.
In the current study, we explored the effects of SD, delay, and their interaction on the number and fidelity of VSTM representations. To extend previous work, we used the simultaneous presentation of stimuli that were not easily recoded (e.g., into a verbal form) and employed an eye-tracker to reduce the effects of eye closure on our inferences about memory failures. To separately measure the number of items stored in VSTM and the fidelity of each representation, we used a continuous report procedure.14 For each trial, participants attempted to maintain the location and color of three stimuli over a delay.26 After retention intervals of either 1 or 10 s, participants were asked to recall the color of the item at the cued location. They reported the color of the item as best they could remember by selecting it on a color wheel.
Each response was assumed to fall into one of three categories. If the color and location of the probed item were maintained in VSTM, the reported color value would likely be near the original color. On average, the fidelity of color information stored would be inversely related to the distance between the reported and original colors. If the probed item was forgotten or never encoded, however, the participant might make a random guess. These responses would be randomly distributed around the color wheel. Finally, the participant could attempt to report the color of one of the nonprobed items, because she either incorrectly bound the color and location in memory28 or simply guessed nonrandomly. We used a mixture-modeling analysis to determine the likely frequencies of these different response types as well as the precision of the reported items.28,29 By assessing these estimates at contrasting delays of 1 and 10 s, we aimed to provide a more complete assessment of how VSTM degrades following SD.
METHODS
Participants
Nineteen healthy, right-handed participants (mean age 21.6 ± 1.0 y; 7 females) were studied after providing informed consent, in compliance with a protocol approved by the National University of Singapore Institutional Review Board. They were selected from respondents of a web-based questionnaire who (1) were right-handed, (2) had regular sleeping habits, (3) slept no less than 6.5 h/night, (4) were not on any long-term medications, (5) had no symptoms or history of sleep disorders, (6) had no history of psychiatric or neurological disorders, and (7) drank fewer than three caffeinated drinks per day. All reported having normal color vision and normal or corrected-to-normal visual acuity. Participants were tested for color blindness using Ishihara plates.
The sleeping pattern of each participant was monitored throughout the entire duration of the study. Only those whose actigraphy data (see below) indicated habitual good sleep underwent evaluation of VSTM. Participants were not allowed to smoke or consume any medication, stimulants, caffeine, or alcohol for at least 24 h prior to each experimental session.
Study Procedure
Participants made three visits to the laboratory. The first was a briefing session, during which they were informed about the study protocol and requirements. Suitable volunteers were familiarized with the experimental apparatus and practiced six trials of the study task. At the end of this session, the participants were given a wrist actigraph (Actiwatch, Philips Respironics, Andover, MA, USA) to wear throughout the study. This device verified regular and adequate sleeping patterns. Each participant performed the experimental task twice, once during rested wakefulness (RW) and once following a night of SD. The order of the sessions was counterbalanced across participants, and the sessions were separated by approximately 1 week to minimize residual effects of SD.
The RW session took place at about 08:00. For the SD session, participants arrived at the laboratory at 19:30, after staying awake the entire day without napping. They were subsequently monitored in the laboratory. VSTM evaluation took place at approximately 06:00 the next day. During the SD session, participants were allowed to engage in nonstrenuous activities such as reading, watching videos, and conversing. Hourly during the study night, participants rated their current sleepiness levels using the Karolinska sleepiness scale30 and the Stanford sleepiness scale.31
Before the experimental task, participants performed a computerized version of the Psychomotor Vigilance Task.17 During all tasks, participants sat with their head supported by a chin rest and viewed the monitor at a distance of 60 cm in a dimly illuminated room.
Stimuli and Task
Stimuli were generated by the Psychophysics Toolbox32,33 for MATLAB 7.8.0 (MathWorks, Natick, MA, USA) and were displayed on a 19-inch flat panel display (resolution: 1,024 × 768 pixels, refresh rate: 60 Hz) with a gray background. Throughout the experiment, gaze position was monitored online at 60 Hz using a Tobii X60 (Tobii Technology, Sweden) remote infrared eye-tracker.
Each trial began with the presentation of a central fixation cross (black, 0.7° of visual angle; Figure 1). Once stable fixation on the cross was achieved for 2,000 ms, a memory array consisting of three colored squares was presented for 1,000 ms. Each square subtended 1.1° × 1.1° of visual angle and was randomly presented at one of six possible positions on an invisible circle with a 5.6° radius. Each square was independently and randomly assigned to one of 180 equiluminant colors evenly distributed along a circle in the CIE L*a*b* color space (centered at L = 70, a = 20, b = 38, with a radius of 60). Each item in the memory array was then followed by a 100-ms multicolor mask,29 which was presented to erase the item from iconic or perceptual memory.34 Critically, we used real-time eye tracking to ensure that the eyes were open and fixated during the encoding phase of each trial. Any trial in which eye position could not be determined was restarted from fixation with new feature values. Similarly, if gaze deviated more than 6° (the radius of the invisible circle in which the memory array was presented) from the central cross during presentation of the memory array, the trial was restarted. This procedure reduced the likelihood of encoding failures due to lapses and microsleeps while ensuring an equal number of trials across sessions and conditions.
Stimuli from the color recall task. Participants were presented with an array of colored squares, followed by a multicolor mask. After a blank retention interval of 1 or 10 s, one of the locations was cued. Participants adjusted the position of the indicator bar on the color wheel to match the remembered color at the cued location.
A delay of either 1 or 10 s followed the presentation of the memory array. This delay manipulation allowed us to investigate the effects of SD on maintenance and/or retrieval, as the two delay conditions involved similar encoding demands.26,35,36 At the end of the delay period, a probe display consisting of black square outlines at the sample locations appeared. A thicker outline indicated the item to be reported. Participants reported this item using a color wheel (6.7° radius, 1.1° thick), a ring of 180 colored segments that encircled the probe display. A black indicator bar, with position determined by the angular position of the cursor, appeared outside the color wheel to indicate the selected color. Participants used the mouse to adjust the position of this bar until it matched the desired color value and then clicked to make their report. To prevent response biases, the indicator bar's initial position and the rotation of the color wheel were randomly assigned on each trial. Accuracy was stressed, and responses were not timed, although a beep alerted participants to respond if they had not done so after 10 s. After the response, a feedback screen provided the response error in degrees. The next trial began after an intertrial interval of 1,000 ms.
The 1- or 10-s delay intervals were equally likely and were randomly intermixed within each block. Fifteen trials were presented at each delay within each block, and participants completed eight memory blocks in each session. Thus, participants completed 240 trials per session, each with 120 trials per delay condition.
Statistical Analyses
For each trial, the target response error was calculated as the deviation of the response value (i.e., the reported color) from the target value (i.e., the color of the probed item) in degrees. Nontarget response errors were also calculated for the two non-probed items.
Bays and colleagues28 recently proposed a probabilistic model attributing the observed response error distribution to three different types of responses (Figure 2). According to this model, the response error distribution is a mixture of three distributions:
A von Mises distribution (the circular analog of the Gaussian distribution) of response errors relative to the probed target, with the concentration parameter κ reflecting the precision of the report.
A von Mises distribution of response errors relative to the nonprobed items, with the same concentration parameter κ as above.
A uniform distribution corresponding to responses in which the participant guessed at random.
Components of error in the color recall task.53 The response distribution is assumed to comprise a mixture of three distributions: A von Mises distribution with variability σ (converted from κ; see text) around the target (T) color (top), von Mises distributions with variability σ around each of the nontarget (NT) colors (middle), and a uniform distribution across the response space (bottom). The figure represents hypothetical response distributions for each component. Note that the distances between the target and nontarget items varied across trials; the figure shows an example.
I0 and I1 are the modified Bessel functions of order 0 and 1 respectively. σ represents the inverse precision of report, with higher σ corresponding to lower precision.
The mixture model described the data well across all conditions (Figure 3). The individual subject estimates for each parameter by condition were submitted to a two-way repeated-measures analysis of variance with factors of State and Delay. All post hoc comparisons were run as paired-sample t-tests. Statistical analyses were performed in R (http://www.r-project.org).
Histograms of the difference between the reported color and the original color value (in degrees) for all participants, separated by State and Delay. The solid black line in each panel shows the best fitting mixture distribution for the data combined across participants. Note that the nontarget (NT) distributions cannot be seen because the color values of the NT items are random relative to the target. SD, sleep deprivation; RW, rested wakefulness.
RESULTS
Psychomotor Vigilance Task
SD impaired performance on the psychomotor vigilance task (PVT; Table 1). There were significant increases in both median reaction time (mean ± 95% confidence interval [CI]: 57 ± 20 ms; t18 = 6.031, P < 0.001) and percentage of lapse trials (defined as reaction time > 500 ms; mean ± 95% CI: 11.7% ± 5.1; t18 = 4.859, P < 0.001).
Probability of Reporting the Target Item (α)
The ability to sustain a retrievable, integrated VSTM representation was reduced across a delay, evidenced by lower α for trials with a 10-s delay compared to trials with a 1-s delay (F1,18 = 47.89, P < 0.001; Figure 4A, Table 2). SD also reduced this estimate of storage (F1,18 = 73.10, P < 0.001). Critically, there was a significant interaction between Delay and State (F1,18 = 36.45, P < 0.001), such that the delay-dependent decline in correct target report was more pronounced in SD than in RW.
(A) The probability that the probed item was stored in memory over the delay (α) as a function of State and Delay. (B) The inverse precision of the stored representations (σ; a higher number reflects lower precision) as a function of the same. Error bars represent 95% confidence intervals based on within-subject standard error.59,60 SD, sleep deprivation; RW, rested wakefulness.
Post hoc t-tests confirmed that the effect of Delay was significant in both RW (t18 = 3.45, P = 0.003) and SD (t18 = 7.13, P < 0.001). There was also an effect of State even when the delay was short (t18 = 7.34, P < 0.001; long delay: t18 = 8.04, P < 0.001).
Precision of Report (σ)
Delay between encoding and recall significantly reduced the precision of stored representations (F1,18 = 8.43, P = 0.009; Figure 4B, Table 2), with higher σ for 10-sec delay trials compared to 1-s delay trials. Interestingly, the precision of report showed neither a significant main effect of State (F1,18 < 1, not significant.) nor a significant State by Delay interaction (F1,18 = 1.66, P = 0.213). VSTM representations thus became noisier (i.e., had lower precision) when they were maintained for longer durations. However, the precision of these memories appeared to be unaffected by SD.
Probability of Reporting a Nontarget Item (β)
Observers often reported the color of one of the two nontarget items. Such errors could represent the incorrect binding of color and location in memory28 (see Discussion section). The longer maintenance period between encoding and recall resulted in an increased probability of these errors, as evidenced by elevated β (F1,18 = 4.42, P = 0.049; Table 2). SD had a similar effect (F1,18 = 9.33, P = 0.007), but there was no significant interaction between State and Delay (F1,18 < 1, not significant).
Probability of Reporting a Random Color (γ)
Participants were more likely to guess at random after a long delay (F1,18 = 29.95, P < 0.001; Table 2) and during SD (F1,18 = 40.33, P < 0.001). These factors also interacted (F1,18 = 22.04, P < 0.001).
When participants failed to report the target item, they could report either the color of a nontarget item or guess at random. To determine the relative contribution of these two response types across our conditions of interest, we calculated the percentage of errant responses during which the participant guessed at random (γ / (β + γ)). This percentage did not significantly vary across Delay or State (all F1,18 < 1, not significant). Furthermore, for most of the trials during which participants failed to report the target color, they guessed at random (Table 2).
DISCUSSION
To add to prior studies of short-term and working memory in sleep-deprived persons, the current study used stimuli that are neither readily recoded verbally nor easily grouped and whose precision of representation is measurable. In addition, the requirement for the eyes to remain open and fixated reduced the likelihood of encoding failures caused by lapses and microsleeps, whereas the delay manipulation allowed us to investigate the effects of SD on maintaining and/or retrieving short-term memories.26,35,36 These design considerations enabled us to evaluate the number and precision of stored memory representations and how delay interacts with SD to accelerate their degradation.
We found that short-term memory representations are more likely to fail when maintained across a longer delay, an effect that was exacerbated by SD. SD alone also reduced the number of items in VSTM (as indexed by α13), even over a short delay interval. This result is consistent with earlier reports of reduced VSTM capacity in SD.10,11 A small number of report failures may be attributable to binding errors, but most are caused by the complete absence of a retrievable item in VSTM. In stark contrast to the reduced number of stored items, SD did not impair the precision of representations held in VSTM. The delay itself, however, did reduce precision in these representations.
SD Increases the Probability of VSTM Failure
The passage of time38,39 or interference from other stimuli during delay intervals causes memories to decline.40 In agreement with our findings, even delays containing no intervening distractors lead to reduced probability of successful item recall during continuous report tasks.26,27 Because the current study shows that SD augments these delay-related declines, lack of sleep appears to compromise the active maintenance of VSTM.
Neurophysiological experiments support the notion of active memory maintenance over delay intervals. Memoranda in short-term memory are maintained in reverberatory neural loops that involve visual cortex21,41,42 and top-down attentional control regions in frontal and parietal cortices.22,43–45 The involvement of attentional regions underscores the close link between attention and VSTM, two processes that are conceptually and functionally intertwined.46 In addition, activity in these brain regions is attenuated during SD across a variety of tasks, including those involving preparatory attention,47 VSTM,10,11 selective attention,48 and visual processing capacity.1,2 Such changes provide a neural basis for the State by Delay interaction in VSTM capacity observed in the current experiment.
Although our results show that delay exacerbates the effects of SD on VSTM, we also observed a significant State effect even at the 1-sec delay. This finding is consistent with two possibilities. First, items maintained in VSTM may show a noticeable decline even after a brief retention interval. Second, participants may fail to encode some or all of the items in the memory array during SD. Such an encoding failure may occur even though we ensured that participants' eyes were open and their gaze was directed at the monitor throughout the encoding period. These procedures might be expected to reduce state effects, so it is perhaps curious that several previous studies have failed to observe large state effects with short retention spans.23–25,49 Key differences in study design could account for the divergent findings. For example, our stimuli were backward masked, which required participants to form robust memories quickly. Our participants may have been more impaired generally, as we tested them after a night of SD at the circadian nadir of behavioral performance (at approximately 06:00) instead of during the early evening. Finally, our continuous report technique is likely more sensitive than change detection methods.50
In addition to being more sensitive to our manipulations, the continuous report technique also allowed us to assess the relative frequency of two different types of errors: reports of nontarget items (β) and random guesses (γ). Nontarget items may correspond to misbinding of colors and locations in memory.28,51 Alternatively, when participants cannot remember the probed item, they may report the color of one of the items they do remember instead of responding randomly. β estimates may therefore overestimate binding errors. Regardless, these errors are less frequent than random guesses across all conditions (Table 2), suggesting that successful recall of items and features—not just the ability to form and maintain bound representations—is impaired by delay and SD.
SD Does Not Affect the Precision of Items in VSTM
In marked contrast to SD's reduction of the number of items in VSTM, SD had no effect on the precision of remembered items. This result is consistent with the notion that representations in VSTM do not decay gracefully, but instead terminate suddenly and completely.26,27,52 On this account, a small change in a system component (e.g., the availability of VSTM resources) could lead to a large change in behavior (e.g., the ability to report an item). Such nonlinear dynamics have been posited in computational models of short-term or working memory.52
Items in VSTM, however, do appear to degrade gracefully under certain conditions. For example, changes in precision have been demonstrated for both simple objects28,53 and feature conjunctions.50 Indeed, in the current study we observed a significant decrease in memory precision across our tested delay. Earlier studies may have failed to observe this effect due to a lack of power26 or a relatively short delay period.27
Although we show that memory representations in our task can lose precision when confronted with certain challenges (i.e. delay), the fact that SD induces no such change while reducing the number of stored items implies that SD's effects are qualitatively different. We speculate that system instability, indexed by behavioral lapses17,54 or local sleep,18,19 causes complete failures in VSTM instead of the more gradual effects that a change in precision represents. Intriguingly, such failures need not encompass more than the probed item or even a given feature.55 Indeed, the presence of nontarget item reports (β) is consistent with successful recall of at least the color of nontargets. A binding error implies that a nonprobed item's color was successfully reported even though its location was not.
We previously suggested that neural changes associated with the SD-related loss of VSTM capacity are reflected in the reduction of task-related activation observed with functional magnetic resonance imaging.10 As relatively lower activation occurred even for remembering a single item, it is possible that redundant neural circuits are recruited to maintain VSTM representations in the rested state. This redundancy may be important to ensuring optimal performance. Although a critical minimum level of activation may often allow for task goals to be fulfilled in SD, slight perturbations to the network may lead to transient, stochastic failures in maintaining attention or short-term memoranda.47
CONCLUSION
SD selectively reduces the number of integrated representations that can be retrieved after a delay, while leaving the precision of object information in the stored representations intact. This dissociation adds to the evidence that SD may unevenly affect select cognitive processes,56–58 perhaps aiding our understanding of their normal functions and limitations.
A commentary on this article appears in this issue on page .
ACKNOWLEDGMENTS
This work was supported by grants awarded to Dr Michael Chee from the National Medical Research Council Singapore (StaR/0004/2008).
Comments