## Abstract

The visual system is confronted with rapidly changing stimuli in everyday life. It is not well understood how information in such a stream of input is updated within the brain. We performed voltage-sensitive dye imaging across the primary visual cortex (V1) to capture responses to sequences of natural scene contours. We presented vertically and horizontally filtered natural images, and their superpositions, at 10 or 33 Hz. At low frequency, the encoding was found to represent not the currently presented images, but differences in orientation between consecutive images. This was in sharp contrast to more rapid sequences for which we found an ongoing representation of current input, consistent with earlier studies. Our finding that for slower image sequences, V1 does no longer report actual features but represents their relative difference in time counteracts the view that the first cortical processing stage must always transfer complete information. Instead, we show its capacities for change detection with a new emphasis on the role of automatic computation evolving in the 100-ms range, inevitably affecting information transmission further downstream.

## Introduction

Characterization of stimulus–response relationship is the most fundamental approach to accessing cortical coding behavior. This procedure starts with the assumption that neuronal populations sample sensory input and form faithful internal representations of its actual content. In fact, classical reverse-correlation techniques enable us to determine neuronal tuning properties by backtracking responses to stimulus variations across rapid sequences of presentation (Eckhorn et al. 1993; Ringach et al. 1997). Thus, these techniques build on the idea that neuronal activity is permanently updated by current stimulation, maintaining an ongoing representation of the outer world (Jonides et al. 1982).

To measure the continuous dynamics of cortical population activity, we used voltage-sensitive dye imaging, which reflects gradual changes in membrane potentials across several square millimeters of cortex with an emphasis on superficial layers (Grinvald et al. 1994; Petersen et al. 2003; Jancke et al. 2004; Chen et al. 2006; Roland et al. 2006; Berger et al. 2007; Sit et al. 2009; see Grinvald and Hildesheim 2004 for review). This method does not provide single neuron activity or its dependence on different cortical layers. On the upside, it avoids biased sampling of neurons and it captures population activity irrespective of receptive field locations and preferred feature selectivities (Lee et al. 1988; Vogels 1990; Jancke et al. 1999; Tsodyks et al. 1999; Jancke 2000; Dinse and Jancke 2001; Pouget et al. 2003; Graf et al. 2011; Gilad et al. 2012; Lewis and Lazar 2013), hence, providing the global tuning of the cortex across millions of neurons under different stimulus conditions.

We report, using recordings in cat visual cortex (V1), that ongoing encoding can be found for the representation of briefly presented stimulus sequences (33 Hz) consistent with earlier studies (Ringach et al. 1997; Benucci et al. 2009). However, slower image sequences (10 Hz) reveal an essential addition. Population tuning in the primary visual cortex no longer represents the complete image content, but rather those orientations that were newly added or removed. We propose that such a precise detection of change across sequences of natural scene contours involves the interplay between 2 well-known neuronal behaviors, adaptation and stimulus off-responses (Movshon and Lennie 1979; Duysens et al. 1996; Müller et al. 1999; Bair et al. 2002; Dragoi et al. 2002; Felsen et al. 2002) that are important for stimulus transitions (Eriksson et al. 2008, 2012) and interact here to encode the difference to past images. In combination with eye movements at different spatiotemporal scales, the observed frequency-dependent encoding of image content might help to remove predictable input correlations in order to emphasize object borders and discontinuities within natural scenes (Desbordes and Rucci 2007; Rucci et al. 2007; Rucci 2008; Kuang et al. 2012). We conclude that input timing may entail predictive encoding (Rao and Ballard 1999; Friston 2005) at the very first cortical processing stages without the involvement of voluntary or attentional top-down mechanisms.

## Materials and Methods

### Visual Stimuli and Presentation

#### Construction of Oriented Stimuli (V, H) and Superpositions (VH)

We presented sequences of stimuli to anesthetized cats (11 males, 4 females, adult). First, we used 128 natural images (64 urban and 64 nature scenes) in grayscale to construct stimuli with dominant vertical (V) and horizontal (H) orientation. We derived these oriented stimuli by filtering the natural images in Fourier space with real-valued, polar-separable filters. The angular function of these filters was a triangular hat function, symmetric across 180°, with its maxima of 1 at either vertical (for stimulus type V) or horizontal (for stimulus type H) frequency. The half width at half maximum was 45° for the construction of broadly filtered images, and 2.8° for the construction of narrowly filtered images. The radial function (adapted from Simoncelli and Farid 1996) included a low-pass filter with cutoff at 6.6 c/° visual angle, and the DC component was set to zero. From the first harmonic up to 6.0 c/° (start of low-pass transition range), the function was 1. Thus, in this range, we preserve the relative amplitudes of the original image, including the characteristic 1/f fall-off for natural stimuli.

Next, we constructed superpositions (VH) of vertical (V) and horizontal (H) stimuli by summing them (in image space). We normalized the stimuli such that the global contrast of all superpositions was the same (rms contrast: 0.71); this means that component stimuli (V and H) were scaled with the same factor as their superposition (VH). During this normalization, we had to clip some pixels of images with a large intensity range (in average 2%, at most 15%).

We also presented vertical (V) and horizontal (H) square-wave gratings (0.2 c/°) and their linear superpositions (VH, contrast also 0.71, phases were varied over repetitions). In addition to oriented stimuli and their superpositions, we used an isoluminant screen as blank (B) stimulus. All stimuli were gamma-corrected according to the presentation monitor (100 Hz, Sony Triniton GDM-FW900, Japan). Mean luminance of each stimulus, including blank, was 20 cd/m2. Stimuli covered a visual field of 31° × 31°.

#### Stimulus Sequences

From the 4 stimulus types: (V) vertical stimulus, (H) horizontal stimulus, (VH) superposition, and (B) blank stimulus, we created pseudorandom sequences including all 16 (42) possible transitions, that is, switches, between them, resulting in 17 stimuli per sequence. We constructed 64 different sequences optimizing the following criteria: As response variance across different experimental trials is relatively high, we applied the constraints that all stimulus types (V, H, VH, and B) should occur equally often (4 times within each 10-Hz sequence) and that every switch should occur equally often (once within each 10-Hz sequence). To avoid systematic effects in the responses to particular switch types in dependence of their position in the sequence (start, middle, or end), we randomized positions of both stimulus type and transition type across the 64 sequences (see Fig. 1 for a sketch of the paradigm and averaging procedure).

Figure 1.

Illustration of the switch-triggered average. Three pseudorandom stimulus sequences are exemplified (5 stimuli are shown; in the experiments we used at least 64 different sequences of at least 17 stimuli, see Materials and Methods). Sequences were aligned to a specific switch between a pair of stimuli (blank to horizontal in this example) to illustrate our averaging procedure. Note that when aligning sequences to a particular switch type for averaging, the stimuli before and after were different. For switch-triggered averaging, the procedure was used to average the cortical responses to a particular stimulus pair across all sequences shown.

Figure 1.

Illustration of the switch-triggered average. Three pseudorandom stimulus sequences are exemplified (5 stimuli are shown; in the experiments we used at least 64 different sequences of at least 17 stimuli, see Materials and Methods). Sequences were aligned to a specific switch between a pair of stimuli (blank to horizontal in this example) to illustrate our averaging procedure. Note that when aligning sequences to a particular switch type for averaging, the stimuli before and after were different. For switch-triggered averaging, the procedure was used to average the cortical responses to a particular stimulus pair across all sequences shown.

These 64 sequences were repetitively shown with 2 different presentation frequencies, at 10 or 33 Hz. For 10-Hz presentation (100-ms presentation of each stimulus), one sequence was shown per trial. Here, stimulus presentation per trial lasted 1700 ms. For 33-Hz presentation (30-ms presentation of each stimulus), we showed 3 sequences in 1 trial (1530 ms per trial). Optical data were recorded for 2 s per trial, including a 200-ms baseline, in both cases. The relatively short trial durations were used to avoid dye bleaching effects, photodynamic damage of the cortical tissue, and possible contamination by intrinsic signals (Grinvald and Hildesheim 2004). The intertrial interval, in which a blank stimulus was shown, was set to a minimum of 5 s. When 2 stimulus conditions were used (such as narrowly filtered and broadly filtered), these were randomized across trials. In between stimulus trials, we recorded blank conditions, where an isoluminant gray screen was shown for 2 s, to allow correction of breathing and heart beat artifacts (2 blanks for 16 stimulus trials).

#### Experimental Protocols

In the main body of imaging experiments, we used 10-Hz presentation frequency and the 2 stimulus conditions narrowly filtered and broadly filtered images (Fig. 4 and 7, 12 experiments, 256–896 repetitions of each switch type). In one additional imaging experiment, we used oriented square-wave gratings for 10-Hz presentation (Fig. 4 (bottom row) and 5 (imaging trace), 236 repetitions of each switch type). To recapitulate previous optical imaging results (Benucci et al. 2009), in one experiment (Figs 2, 3, and 4 (upper row), 384 repetitions of each switch type), we used 33-Hz presentation frequency and 2 different stimulus conditions: “gratings” and narrowly filtered images. Finally, instead of using long stimulus sequences we presented 2 stimuli (i.e. isolated single switches) in 1 experiment. Here, we concentrated on the most informative switch type, superpositions to a single orientation (VH to V, and VH to V). Gratings were used in 3 different timing conditions: The superposition (VH) was shown for 33, 100, and 500 ms before switches to a single orientation (V or H) occurred (Fig. 6, 80 repetitions).

Figure 2.

Visual stimulation and cortical representation of oriented contours. Top: Example stimulus, a natural image (left, purple), and its horizontally (red) and vertically (blue) filtered versions using broad filters in Fourier space (second column). Here and in subsequent figures colored edges of stimulus icons indicate stimulus type (horizontal stimulus (H): red horizontal edge; vertical stimulus (V): blue vertical edge; superposition (VH): purple horizontal and vertical edges). (AC) Time sequences of stimuli with different orientation content, presented either at 30- or 100-ms stimulus periods (see outlined arrows): broad filters (A), narrow filters (B), and square-wave grating (C). (D) Switch from a vertical to a horizontal square-wave grating (30-ms stimulus period): the most active 30% of pixels in each 10-ms camera frame are shown and overlaid on the vascular cortical image (averages over 384 switch repetitions). Here and in subsequent plots, data were shifted by 50 ms to account for average latency; the green line at zero marks relative time to switch. Pixel color denotes preferred orientation derived from a conventional orientation map (most left), which was recorded separately (see Materials and Methods). (E) Population tuning curves for each time frame obtained by averaging activities over pixels with the same orientation preference (preferences were binned into 18 classes of 10°). Horizontal orientation bin (0°, red) is displayed twice. Abscissa shows amplitudes of tuned activity (ΔF/F, see Materials and Methods). (F) Switch-triggered average as compact illustration of the data in E. Colorbar shows amplitudes of tuned activity. Orientation tuning curves to the left and right of the central frame illustrate time averages (−40 to 0, and 0 to +40 ms). Dashed faint lines illustrate time shifted (50 ms) positions of transitions from and to other stimuli (before and after the switch), which were averaged across all sequences by switch-triggered averaging (see Fig. 1 and Materials and Methods).

Figure 2.

Visual stimulation and cortical representation of oriented contours. Top: Example stimulus, a natural image (left, purple), and its horizontally (red) and vertically (blue) filtered versions using broad filters in Fourier space (second column). Here and in subsequent figures colored edges of stimulus icons indicate stimulus type (horizontal stimulus (H): red horizontal edge; vertical stimulus (V): blue vertical edge; superposition (VH): purple horizontal and vertical edges). (AC) Time sequences of stimuli with different orientation content, presented either at 30- or 100-ms stimulus periods (see outlined arrows): broad filters (A), narrow filters (B), and square-wave grating (C). (D) Switch from a vertical to a horizontal square-wave grating (30-ms stimulus period): the most active 30% of pixels in each 10-ms camera frame are shown and overlaid on the vascular cortical image (averages over 384 switch repetitions). Here and in subsequent plots, data were shifted by 50 ms to account for average latency; the green line at zero marks relative time to switch. Pixel color denotes preferred orientation derived from a conventional orientation map (most left), which was recorded separately (see Materials and Methods). (E) Population tuning curves for each time frame obtained by averaging activities over pixels with the same orientation preference (preferences were binned into 18 classes of 10°). Horizontal orientation bin (0°, red) is displayed twice. Abscissa shows amplitudes of tuned activity (ΔF/F, see Materials and Methods). (F) Switch-triggered average as compact illustration of the data in E. Colorbar shows amplitudes of tuned activity. Orientation tuning curves to the left and right of the central frame illustrate time averages (−40 to 0, and 0 to +40 ms). Dashed faint lines illustrate time shifted (50 ms) positions of transitions from and to other stimuli (before and after the switch), which were averaged across all sequences by switch-triggered averaging (see Fig. 1 and Materials and Methods).

Figure 3.

Ongoing cortical encoding of current orientation with sequences of short 30-ms stimulus periods. Frames resolve population tuning around stimulus switch-time (green line) in 10-ms steps (−20 to +40 ms, averages over 384 repetitions of each switch, same conventions as in Fig. 2F). The icons on top depict switch conditions. Black traces to the right of each frame show time average: 0 to +40 ms. The first four conditions on the left are characterized by a change in horizontal orientation (either turned on or off). Each stimulus has different orientation content after the switch, except for the switch to blank (4th frame). The last four conditions cover corresponding changes in vertical orientation. Note that responses before the switch in 30-Hz sequences can partly include responses to previous stimuli (first dashed line Fig. 2F), particularly visible in columns #3, when the switch occurred from blank that naturally leads to a low amplitude of activity. Tuning curves obtained with voltage-sensitive dye imaging (Sharon and Grinvald 2002) are generally broader than for spike recordings (Benucci et al. 2009) and hence, superimposed gratings caused per se relatively flat distributions. Also our stimuli were not of highest contrast and evoked lower modulation depth, which in turn produces higher sensitivity to noise (Grabska-Barwinska et al. 2009). Thus, responses to superimposed gratings did not reveal a clear bimodal distribution when averaged over brief time intervals (but see Fig. 6 for longer time averages). All switches P < 0.001 in comparison to flat response; except H to VH: P = 0.97; after fitting for all residuals P > 0.99; except VH to H: res. P = 0.87; B to V: res. P = 0.98; H to B: res. P = 0.73; V to B: res. P = 0.26 (see Materials and Methods and Supplementary Tables S1–S3 for details).

Figure 3.

Ongoing cortical encoding of current orientation with sequences of short 30-ms stimulus periods. Frames resolve population tuning around stimulus switch-time (green line) in 10-ms steps (−20 to +40 ms, averages over 384 repetitions of each switch, same conventions as in Fig. 2F). The icons on top depict switch conditions. Black traces to the right of each frame show time average: 0 to +40 ms. The first four conditions on the left are characterized by a change in horizontal orientation (either turned on or off). Each stimulus has different orientation content after the switch, except for the switch to blank (4th frame). The last four conditions cover corresponding changes in vertical orientation. Note that responses before the switch in 30-Hz sequences can partly include responses to previous stimuli (first dashed line Fig. 2F), particularly visible in columns #3, when the switch occurred from blank that naturally leads to a low amplitude of activity. Tuning curves obtained with voltage-sensitive dye imaging (Sharon and Grinvald 2002) are generally broader than for spike recordings (Benucci et al. 2009) and hence, superimposed gratings caused per se relatively flat distributions. Also our stimuli were not of highest contrast and evoked lower modulation depth, which in turn produces higher sensitivity to noise (Grabska-Barwinska et al. 2009). Thus, responses to superimposed gratings did not reveal a clear bimodal distribution when averaged over brief time intervals (but see Fig. 6 for longer time averages). All switches P < 0.001 in comparison to flat response; except H to VH: P = 0.97; after fitting for all residuals P > 0.99; except VH to H: res. P = 0.87; B to V: res. P = 0.98; H to B: res. P = 0.73; V to B: res. P = 0.26 (see Materials and Methods and Supplementary Tables S1–S3 for details).

Figure 4.

Primary visual cortex represents difference in orientation content compared with past. Conventions and layout as in Figure 3 columns #1 and #2. First row is for comparison with short 30-ms stimulus period: Responses to narrow filtered natural stimuli, 1 experiment (384 switch repetitions, V/H to VH: P > 0.99 no tuning; VH to V: P < 0.01 tuned [residual P = 0.99]; VH to H: P < 0.02 tuned [res. P = 0.88]). Bottom rows: 100-ms stimulus period, narrowly filtered natural stimuli (second row; average modulation depth: 0.6 ± 0.2 ×10−5), broadly filtered (third row; average modulation depth: 0.5 ± 0.1 × 10−5), and gratings (last row; average modulation depth: 2.1 × 10−5). Blue arrows highlight positions of vertical tuning (90°), and red arrows indicate horizontal position (0°). Note that tuning peaked around removed (or added) orientations instead of currently presented orientations (for narrowly and broadly filtered stimuli, all displayed switches: P < 0.001 tuned [residuals: P > 0.99], 12 experiments [256–896 switch repetitions each], average is shown; for gratings, all displayed switches: P < 0.001 tuned [res.: V to VH: P = 0.96, VH to V: P = 0.69, H to VH: P = 0.65, VH to H: P > 0.99], 1 experiment, 236 switch repetitions).

Figure 4.

Primary visual cortex represents difference in orientation content compared with past. Conventions and layout as in Figure 3 columns #1 and #2. First row is for comparison with short 30-ms stimulus period: Responses to narrow filtered natural stimuli, 1 experiment (384 switch repetitions, V/H to VH: P > 0.99 no tuning; VH to V: P < 0.01 tuned [residual P = 0.99]; VH to H: P < 0.02 tuned [res. P = 0.88]). Bottom rows: 100-ms stimulus period, narrowly filtered natural stimuli (second row; average modulation depth: 0.6 ± 0.2 ×10−5), broadly filtered (third row; average modulation depth: 0.5 ± 0.1 × 10−5), and gratings (last row; average modulation depth: 2.1 × 10−5). Blue arrows highlight positions of vertical tuning (90°), and red arrows indicate horizontal position (0°). Note that tuning peaked around removed (or added) orientations instead of currently presented orientations (for narrowly and broadly filtered stimuli, all displayed switches: P < 0.001 tuned [residuals: P > 0.99], 12 experiments [256–896 switch repetitions each], average is shown; for gratings, all displayed switches: P < 0.001 tuned [res.: V to VH: P = 0.96, VH to V: P = 0.69, H to VH: P = 0.65, VH to H: P > 0.99], 1 experiment, 236 switch repetitions).

#### High-Contrast Moving Gratings

For calibration and mapping of orientation preference, we used moving square-wave gratings (rms contrast: 1, 0.2 c/°, 6 Hz, mean luminance 53 cd/m2 [8 hemispheres], or 38 cd/m2 [9 hemispheres]) with 4 different orientations (0°, 45°, 90°, 135°) and both motion directions. We recorded for 1 s in each trial, including 200-ms prestimulus time. These calibration trials were recorded throughout the entire experiment in between blocks of the main stimulus protocol of each experiment (specified above).

### Preparations for Optical Imaging

All animal experiments were carried out in accordance with the European Union Community Council guidelines and approved by the German Animal Care and Use Committee (application number: AZ 9.93.2.10.32.07.032) in accordance with the Deutsches Tierschutzgesetz (§ 8 Abs. 1) and the NIH guidelines. For further details, see Onat, Nortmann et al. (2011). In brief, animals were initially anesthetized with ketamine (20 mg per kg i.m.) and xylazine (1 mg per kg i.m.), artificially respirated, continuously anesthetized with 0.8–1.5% isoflurane in a 1:1 mixture of O2/N2O, and fed intravenously. Both weak effects on neuronal tuning properties (e.g. Niell and Stryker 2010) and strong modulations (e.g. Adesnik et al. 2012) have been reported when comparing anesthetized and awake states. This might additionally depend on the type of anesthetics used. Therefore, it is an interesting question, and it remains to be generally tested, in how far results obtained under anesthesia hold in behavioral settings. However, our anesthetized and paralyzed preparation provides the advantage that eyes are fixed and hence allow complete control of the dynamics of the visual input. We administered 0.4 mg/kg dexamethasone i.m. and 0.05 mg/kg atropine sulfate i.m. daily and 20 mg/kg cephazolin twice a day. In few control experiments, we used contact lenses with a 3-mm diameter pupil. Heart rate, intratracheal pressure, exhaled CO2, and body temperature were monitored. The skull was opened above area V1 (A18, occasionally parts of A17), the dura was removed, a chamber was mounted, the cortex was stained for 3 h (and occasionally re-stained) with voltage-sensitive dye (RH-1691), and unbound dye was washed out.

### Data Acquisition and Preprocessing

Optical imaging was conducted with Imager 3001 (Optical Imaging, Inc., Mountainside, NY, USA). The camera was focused ∼500 µm below the cortical surface. Data acquisition onset was synchronized with heartbeat and respiration. For detection of changes in fluorescence, the cortex was illuminated with light of wavelength 630 ± 10 nm, and emitted light at wavelengths above 665 nm was collected. The frame rate was set to 100 Hz. We performed normalization by dividing each pixel value by its average 200-ms prestimulus activity; heartbeat and respiration-related artifacts were removed by subtracting the average blank signal. These preprocessing steps lead to a unitless relative signal of fluorescence, denoted by ΔF/F. For the main paradigm (10-Hz narrowly/broadly filtered), we excluded 5 hemispheres from analysis because of insufficient staining. Data were used when the amplitude of the evoked response in the Fourier power spectrum at 10 Hz (the switch-type unspecific response) was at least 3 times larger than at surrounding frequencies (±2 Hz).

### Electrophysiology

Electrode recordings served as a control of the voltage-related responses reported by the fluorescent optical signals. The recorded units were collected at a depth between 400 and 700 μm. Before electrophysiology, a vascular map of the brain was captured by illumination with green light (546 nm) from 2 optic fiber light guides. Additionally, the afterward measured orientation maps were overlaid. This combined map was then used to guide electrode penetrations to orientation-selective domains. Spikes were sorted online by a multiple spike detector, MSD (Alpha Omega Engineering, Ltd., Israel). Cells were selected upon differences in spike-wave forms. Multiple unit activity (MUA, mostly 3–4 cells, occasionally we recorded single units in addition to MUA (5 of 27 sites)) was recorded with tungsten electrodes (0.8–2 M

Ω
, WPI, Inc., USA). For display purposes (Fig. 5A, D), the data were convoluted in a 5 ms window.

Figure 5.

Electrophysiological recordings. (A) Multiunit activity (MUA) versus optical signal. Blue trace shows averages over 32 recordings (4 different hemispheres) and over randomized 10-Hz sequences of gratings. Black trace outlines the time course of the unfiltered optical signal (average over space and all 10-Hz sequences, 1 experiment). (B) Log power spectrum of the evoked dye signal (in the steady phase 900–1900 ms). Spectrum of trace in A (black bold line) and the log spectra for filtered natural stimuli of all 12 experiments (narrowly filtered stimuli, gray dotted line shows mean, shaded area shows std). First arrow points to stimulus frequency (10 Hz), the dotted arrow indicates first harmonic (20 Hz). (C) Example spike recording. Spike responses to moving gratings of four different orientations (temporal average over 800 ms) indicating orientation tuning to horizontal (red axis). (D) Responses to either horizontal (red) or vertical (blue) orientation after presentation of their superposition, black trace plots difference. (E) Summary: Differences between responses to vertical and horizontal moving gratings (i.e., orientation tuning, cf. black outline in C) are plotted against differences in responses to vertical and horizontal after presentation of their superposition (cf. black trace in D). Time averages across the same time window as used for the imaging data (see gray shading in D). Each circle plots MUA (n = 32, red dot represents example shown in C, D). Spike responses tend to be higher after the preferred orientation was switched-off from the superposition (gray shaded quadrants) than when it was the remaining orientation; Fisher's exact test one-tailed P = 0.016; inset shows count for each quadrant.

Figure 5.

Electrophysiological recordings. (A) Multiunit activity (MUA) versus optical signal. Blue trace shows averages over 32 recordings (4 different hemispheres) and over randomized 10-Hz sequences of gratings. Black trace outlines the time course of the unfiltered optical signal (average over space and all 10-Hz sequences, 1 experiment). (B) Log power spectrum of the evoked dye signal (in the steady phase 900–1900 ms). Spectrum of trace in A (black bold line) and the log spectra for filtered natural stimuli of all 12 experiments (narrowly filtered stimuli, gray dotted line shows mean, shaded area shows std). First arrow points to stimulus frequency (10 Hz), the dotted arrow indicates first harmonic (20 Hz). (C) Example spike recording. Spike responses to moving gratings of four different orientations (temporal average over 800 ms) indicating orientation tuning to horizontal (red axis). (D) Responses to either horizontal (red) or vertical (blue) orientation after presentation of their superposition, black trace plots difference. (E) Summary: Differences between responses to vertical and horizontal moving gratings (i.e., orientation tuning, cf. black outline in C) are plotted against differences in responses to vertical and horizontal after presentation of their superposition (cf. black trace in D). Time averages across the same time window as used for the imaging data (see gray shading in D). Each circle plots MUA (n = 32, red dot represents example shown in C, D). Spike responses tend to be higher after the preferred orientation was switched-off from the superposition (gray shaded quadrants) than when it was the remaining orientation; Fisher's exact test one-tailed P = 0.016; inset shows count for each quadrant.

For these electrophysiological recordings, we presented oriented square-wave gratings at 10 Hz. We adopted our corresponding imaging paradigm (see above) to use longer trial durations. This allowed us to show sequences of 66 stimuli (6600-ms presentation duration per trial) consisting of the 64 possible triplets of the 4 stimulus types (V, H, VH, and B). Positions of triplet types within sequences were randomized across trials. Data were recorded in 4 hemispheres (in 2 of which we also performed optical imaging).

### Analysis

#### Orientation Maps

Maps were computed using data from high-contrast, square-wave moving gratings. After trial-wise preprocessing, we averaged data over repetitions (50–119 repetitions), motion direction, and time. The resulting dataset was spatially band-pass filtered from 1 to 3 c/mm. The vertical-horizontal (VH) orientation maps (used for correlation analysis in Fig. 6E and Supplementary Fig. S1) were obtained by subtracting the horizontal from the vertical map. We also computed additive VH maps as a control (Fig. 6E). Orientation maps, which cover the full range of orientations, were computed based on the 4 measured orientations using vector summation, and downsampled to 18 bins of 10° each.

Figure 6.

Single switches. (A) Superimposed gratings were presented for 30, 100, or 500 ms (see dark gray box) before switching to either vertical or horizontal orientation (here and in all subsequent plots these switches are marked in black and gray, respectively). The intertrial interval, in which a blank stimulus was shown, was set to a minimum of 5 s (200 ms were recorded when a new stimulus started to obtain baseline activity, see white boxes). (B) Population tuning in response to the 6 stimulus conditions; switches to vertical along top row, horizontal switches at bottom. Same conventions as in previous figures (cf. Figs 2F, 3, 4). Temporal structure of stimulation is indicated at bottom (cf. A). Green lines and boxes mark the time window used to calculate effect size in C and maps in E (50–90 ms after switch). Insets: Bimodal distribution representing the constituent orientations of the superimposed gratings (time averages 50–150 ms after response onset). (C) Effect size. Data points specify average modulation depth (see Materials and Methods) dependent on different presentation times of the superimposed gratings (x-axis). Bars depict standard error (80 stimulus repetitions). Positive values indicate higher activity at vertically tuned pixels (blue icon) and negative values higher activity at horizontal pixels (red icon). When testing for differences between timing conditions, pooling vertical and horizontal conditions (sign inverting the latter), we found significant differences between the 30- and the 100-ms condition (pairwise two-tailed t-test: 30 vs. 100 ms P< 0.001, n = 160), but no significant differences between the 100- and 500-ms conditions (P = 0.79). (D) Time courses of global activity (unfiltered spatial averages across imaging frames) in response to the 6 stimulus conditions. Shaded areas depict the temporal structure of stimulus conditions (cf. A). (E) Correlation of the cortical activity patterns with a standard VH orientation map (see Materials and Methods and Supplementary Fig. S1). Positive values indicate similarity with activity pattern representing vertical orientation (icon on top), negative values indicate correlations to the horizontal map. On the right of each graph the 30% most active pixels across the imaged cortex are shown for each condition (time averages of 4 imaging frames, 50–90 ms after switches). Colors code preferred orientation at each pixel location (same conventions as in Fig. 2D). Upper maps visualize population tuning to a switch to vertical (outlined black), lower maps show switches to horizontal (outlined gray). Dotted lines show correlations with the additive map instead of the standard subtractive VH orientation map (see Materials and Methods) as control.

Figure 6.

Single switches. (A) Superimposed gratings were presented for 30, 100, or 500 ms (see dark gray box) before switching to either vertical or horizontal orientation (here and in all subsequent plots these switches are marked in black and gray, respectively). The intertrial interval, in which a blank stimulus was shown, was set to a minimum of 5 s (200 ms were recorded when a new stimulus started to obtain baseline activity, see white boxes). (B) Population tuning in response to the 6 stimulus conditions; switches to vertical along top row, horizontal switches at bottom. Same conventions as in previous figures (cf. Figs 2F, 3, 4). Temporal structure of stimulation is indicated at bottom (cf. A). Green lines and boxes mark the time window used to calculate effect size in C and maps in E (50–90 ms after switch). Insets: Bimodal distribution representing the constituent orientations of the superimposed gratings (time averages 50–150 ms after response onset). (C) Effect size. Data points specify average modulation depth (see Materials and Methods) dependent on different presentation times of the superimposed gratings (x-axis). Bars depict standard error (80 stimulus repetitions). Positive values indicate higher activity at vertically tuned pixels (blue icon) and negative values higher activity at horizontal pixels (red icon). When testing for differences between timing conditions, pooling vertical and horizontal conditions (sign inverting the latter), we found significant differences between the 30- and the 100-ms condition (pairwise two-tailed t-test: 30 vs. 100 ms P< 0.001, n = 160), but no significant differences between the 100- and 500-ms conditions (P = 0.79). (D) Time courses of global activity (unfiltered spatial averages across imaging frames) in response to the 6 stimulus conditions. Shaded areas depict the temporal structure of stimulus conditions (cf. A). (E) Correlation of the cortical activity patterns with a standard VH orientation map (see Materials and Methods and Supplementary Fig. S1). Positive values indicate similarity with activity pattern representing vertical orientation (icon on top), negative values indicate correlations to the horizontal map. On the right of each graph the 30% most active pixels across the imaged cortex are shown for each condition (time averages of 4 imaging frames, 50–90 ms after switches). Colors code preferred orientation at each pixel location (same conventions as in Fig. 2D). Upper maps visualize population tuning to a switch to vertical (outlined black), lower maps show switches to horizontal (outlined gray). Dotted lines show correlations with the additive map instead of the standard subtractive VH orientation map (see Materials and Methods) as control.

#### Switch-Triggered Responses

To obtain responses to each of the 16 possible switches (i.e. the pairs of 4 different stimulus types: V, H, VH, and B (blank)), we first removed responses unspecific to switch type by pixel-wise subtraction of the average response across the entire stimulus sequences. We then computed switch-triggered responses for each of the 16 switch types. This was done by aligning responses to a particular switch in time and then averaging over repetitions measured in different sequences. Thereby we averaged over responses to varying stimuli before and after a specific switch. The procedure to compute switch-triggered averages is illustrated in Figure 1 (a comparison to stimulus-triggered averaging is provided in Supplementary Fig. S2). When using filtered natural stimuli the 2 image categories (urban/nature scenes) were pooled.

#### Population Tuning

The data were spatially band-passed from 1 to 3 c/mm. Population tuning was computed by averaging responses across pixels with the same orientation preference (as determined by the vector-based orientation map). Overall population tuning was then obtained by averaging across experiments (10 Hz narrowly/broadly filtered n = 12, 10 Hz gratings n = 1, 33 Hz narrowly filtered/gratings n = 1) and time (50–90 ms after switch).

#### Modulation Depth and Tuning Width

Modulation depth was calculated as the average difference between activity at pixels preferring 90 ± 15° and 0 ± 15° (time window 50–90 ms after switch). We tested for differences between broadly and narrowly filtered stimuli, using responses from blank (B) to vertical (V) and horizontal (H) orientations. We first averaged over vertical and horizontal conditions, sign-inverting the latter, and subsequently we used a pairwise 2-tailed t-test to compare filter conditions within experiments (n = 12). Tuning width was estimated by fitting a Gaussian with 3 parameters (amplitude, width, and basis level) for each experiment and filter condition, and using a paired 2-tailed t-test comparing filter conditions within experiments. We only used experiments for this subsequent test, in which the fitted Gaussian yielded a fit better than 80% of variance explained (the residual variance divided by the variance of the data, subtracted from one), which was the case in 8 of 12 experiments. In general, before conducting t-tests, we tested the respective distributions for normality using a Lilliefors test (α = 0.05).

#### Statistical Evaluation of Population Tuning

To evaluate population tuning statistically we used a 3-step procedure. First, we quantified how much the time-averaged orientation-tuning curves deviated from a flat orientation-tuning curve (zero-baseline). To quantify this difference to zero, we computed one χ2-test statistic. This allowed us to include the data from all experiments in an overall quantification, while at the same time accounting for differences in sample sizes, variances, and response levels across different experiments. The χ2-test statistic is computed across all experiments (experiment e = 1,…,n; where n is the number of experiments), and orientation bins (bin

θ
= 1,…,18; bins of 10° each), taking the standard error over repetitions of a switch type (repetition re = 1,…,me; where re is a repetition in experiment e, and me is the overall number of repetitions in experiment e) in each experiment as a normalization factor:
$χ2=∑e∑θse,θ−⟨de,θ,re⟩reσ/me2$

$σ=1me−1∑re(de,θ,re−⟨de,θ,re⟩re)2$

Here, $⟨⟩re$ denotes the average over repetitions in experiment e and $de,θ,re$ the response in experiment e, orientation bin

θ
, and repetition re. In this first step, we wanted to quantify the difference between our data and zero. Thus we used se,
θ
= 0 as constant baseline. The degrees of freedom are df1 = 18n−1, corresponding to 18 orientation bins and n experiments.

In the second step of the evaluation of population tuning, we fitted sinusoids to the population tuning curves and repeated the quantification on the residuals. We used the sinusoid function $se,θ:=aecos(10∘θ(2π/18))$ with amplitude ae, which is fitted for each experiment, e, by minimizing $ϵe$ (for each experiment e).

$ϵe:=∑θ∑re(se,θ−de,θ,re)2$

Goodness of fit was then evaluated by computing the χ2-test statistic on the residuals as specified above (using the fitted sinusoids in place of the constant baseline).

In this second step, there are fewer degrees of freedom, df2 = 18n−1−n, because we did fit n parameters (10 Hz narrowly/broadly filtered n = 12, 10 Hz gratings n = 1, 33 Hz narrowly filtered/gratings n = 1).

In the third step of the procedure, we investigated the direction of population tuning. It is characterized by the sign of the amplitude of the sinusoid, ae, which was fitted in the second step. When ae is positive, the maximum of the sinusoid is at bin 18, which covers orientation preferences 175°–185°, and its minimum is at bin 9, which covers 85°–95°. Thus, positive amplitude ae indicates horizontal population tuning. Negative amplitude, correspondingly, indicates vertical population tuning.

## Results

To mimic natural viewing conditions (Betsch et al. 2004), stimuli were rapid sequences comprising blank (B), vertical (V), horizontal (H), superimposed orientations (VH) with different degrees of complexity (Fig. 2AC). We set up pseudorandom sequences from these 4 stimulus types such that each sequence included all 16 possible transitions. Activity was continuously recorded in 10-ms time frames (Fig. 2D), and switch-triggered averages were generated for each individual transition.

### Representation of Current Orientations

First, we verified that we can reproduce former findings within our settings. Figure 2DF depicts cortical responses to a switch (green line) from vertical to horizontal when square-wave gratings (Fig. 2C) were presented within short 30-ms stimulus sequences. Note the rapid change in color (bluish to reddish) that indicates subsequent activation of spatially distinct populations of neurons representing each stimulus orientation briefly before and after the switch. To obtain a compact depiction of overall population tuning over time, we remapped the data onto orientation space (Fig. 2E). Figure 2F summarizes, in 10-ms time frames, how peak population activity shifted from vertical to horizontal, with a transition phase of typically one to two 10-ms frames (color bar for activity levels, tuning profiles on left and right). Thus, as described in previous electrophysiological studies (Ringach et al. 1997; Gillespie et al. 2001) as well as in a recent work which evaluated both voltage-sensitive dye imaging and extracellular measurements for this paradigm (Benucci et al. 2009), we showed how the primary visual cortex acts as a straightforward “instantaneous decoder” (Benucci et al. 2009) by mapping of currently presented orientations.

Next, we tested whether this coding scheme also holds for complex stimuli that contain more than just a change to a single orientation. To answer this question, we introduced switches from a single grating orientation (V or H) to superposition (VH plaid) and vice versa (Eriksson et al. 2010; Nortmann et al. 2011). Superimposing an orthogonal orientation to the present one resulted in relatively flat distributions, representing the average of the individual orientation patterns (Fig. 3, columns #1, stimulus conditions on top; see figure legend), indicating an unbiased processing of both the sustained and added orientation (Busse et al. 2009; MacEvoy et al. 2009). Importantly, a change back from superposition to a single orientation led to responses tuned to the orientation present after the switch (Fig. 3, columns #2, see blue and red arrows). Therefore, current orientations were again directly encoded with a processing delay of ∼50 ms, similar to when turned on from blank (Fig. 3, compare columns #3). This scheme deviated exclusively for switches from an oriented stimulus back to blank (Fig. 3, columns #4). In this case, activity showed persistent tuning after the stimulus was turned off (Coltheart 1980; Duysens et al. 1985). In summary, our data using 33-Hz square-wave grating sequences confirmed the ongoing encoding of orientation, including the known exception of the tuned response after stimulus offset (Benucci et al. 2009).

Response characteristics obtained with simple artificial stimuli, like gratings of optimal spatial frequency used so far, do not necessarily generalize towards input of ecological relevance (Smyth et al. 2003; David et al. 2004; David and Gallant 2005; Felsen et al. 2005; Haider et al. 2010; Fournier et al. 2011; Onat, König et al. 2011). That is, response behavior to natural input can deviate significantly from predictions based on simple parameterized stimuli, probably due to the extensive spatial context in natural images (see Carandini et al. 2005; Olshausen and Field 2005 for reviews). Thus, it may be important to validate findings obtained with artificial stimuli using more natural stimulus conditions (Felsen and Dan 2005). To address this point in a first step, we extracted oriented contours from 128 different natural images (see Materials and Methods) by filtering them along the orientation dimension in Fourier space. In contrast to gratings, these images retain important properties of natural stimuli, such as the phase relationships and the typical 1/f fall-off of amplitudes along the spatial dimension (Simoncelli and Olshausen 2001; Geisler 2008; Hyvärinen et al. 2009; see Fig. 2B for example). Indeed, population tuning had an overall lower amplitude than for gratings (∼30%, cf. scale bars in Figs 3 and 4), most likely due to the heterogeneity of local contrasts in the natural stimuli, which may engage widespread population gain control reflected in the dye signal (Sit et al. 2009). Nevertheless, also for these stimuli, actual orientations after the switch were well represented (compare Fig. 4 first row with Fig. 3, columns #1 and #2).

### Representation of the Difference Between Past and Present Orientations

During natural vision changes of contour orientation can occur on relatively slow time scales (Gallant et al. 1998; Dragoi et al. 2002; Betsch et al. 2004; Kayser et al. 2004). Hence, in the following, we contrast the above scheme of ongoing encoding with the processing characteristics found for slower sequences of natural scene contours, using 100-ms stimulus periods (Fig. 4, bottom 3 rows). Strikingly here, responses to a switch from the vertical orientation to superimposed horizontal and vertical (Fig. 4, left column) represented almost exclusively the horizontal orientation, hence the orientation that was added rather than the present superposition. Likewise, switches from superposition to vertical (Fig. 4, second column) were followed by responses that were tuned to horizontal, thus representing the removed orientation instead of the remaining vertical orientation (compare first-row blue with bottom red arrows). The same characteristics were found for changes in vertical orientations (Fig. 4, 2 right columns; see Supplementary Fig. S1 for the correlation between population tuning over time and a standard VH orientation map).

Most remarkably, the mechanism worked precisely for natural scenes (Fig. 4, third row) in which the superposition of broadly filtered horizontal and vertical versions was almost identical to original images (see example Fig. 2, top). Therefore, even for the most complex stimuli that contained a rich mixture of multiple orientations, we found sensitive cortical tuning for changing orientations rather than for currently presented orientations.

Population tuning to turned-off orientations after a switch from the superposition to a single orientation (VH to V or H, 0.29 ± 0.12 mad, ×10−5 ΔF/F; see Materials and Methods and Supplementary Tables S1–S3; n = 48: medians across 12 experiments, both orientations, and both filter conditions of the natural images) suggests that responses to orientations that were turned-off are stronger than responses to orientations that sustained. When these components were measured directly, tuning amplitudes were indeed higher for turned-off orientations (switches from H or V to blank, 0.58 ± 0.28 mad, ×10−5 ΔF/F) than for sustained orientations (V to V or H to H, 0.14 ± 0.16 mad, ×10−5 ΔF/F; n = 48; paired two-tailed sign test P < 0.0001).

When comparing responses to narrowly and broadly filtered images, we calculated for the latter a decrease in modulation depth of 24 ± 7% (the difference between preferred and orthogonal responses, paired t-test P < 0.02), while we did not observe differences in tuning width (broadly filtered, HWHM 46 ± 2° sem, narrowly filtered, 49 ± 3° sem, pairwise difference 3 ± 3° sem, n = 8 experiments, P = 0.33). Because broadly filtered images provide enriched orientation content, a divisive normalization across populations of neurons with different preferred orientations (Busse et al. 2009; MacEvoy et al. 2009) may cause the observed decrease in modulation depth. For a proof of principle, we finally applied sequences of square-wave gratings in an additional experiment (Fig. 4 bottom). As expected, modulation depth was large (cf. colorbar) and also for those stimuli, we found a dominant representation of orientation change.

Even though VSD imaging may be a powerful tool to measure neuronal population dynamics with high spatiotemporal resolution, the relationship between the imaging signal and spiking activity is not entirely clear. Eriksson et al. (2008) suggest a close relationship between spike rate and the derivative of the VSD response rather than its magnitude. Such behavior might especially apply to the rising phase of the membrane potential after stimulus onsets (Jancke et al. 2004; Sit et al. 2009). Moreover, combined VSD and calcium-sensitive dye imaging suggest that the relationship between spiking activity and the amplitude of the VSD response depends on stimulus intensity (Berger et al. 2007). Chen et al. (2012) propose that these relationships are well captured by a power function with an exponent of ∼4, similarly as observed for the relationship between average membrane potential and spike rates in single V1 neurons (Anderson et al. 2000; Finn et al. 2007). However, dependencies on individual experimental settings, on the particular stimuli used, and on the likely differences between species, are widely unexplored. Hence, to address this issue, and to directly exclude the possibility that the dye signal levels reporting the difference to past orientations would merely reflect subthreshold activity (Petersen et al. 2003; Jancke et al. 2004; Berger et al. 2007; Eriksson et al. 2008), we additionally performed electrophysiological recordings in 4 hemispheres.

Figure 5A shows average multiunit activity (MUA, blue trace) recorded after imaging (black trace) in response to grating sequences. Along the entire stimulus sequence small bumps in the dye signal level coincided precisely with the generation of spiking activity, even at low levels during the early phase of the imaged response. Meaningful neuronal signals in VSD recordings can indeed be small in relation to overall activity (cf. Sharon and Grinvald 2002, for a signature of cross-inhibition suppression). Benucci et al. (2007) showed that small oscillations in the dye signal correlated with the frequency of counterphase oscillating gratings. In Onat, Nortmann et al. (2011), we showed for the first time that small bumps of activity represented exactly the retinotopic propagation of moving gratings. Very recently, using longer sequences (>10 s) of flashed gratings with different spatial phase, these formerly elusive retinotopic components in the imaging signal were shown in awake monkey (Omer et al. 2013).

For each electrode-recording site, we determined the preferred orientation tuning of MUA. Figure 5C depicts average spiking responses to moving gratings of different orientations. In this example, the neurons' preferred orientation was horizontal (red axis). Next, in Figure 5D, their time-resolved responses to a switch from superposition to either a horizontal or a vertical orientation are shown (see icons). Despite the fact that these neurons were tuned horizontally (Fig. 5C), responses for the switch to the nonpreferred vertical orientation were larger than for the switch to the preferred horizontal orientation (black trace shows the difference). In Figure 5E, spiking activity from all recorded units is summarized. The plot indicates that the recorded neurons responded stronger when their preferred stimulus was removed rather than when it was present, similarly as observed in our imaging signals (Fig. 4, bottom 3 rows).

Finally, we used a single-switch paradigm (Fig. 6; cf. Eriksson et al. 2010, 2012), also to rule-out that stimulus frames other than the switch-pair under analysis (Felsen et al. 2002; see Materials and Methods) may have significantly affected the observed response behavior using continuous sequences. The superposition of 2 orientations was presented for 30, 100, and 500 ms followed by a sudden switch to a single orientation (Fig. 6A). Whereas switches after 30-ms delay produced activity that was tuned to the orientation present after the switch, longer delay times yielded the opposite effect (Fig. 6B). Here again, population activity represented the orientation that was removed rather than the sustained orientation (Eriksson et al. 2010, 2012). This effect was stable and slightly increased (though not significantly) for 500 ms delays (Fig. 6C; in support of these findings, see Eriksson et al. (2012) for similar results with a stimulus duration of 250 ms). In Figure 6D, the time-courses of global population activity are shown. For each condition, both the initial responses to the superposition as well as the second response to a single orientation were represented by peaks of activity that were separated in time with longer switch delays (dark gray area). For 100 and 500 ms delays, activity to the superposition nearly adapted to baseline levels (occasionally we observed some further oscillations, see gray trace in the first plot and black trace in third plot), followed by a strong response after the stimulus changed to a single orientation. In contrast, for the 30-ms switch, activity did not adapt to baseline and responses to both stimuli were merged within a double-peak transient. To demonstrate that the orientation-selective part of the responses was directly traceable across the cortical activation patterns, we show time-averages of the most active pixels in each condition (Fig. 6E, see maps). Note the opposing cortical patterns in the maps depending on the switch direction and its delay times. The temporal evolution of the orientation specific patterns is shown as their correlation to a standard VH orientation map over time (black and gray traces in Fig. 6E; cf. Supplementary Fig. S1). Thus, by using only a single switch, these measurements confirmed our findings obtained with stimulus sequences: a short 30-ms period of stimulation resulted in the representation of current orientations (Fig. 6E left), whereas longer stimulus periods (100 and 500 ms) caused representation of the difference to past orientations (Fig. 6E, second and third graphs).

## Discussion

Our main result is that for sequences of natural scene contours presented with 10 Hz, activity no longer led to population tuning that represented actual stimulus contours. Instead, when compared with the preceding image, the cortical activity patterns characterized exactly the difference in orientations. Consequently, large amounts of incoming data were relatively suppressed, reminiscent of differencing methods (Fowler et al. 1995) used for video data compression in communication technology. For higher temporal frequency (33 Hz), activity was instead updated linearly, providing an ongoing representation of current stimulus orientation (Ringach et al. 1997; Benucci et al. 2009). Because we opposed 2 stimulation dynamics selected from a wide range of possible sequence frequencies, the exact time course of the transition between the different encoding schemes remains to be determined. Interestingly, in the same vein, using single squares of light Eriksson et al. (2008) showed in ferrets that V1 responses to rapidly presented stimulus switches (<83 ms) were dominated by stimulus onsets, whereas both VSD imaging and spiking responses to longer stimuli (>133 ms) carried additional prominent information about stimulus offset. We speculate that joint processing at various stimulus temporal frequencies is required to produce a coherent interpretation of a visual scene (Jonides et al. 1982; Rucci et al. 2007; Belitski et al. 2008; Rucci 2008; Nikolić et al. 2009; Jurjut et al. 2011; Onat, Nortmann et al. 2011; Eriksson et al. 2012), which might be implemented through differences in coherence between neuronal signals carrying different information at different frequencies, as recently shown for orientation tuning in monkey V1 (Gilad et al. 2012; Womelsdorf et al. 2012).

### Interaction Between Adaptive- and Off-Response Components

Because our results were dependent on stimulus frequency, we provide strong evidence that input history has a decisive effect on cortical orientation tuning. Several time-dependent changes in cortical orientation selectivity could be accounted for by mechanisms of adaptation. Specifically, stimulus-selective adaptation of visual cortical neurons has been shown to reduce responsiveness and causing a shift in tuning curves away from the adapting orientation on short timescales (Movshon and Lennie 1979; Müller et al. 1999; Felsen et al. 2002). In our experiments, neuronal adaptation mechanisms (Galaretta and Hestrin 1998; Varela et al. 1999; Sanchez-Vives et al. 2000) and immediate tuned suppression (Nelson 1991) may decrease activity for the sustained component. Adaptation to a single orientation was shown to also enhance the representation of orthogonal orientations (Dragoi et al. 2002), which in our case would furthermore boost the on-response for the newly added orientation. Thus, for switches within sequences from a single, (V) or (H), orientation to a superimposed (VH) stimulus, adaptation alone would explain the resulting dominant representation of the newly added orientation (instead of both of the current orientations).

These mechanisms, however, cannot entirely explain the prominent representation of the disappeared orientation after exposure to superimposed orientations. The latter provide no bias that could induce adaptation of a particular orientation before the switch. Thus, in addition, responses to the disappeared orientation must be involved (Bair et al. 2002; Sit et al. 2009; Eriksson et al. 2010, 2012). Signals following stimulus removal are commonly referred to as visual off-responses, which tend to increase with stimulus duration (Duysens et al. 1996), as similarly found in the somatosensory cortex (Kyrazi et al. 1994). A likely assumption is that off-responses result from post-inhibitory rebound due to sustained hyperpolarization arising from synaptic inhibition (Pernberg et al. 1998, but see Scholl et al. (2010) for auditory cortical neurons), supposedly mediated by tuned push-pull mechanisms within the cortical circuitry (Hirsch et al. 2003). Importantly, such tuned off-responses and adaptive contributions can be disentangled in our study. A change from superimposed orientations to a single orientation has 2 underlying constituents: A switch from one orientation to blank (off-component), and an overlaid continuous presentation of the orthogonal orientation (sustained, adaptive component). Figure 7 summarizes such a composition for a switch from superposition (VH) to a single orientation (V or H). Median fits for responses to each of the constituent stimuli are shown (blue/red; see Results), the gray curve outlines their average (i.e. divisive normalization, MacEvoy et al. 2009). The black curve shows the median fit for the responses that we obtained for the composite switch (Supplementary Tables S1–S3), indicating tuning to the orientation that was turned off.

Figure 7.

Disentangling components that produce change coding within 10-Hz sequences. Schematic of the response to a switch from superimposed orientations to a single orientation (bottom icon, black box) and the responses to the constituents of this switch (middle icons): Median of fitted population tuning curves (see Materials and Methods and Results) for off-component (blue), adaptive component (red), and composite switch (black; n = 48, medians across 12 experiments, both orientations, and both filter conditions of the natural images). Note the blue arrows, indicating deviations from the component average (gray, see main text for quantification). Overall, the measured response to the combined stimulus represents the difference between the past stimulus (i.e., superposition, purple icon at bottom) and the current orientation (horizontal, red icon at bottom). This difference is the orientation that was turned off (see peak of black curve at position of vertical orientation tuning, blue). The opposite result was found for 30-Hz sequences (see example in inset; nonfitted data of an experiment obtained with narrowly filtered natural images).

Figure 7.

Disentangling components that produce change coding within 10-Hz sequences. Schematic of the response to a switch from superimposed orientations to a single orientation (bottom icon, black box) and the responses to the constituents of this switch (middle icons): Median of fitted population tuning curves (see Materials and Methods and Results) for off-component (blue), adaptive component (red), and composite switch (black; n = 48, medians across 12 experiments, both orientations, and both filter conditions of the natural images). Note the blue arrows, indicating deviations from the component average (gray, see main text for quantification). Overall, the measured response to the combined stimulus represents the difference between the past stimulus (i.e., superposition, purple icon at bottom) and the current orientation (horizontal, red icon at bottom). This difference is the orientation that was turned off (see peak of black curve at position of vertical orientation tuning, blue). The opposite result was found for 30-Hz sequences (see example in inset; nonfitted data of an experiment obtained with narrowly filtered natural images).

Approximating the measured change response for this type of switch (VH to H and VH to V) by a weighted average of the 2 component responses (Busse et al. 2009; MacEvoy et al. 2009), resulted in a significantly higher average contribution of 60% from the off-component, compared with 40% from the adaptive component (see Supplementary Table S4 for details). Thus, for a 100-ms stimulus period the off-component is facilitated (see blue arrows in Fig. 7) and overrides the response component that undergoes adaptation. Accordingly, due to increased contribution of orientation-selective off-responses, the combination of adaptive- and off-response components results in a representation of the difference between the past and the present image. Taken together, our data suggest mutual interaction across population responses to changing and nonchanging, that is, sustained, features. As a consequence, after periods of longer stimulation, a stimulus-change within stimulus sequences triggers activity to report that a particular feature has disappeared in comparison to what is left.

### Coding of Stimulus Differences Between Past and Present

Representation of difference in the primary visual cortex may lead to attenuation of redundancies (Attneave 1954) over time and increased sensitivity to dissimilar structures, such as differently oriented borders of objects (Das and Gilbert 1997; Downar et al. 2000; Dragoi et al. 2002; Desbordes and Rucci 2007; Rucci et al. 2007; Rucci 2008; Beste et al. 2011). The proposed activity dynamics of adaptive- and off-response components might be viewed as short-term memory processes (Sperling 1960; DiLollo 1977; Coltheart 1980), which begin with stimulus onset (DiLollo 1977), trigger recurrent networks (McCormick et al. 2003), and may be coupled with feed-back from higher areas (Rockland and Pandya 1979; Roland et al. 2006; Golomb et al. 2010; Vetter et al. 2015) to allow prolonged influence of past activity (Coltheart 1980; Duysens et al. 1985; McCormick et al. 2003) on the processing of current input (Gould 1967; Jonides et al. 1982; Jancke 2000; Eagleman et al. 2004; Eriksson et al. 2008, 2010, 2012; Nikolić et al. 2009; Glasser et al. 2011).

During free viewing of natural scenes, intersaccadic durations (in human ∼250 ms on average, Kuang et al. 2012; in cat >2000 ms, Moeller et al. 2004) can be even larger than our long stimulus intervals (>100 ms). After the onset of each new fixation (i.e., at low temporal frequencies), difference representation might facilitate cortical encoding of luminance discontinuities and edges at spatial scales larger than covered by retinal ganglion cells and also by neighboring cortical cells with similar orientation tuning (Müller et al. 1999). Specifically, cortical difference representation might compensate the extensive luminance correlations (and thus, reduce redundancies) conveyed by transient activity of retinal ganglion cell populations immediately after a saccade (Desbordes and Rucci 2007).

High correlation among activity of ganglion cells immediately after a saccade signals long-range correlations in natural images (Kuang et al. 2012). A second regime brought in by microscopic eye movements, operating at higher temporal frequencies, is proposed to cause whitening of input while decorrelating retinal activity (Kuang et al. 2012). Hence, these retinal signals emphasize small spatial details in visual structures during fixation (Desbordes and Rucci 2007; Rucci et al. 2007; Rucci 2008; Kuang et al. 2012), even of contour orientations (Rucci and Desbordes, 2003). The here reported cortical representation of current orientations at higher frequencies (33 Hz) may reflect the transmission of the acquired information further downstream. In conclusion, the 2 temporal regimes of eye movements may allow complementary contributions (Snodderly et al. 2001) in a motion-based coarse-to-fine processing of visual information (Parker et al. 1992; Jancke 2000; Ahissar and Arieli 2001, 2012; Geisler et al. 2001; Henning et al. 2002; Desbordes and Rucci 2007; Rucci 2008; Meirovithz et al. 2012) and could be an efficient mechanism for perception of salient structures in the environment. Whether these regimes act linearly at the cortical level cannot ultimately be decided upon our data. Modeling at the retinal level suggests that the 2 regimes can however be well captured using linear approaches (Desbordes and Rucci 2007).

More generally, the time-dependent cortical coding of the difference to past events can be interpreted as an early cortical signature of mismatch signals between ongoing stimulation and abrupt stimulus changes, as first described in the auditory domain (Näätänen et al. 1978). Most recently, mismatch signals have been shown in the primary cortex of the mouse to be cooperatively influenced by motor-related input (Keller et al. 2012). We suggest that both adaptation (Jääskeläinen et al. 2004) and stimulus off-responses play in important role in generating mismatch signals.

Finally, responses to stimulus differences fit well conceptually with predictive coding principles, proposing that deviations from cortically generated predictions are propagated up the visual hierarchy as error signals (Friston 2005; Garrido et al. 2009). Given the prediction that contour orientations remain stable over prolonged periods of time, as during periods of fixation, error signals would correspond to representations of change, as measured here. Predictive coding principles were found as early as in the retina (Srinivasan et al. 1982; Hosoya et al. 2005) and have been used in recent modeling frameworks of cortical visual responses (Rao and Ballard 1999; Friston 2005; Spratling 2010, 2012; Boerlin and Dèneve 2011), also suggesting a combination of stimulus and error-like coding within single neurons (Eriksson et al. 2012). The exact timescales (Thorpe et al. 1996) at which different predictive states may evolve under natural viewing conditions need to be further explored and elaborated in computational models to account for the qualitatively different behaviors of cortical responses—from ongoing representation to representation of difference—that are reported here.

## Funding

Funding to pay the Open Access publication charges for this article was provided by the DFG (Deutsche Forschungsgemeinschaft), SFB-874 (TP A2 Eysel, Jancke) and the German-Israeli Project Cooperation (DIP, JA 945/3-1, SL 185/1-1).

## Notes

We thank BMBF (Bundesministerium für Bildung und Forschung), DFG (Deutsche Forschungsgemeinschaft), SFB-874 (TP A2 Eysel, Jancke), German-Israeli Project Cooperation (DIP, JA 945/3-1, SL 185/1-1), ERC-2010-AdG (269716 Multisense, König), and the Ministry for Science and Culture of Lower Saxony, Germany (Lichtenberg Scholarship) for financial support. We also thank Drs. Saskia Nagel and Andrea Benucci for their helpful comments on an early version of the manuscript and Drs. Eyal Seidemann and David Eriksson for the discussion of a later version. Conflict of Interest: None declared.

## References

H
Bruns
W
Taniguchi
H
Huang
ZJ
Scanziani
M
.
2012
.
A neural circuit for spatial summation in visual cortex
.
Nature
.
490
:
226
231
.
H
Scanziani
M
.
2010
.
Lateral competition for cortical space by layer-specific horizontal circuits
.
Nature
.
464
:
1155
1160
.
Ahissar
E
Arieli
A
.
2001
.
Figuring space by time
.
Neuron
.
32
:
185
201
.
Ahissar
E
Arieli
A
.
2012
.
Seeing via miniature eye movements: a dynamic hypothesis for vision
.
Front Comput Neurosci
.
6
:
89
.
Allison
T
.
1962
.
Recovery functions of somatosensory evoked responses in man
.
Electroencephalogr Clin Neurophysiol
.
14
:
331
343
.
Anderson
JS
Lampl
I
Gillespie
D
Ferster
D
.
2000
.
The contribution of noise to contrast invariance of orientation tuning in cat visual cortex
.
Science
.
290
:
1968
1971
.
Attneave
F
.
1954
.
Some informational aspects of visual perception
.
Psychol Rev
.
61
:
183
193
.
Bair
W
Cavanaugh
JR
Smith
MA
Movshon
JA
.
2002
.
The timing of response onset and offset in macaque visual neurons
.
J Neurosci
.
22
:
3189
3205
.
Belitski
A
Gretton
A
Magri
C
Murayama
Y
Montemurro
MA
Logothetis
NK
Panzeri
S
.
2008
.
Low-frequency local field potentials and spikes in primary visual cortex convey independent visual information
.
J Neurosci
.
28
:
5696
5709
.
Benucci
A
Frazor
RA
Carandini
M
.
2007
.
Standing waves and traveling waves distinguish two circuits in visual cortex
.
Neuron
.
55
:
103
117
.
Benucci
A
Ringach
DL
Carandini
M
.
2009
.
Coding of stimulus sequences by population responses in visual cortex
.
Nat Neurosci
.
12
:
1317
1324
.
Berger
T
Borgdorff
A
Crochet
S
Neubauer
FB
Lefort
S
Fauvet
B
Ferezou
I
Carleton
A
Luscher
H-R
Petersen
CCH
.
2007
.
Combined voltage and calcium epifluorescence imaging in vitro and in vivo reveals subthreshold and suprathreshold dynamics of mouse barrel cortex
.
J Neurophysiol
.
97
:
3751
3762
.
Beste
C
Wascher
E
Güntürkün
O
Dinse
HR
.
2011
.
Improvement and impairment of visually guided behavior through LTP- and LTD-like exposure-based visual learning
.
Curr Biol
.
21
:
876
882
.
Betsch
B
Einhäuser
W
Körding
KP
König
P
.
2004
.
The world from a cat's perspective—statistics of natural videos
.
Biol Cybern
.
90
:
41
50
.
Boerlin
M
Dèneve
S
.
2011
.
Spike-based population coding and working memory
.
PLoS Comp Biol
.
7
:
1
18
.
Busse
L
AR
Carandini
M
.
2009
.
Representation of concurrent stimuli by population activity in visual cortex
.
Neuron
.
64
:
931
942
.
Carandini
M
Demb
JB
Mante
V
Tolhurst
DJ
Dan
Y
Olshausen
BA
Gallant
JL
Rust
NC
.
2005
.
Do we know what the early visual system does?
J Neurosci
.
25
:
10577
10597
.
Chen
Y
Geisler
WS
Seidemann
E
.
2006
.
Optimal decoding of correlated neural population responses in the primate visual cortex
.
Nat Neurosci
.
9
:
1412
1420
.
Chen
Y
Palmer
CR
Seidemann
E
.
2012
.
The relationship between voltage-sensitive dye imaging signals and spiking activity of neural populations in primate V1
.
J Neurophysiol
.
107
:
3281
3295
.
Coltheart
M
.
1980
.
Iconic memory and visible persistence
.
Percept Psychophys
.
27
:
183
228
.
Das
A
Gilbert
CD
.
1997
.
Distortions of visuotopic map match orientation singularities in primary visual cortex
.
Nature
.
387
:
594
598
.
David
SV
Gallant
JL
.
2005
.
Predicting neuronal responses during natural vision
.
Network
.
16
:
239
260
.
David
SV
Vinje
WE
Gallant
JL
.
2004
.
Natural stimulus statistics alter the receptive ﬁeld structure of v1 neurons
.
J Neurosci
.
24
:
6991
7006
.
Desbordes
G
Rucci
M
.
2007
.
A model of the dynamics of retinal activity during natural visual fixation
.
Vis Neurosci
.
24
:
217
230
.
DiLollo
V
.
1977
.
Temporal characteristics of iconic memory
.
Nature
.
267
:
241
243
.
Dinse
HR
Jancke
D
.
2001
.
Time-variant processing in V1: from microscopic (single cell) to mesoscopic (population) levels
.
Trends Neurosci
.
24
:
203
205
.
Downar
J
Crawley
AP
Mikulis
DJ
Davis
KD
.
2000
.
A multimodal cortical network for the detection of changes in the sensory environment
.
Nat Neurosci
.
3
:
277
283
.
Dragoi
V
Sharma
J
Miller
EK
Sur
M
.
2002
.
Dynamics of neuronal sensitivity in visual cortex and local feature discrimination
.
Nat Neurosci
.
5
:
883
891
.
Duysens
J
Orban
GA
Cremieux
J
Maes
H
.
1985
.
Visual cortical correlates of visible persistence
.
Vision Res
.
25
:
171
178
.
Duysens
J
Schaafsma
SJ
Orban
GA
.
1996
.
Cortical off response tuning for stimulus duration
.
Vision Res
.
36
:
3243
3251
.
Eagleman
DM
Jacobson
JE
Sejnowski
TJ
.
2004
.
Perceived luminance depends on temporal context
.
Nature
.
428
:
854
856
.
Eckhorn
R
Krause
F
Nelson
JI
.
1993
.
The RF-cinematogram. A cross-correlation technique for mapping several visual receptive fields at once
.
Biol Cybern
.
69
:
37
55
.
Eriksson
D
Tompa
T
Roland
PE
.
2008
.
Non-linear population firing rates and voltage sensitive dye signals in visual areas 17 and 18 to short duration stimuli
.
PLoS One
.
3
:
e2673
.
Eriksson
D
Valentiniene
S
Papaioannou
S
.
2010
.
Relating information, encoding and adaptation: decoding the population firing rate in visual areas 17/18 in response to a stimulus transition
.
PLoS One
.
5
:
e10327
.
Eriksson
D
Wunderle
T
Schmidt
KE
.
2012
.
Visual cortex combines a stimulus and an error-like signal with a proportion that is dependent on time, space, and stimulus contrast
.
Front Syst Neurosci
.
6
:
26
.
Felsen
G
Dan
Y
.
2005
.
A natural approach to studying vision
.
Nat Neurosci
.
8
:
1643
1646
.
Felsen
G
Shen
YS
Yao
H
Spor
G
Li
C
Dan
Y
.
2002
.
Dynamic modification of cortical orientation tuning mediated by recurrent connections
.
Neuron
.
36
:
945
954
.
Felsen
G
Touryan
J
Han
F
Dan
Y
.
2005
.
Cortical sensitivity to visual features in natural scenes
.
PLoS Biol
.
3
:
e342
.
Finn
IM
Priebe
NJ
Ferster
D
.
2007
.
The emergence of contrastinvariant orientation tuning in simple cells of cat visual cortex
.
Neuron
.
54
:
137
152
.
Fournier
J
Monier
C
Pananceau
M
Frégnac
Y
.
2011
.
Adaptation of the simple or complex nature of V1 receptive fields to visual statistics
.
Nat Neurosci
.
14
:
1053
1060
.
Fowler
G
Korn
D
North
S
Rao
H
Vo
K
.
1995
.
Vdelta: differencing and compression
. In:
Krishnamurthy
B
, editor.
Practical reusable UNIX software
.
New York
:
Wiley
.
Friston
K
.
2005
.
A theory of cortical responses
.
Philos Trans R Soc Lond B Biol Sci
.
360
:
815
836
.
Galaretta
M
Hestrin
S
.
1998
.
Frequency dependent synaptic depression and the balance of excitation and inhibition in the neocortex
.
Nat Neurosci
.
1
:
587
594
.
Gallant
JL
Connor
CE
Van Essen
DC
.
1998
.
Neural activity in areas V1, V2 and V4 during free viewing of natural scenes compared to controlled viewing
.
Neuroreport
.
9
:
2153
2158
.
Garrido
MI
Kilner
J
Stephan
KE
Friston
KJ
.
2009
.
The mismatch negativity: a review of underlying mechanisms
.
Clin Neurophysiol
.
120
:
543
463
.
Geisler
WS
.
2008
.
Visual perception and the statistical properties of natural scenes
.
Annu Rev Psychol
.
59
:
167
92
.
Geisler
WS
Albrecht
DG
Crane
AM
Stern
L
.
2001
.
Motion direction signals in the primary visual cortex of cat and monkey
.
Vis Neurosci
.
18
:
501
516
.
A
Meirovithz
E
Leshem
A
Arieli
A
Slovin
H
.
2012
.
Collinear stimuli induce local and cross-areal coherence in the visual cortex of behaving monkeys
.
PLoS One
.
7
:
e49391
.
Gillespie
DC
Lampl
I
Anderson
JS
Ferster
D
.
2001
.
Dynamics of the orientation-tuned membrane potential response in cat primary visual cortex
.
Nat Neurosci
.
4
:
1014
1019
.
Glasser
DM
Tsui
JMG
Pack
CC
D
.
2011
.
Perceptual and neural consequences of rapid motion adaptation
.
.
108
:
E1080
1088
.
Golomb
JD
Nguyen-Phuc
AY
Mazer
JA
McCarthy
G
Chun
MM
.
2010
.
Attentional facilitation throughout human visual cortex lingers in retinotopic coordinates after eye movements
.
J Neurosci
.
30
:
10493
10506
.
Gould
JD
.
1967
.
Pattern recognition and eye-movement parameters
.
Perception
.
2
:
399
407
.
Grabska-Barwinska
A
Distler
C
Hoffman
K-P
Jancke
D
.
2009
.
Contrast independence of cardinal preference: stable oblique effect in orientation maps of ferret visual cortex
.
Eur J Neurosci
.
29
:
1258
1270
.
Graf
ABA
Kohn
A
Jazayeri
M
Movshon
JA
.
2011
.
Decoding the activity of neuronal populations in macaque primary visual cortex
.
Nat Neurosci
.
14
:
239
245
.
Grinvald
A
Hildesheim
R
.
2004
.
VSDI: a new era in functional imaging of cortical dynamics
.
Nat Rev Neurosci
.
5
:
874
885
.
Grinvald
A
Lieke
EE
Frostig
RD
Hildesheim
R
.
1994
.
Cortical point-spread function and long-range lateral interactions revealed by real-time optical imaging of macaque monkey primary visual cortex
.
J Neurosci
.
14
:
2545
2568
.
Haider
B
Krause
MR
Duque
A
Yu
Y
Touryan
J
Mazer
JA
McCormick
DA
.
2010
.
Synaptic and network mechanisms of sparse and reliable visual cortical activity during nonclassical receptive field stimulation
.
Neuron
.
65
:
107
121
.
Henning
MH
Kerscher
NJ
Funke
K
Wörgötter
F
.
2002
.
Stochastik resonance in visual cortical neurons: does the eye-tremor actually improve visual acuity?
Neurocomputing
.
44–46
:
115
120
.
Hickmott
PW
.
2010
.
Synapses of horizontal connections in adult rat somatosensory cortex have different properties depending on the source of their axons
.
Cereb Cortex
.
20
:
591
601
.
Hirsch
JA
Martinez
LM
Pillai
C
Alonso
JM
Wang
Q
Sommer
FT
.
2003
.
Functionally distinct inhibitory neurons at the first stage of visual cortical processing
.
Nat Neurosci
.
6
:
1300
1308
.
Hosoya
T
Baccus
SA
Meister
M
.
2005
.
Dynamic predictive coding by the retina
.
Nature
.
436
:
71
77
.
Hyvärinen
A
Hurri
J
Hoyer
PO
.
2009
.
Natural Image Statistics. A probabilistic approach to early computational vision
.
Springer
.
Jääskeläinen
IP
Ahveninen
J
Bonmassar
G
Dale
AM
Ilmoniemi
RJ
Levänen
S
Lin
F-H
May
P
Melcher
J
Stufflebeam
S
Tiitinen
H
Belliveau
JW
.
2004
.
Human posterior auditory cortex gates novel sounds to consciousness
.
.
101
:
6809
6814
.
Jancke
D
.
2000
.
Orientation formed by a spot's trajectory: a two-dimensional population approach in primary visual cortex
.
J Neurosci
.
20
:
1
6
.
Jancke
D
Chavane
F
Naaman
S
Grinvald
A
.
2004
.
Imaging cortical correlates of illusion in early visual cortex
.
Nature
.
428
:
423
426
.
Jancke
D
Erlhagen
W
Dinse
HR
Akhavan
M
Giese
M
Steinhage
A
Schöner
G
.
1999
.
Parametric representation of retinal location: neural population dynamics and interaction in cat visual cortex
.
J Neurosci
.
19
:
9016
9028
.
Jonides
J
Irwin
DE
Yantis
S
.
1982
.
Integrating visual information from successive fixations
.
Science
.
215
:
192
194
.
Jurjut
OF
Nikolic
D
Singer
W
Yu
S
Havenith
MN
Muresan
RC
.
2011
.
Timescales of multineuronal activity patterns reflect temporal structure of visual stimuli
.
PLoS ONE
.
6
:
1
15
.
Kayser
C
Einhäuser
W
König
P
.
2004
.
Temporal correlations of orientations in natural scenes
.
Neurocomputing
.
52–54
:
117
123
.
Keller
GB
Bonhoeffer
T
Hübener
M
.
2012
.
Sensorimotor mismatch signals in primary visual cortex of the behaving mouse
.
Neuron
.
74
:
809
815
.
Kuang
X
Poletti
M
Victor
JD
Rucci
M
.
2012
.
Temporal encoding of spatial information during active visual fixation
.
Curr Biol
.
22
:
510
514
.
Kyrazi
HT
Carvell
GE
Simons
DJ
.
1994
.
Off response transformations in the whisker/barrel system
.
J Neurophysiol
.
72
:
392
401
.
Lee
C
Rohrer
WH
Sparks
DL
.
1988
.
Population coding of saccadic eye movements by neurons in the superior colliculus
.
Nature
.
332
:
357
360
.
Lewis
CM
Lazar
AE
.
2013
.
Orienting towards ensembles: from single cells to neural populations
.
J Neurosci
.
33
:
2
3
.
MacEvoy
SP
Tucker
TR
Fitzpatrick
D
.
2009
.
A precise form of divisive suppression supports population coding in the primary visual cortex
.
Nat Neurosci
.
12
:
637
645
.
Markram
H
Wang
Y
Tsodyks
M
.
1998
.
Differential signaling via the same axon of neocortical pyramidal neurons
.
.
95
:
5323
5328
.
McCormick
DA
Shu
Y
Hasenstaub
A
Sanchez-Vives
M
M
Bal
T
.
2003
.
Persistent cortical activity: mechanisms of generation and effects on neuronal excitability
.
Cereb Cortex
.
13
:
1219
1231
.
Meirovithz
E
Ayzenshtat
I
Werner-Reiss
U
Shamir
I
Slovin
H
.
2012
.
Spatiotemporal effects of microsaccades on population activity in the visual cortex of monkeys during fixation
.
Cereb Cortex
.
22
:
294
307
.
Moeller
GU
Kayser
C
Knecht
F
König
P
.
2004
.
Interactions between eye movement systems in cats and humans
.
Exp Brain Res
.
157
:
215
224
.
Movshon
JA
Lennie
P
.
1979
.
Pattern-selective adaptation in visual cortical neurones
.
Nature
.
278
:
850
852
.
Müller
JR
Metha
AB
Krauskopf
J
Lennie
P
.
1999
.
Rapid adaptation in visual cortex to the structure of images
.
Science
.
285
:
1405
1408
.
Näätänen
R
Gaillard
AWK
Mäntysalo
S
.
1978
.
Early selective-attention effect on evoked potential reinterpreted
.
Acta Psychol
.
42
:
313
329
.
Nelson
SB
.
1991
.
Temporal interactions in the cat visual system. I. Orientation-selective suppression in visual cortex
.
J Neurosci
.
11
:
344
356
.
Niell
CM
Stryker
MP
.
2010
.
Modulation of visual responses by behavioral state in mouse visual cortex
.
Neuron
.
65
:
472
479
.
Nikolić
D
Häusler
S
Singer
W
Maass
W
.
2009
.
Distributed fading memory for stimulus properties in the primary visual cortex
.
PLoS Biol
.
7
:
e1000260
.
Nortmann
N
Rekauzke
S
Onat
S
König
P
Jancke
D
.
2011
.
Representation of difference: Primary visual cortex encodes the relative change in orientation content within rapid stimulus sequences
.
Soc Neurosci Abstr
.
41
:
799.11
.
Olsen
SR
Bortone
DS
H
Scanziani
M
.
2012
.
Gain control by layer six in cortical circuits of vision
.
Nature
.
483
:
47
52
.
Olshausen
BA
Field
DJ
.
2005
.
How close are we to understanding V1?
Neural Comput
.
17
:
1665
1699
.
Omer
DB
Hildesheim
R
Grinvald
A
.
2013
.
Temporally-structured acquisition of multidimensional optical imaging facilitates visualization of elusive cortical representations in the behaving monkey
.
Neuroimage
.
82
:
237
251
.
Onat
S
König
P
Jancke
D
.
2011
.
Natural scene evoked population dynamics across cat primary visual cortex captured with voltage-sensitive dye imaging
.
Cereb Cortex
.
21
:
2542
2554
.
Onat
S
Nortmann
N
Rekauzke
S
König
P
Jancke
D
.
2011
.
Independent encoding of grating motion across stationary feature maps in primary visual cortex visualized with voltage-sensitive dye imaging
.
Neuroimage
.
55
:
1763
1770
.
Parker
DM
Lishman
JR
Hughes
J
.
1992
.
Temporal integration of spatially filtered visual images
.
Perception
.
21
:
147
160
.
Pernberg
J
Jirmann
K
Eysel
UT
.
1998
.
Structure and dynamics of receptive fields in the visual cortex of the cat (area 18) and the influence of GABAergic inhibition
.
Eur J Neurosci
.
10
:
3596
3606
.
Petersen
CC
Grinvald
A
Sakmann
B
.
2003
.
Spatiotemporal dynamics of sensory responses in layer 2/3 of rat barrel cortex measured in vivo by voltage-sensitive dye imaging combined with whole-cell voltage recordings and neuron reconstructions
.
J Neurosci
.
23
:
1298
1309
.
Pouget
A
Dayan
P
Zemel
RS
.
2003
.
Inference and computation with population codes
.
Annu Rev Neurosci
.
26
:
381
410
.
Rao
RPN
Ballard
DH
.
1999
.
Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive field effects
.
Nat Neurosci
.
2
:
79
87
.
Ringach
DL
Hawken
MJ
Shapley
R
.
1997
.
Dynamics of orientation tuning in macaque primary visual cortex
.
Nature
.
387
:
281
284
.
Rockland
KS
Pandya
DN
.
1979
.
Laminar origins and terminations of cortical connections of the occipital lobe in the rhesus monkey
.
Brain Res
.
179
:
3
20
.
Roland
PE
Hanazawa
A
Undeman
C
Eriksson
D
Tompa
T
Nakamura
H
Valentiniene
S
Ahmed
B
.
2006
.
Cortical feedback depolarization waves: a mechanism of top-down influence on early visual areas
.
.
103
:
12586
12591
.
Rucci
M
.
2008
.
Fixational eye movements, natural image statistics, and fine spatial vision
.
Network
.
19
:
253
285
.
Rucci
M
Desbordes
G
.
2003
.
Contributions of fixational eye movements to the discrimination of briefly presented stimuli
.
J Vis
.
3
:
852
864
.
Rucci
M
Iovin
R
Poletti
M
Santini
F
.
2007
.
Miniature eye movements enhance fine spatial detail
.
Nature
.
447
:
852
855
.
Sanchez-Vives
MV
Nowak
LG
McCormick
DA
.
2000
.
Cellular mechanisms of long-lasting adaptation in visual cortical neurons
.
J Neurosci
.
20
:
4286
4299
.
Scholl
B
Gao
X
Wehr
M
.
2010
.
Nonoverlapping sets of synapses drive on responses and off responses in auditory cortex
.
Neuron
.
65
:
412
421
.
Sharon
D
Grinvald
A
.
2002
.
Dynamics and constancy in cortical spatiotemporal patterns of orientation processing
.
Science
.
295
:
512
.
Simoncelli
EP
Farid
H
.
1996
.
Steerable wedge filters for local orientation analysis
.
IEEE Trans Image Proc
.
5
:
1377
1382
.
Simoncelli
EP
Olshausen
BA
.
2001
.
Natural image statistics and neural representation
.
Annu Rev Neurosci
.
24
:
1193
1216
.
Sit
YF
Chen
Y
Geisler
WS
Mikkulainen
R
Seidemann
E
.
2009
.
Complex dynamics of V1 population responses explained by a simple gain-control model
.
Neuron
.
24
:
943
956
.
Smyth
D
Willmore
B
Baker
GE
Thompson
ID
Tolhurst
DJ
.
2003
.
The receptive-ﬁeld organization of simple cells in primary visual cortex of ferrets under natural scene stimulation
.
J Neurosci
.
23
:
4746
4759
.
Snodderly
DM
Kagan
I
Gur
M
.
2001
.
Selective activation of visual cortex neurons by fixational eye movements: implications for neural coding
.
Vis Neurosci
.
18
:
259
277
.
Sperling
G
.
1960
.
The information available in brief visual presentations
.
Psychol Monogr
.
74
:
1
29
.
Spratling
MW
.
2010
.
Predictive coding as a model of response properties in cortical area V1
.
J Neurosci
.
30
:
3531
3543
.
Spratling
MW
.
2012
.
Predicitve coding as a model of the V1 saliency map hypothesis
.
Neural Netw
.
26
:
7
28
.
Srinivasan
MV
Laughlin
SB
Dubs
A
.
1982
.
Predictive coding: a fresh view of inhibition in the retina
.
Proc R Soc Lond B
.
216
:
427
459
.
Thorpe
S
Fize
D
Marlot
C
.
1996
.
Speed of processing in the human visual system
.
Nature
.
381
:
520
522
.
Tsodyks
M
Kenet
T
Grinvald
A
Arieli
A
.
1999
.
Linking spontaneous activity of single cortical neurons and the underlying functional architecture
.
Science
.
286
:
1943
1946
.
Varela
JA
Song
S
Turrigiano
GG
Nelson
SB
.
1999
.
Differential depression at excitatory and inhibitory synapses in visual cortex
.
J Neurosci
.
19
:
4293
4304
.
Vetter
P
Grosbras
M-H
Muckli
L
.
2015
.
TMS over V5 disrupts motion prediction
.
Cereb Cortex
.
25
:
1052
1059
.
Vogels
R
.
1990
.
Population coding of stimulus orientation by striate cortical cells
.
Biol Cybern
.
64
:
24
31
.
Wehr
M
AM
.
2005
.
Synaptic mechanisms of forward suppression in rat auditory cortex
.
Neuron
.
47
:
437
445
.
Womelsdorf
T
Lima
B
Vinck
M
Oostenveld
R
Singer
W
Neuenschwander
S
Fries
P
.
2012
.
Orientation selectivity and noise correlation in awake monkey area V1 are modulated by the gamma cycle
.
.
109
:
4302
4307
.
Zucker
RS
Regehr
WG
.
2002
.
Short-term synaptic plasticity
.
Annu Rev Physiol
.
64
:
355
405
.

## Author notes

Peter König and Dirk Jancke shared senior authorship.