Occipital MEG Activity in the Early Time Range (<300 ms) Predicts Graded Changes in Perceptual Consciousness

Abstract

Two electrophysiological components have been extensively investigated as candidate neural correlates of perceptual consciousness: An early, occipitally realized component occurring 130–320 ms after stimulus onset and a late, frontally realized component occurring 320–510 ms after stimulus onset. Recent studies have suggested that the late component may not be uniquely related to perceptual consciousness, but also to sensory expectations, task associations, and selective attention. We conducted a magnetoencephalographic study; using multivariate analysis, we compared classification accuracies when decoding perceptual consciousness from the 2 components using sources from occipital and frontal lobes. We found that occipital sources during the early time range were significantly more accurate in decoding perceptual consciousness than frontal sources during both the early and late time ranges. These results are the first of its kind where the predictive values of the 2 components are quantitatively compared, and they provide further evidence for the primary importance of occipital sources in realizing perceptual consciousness. The results have important consequences for current theories of perceptual consciousness, especially theories emphasizing the role of frontal sources.

classification, consciousness, magnetoencephalography, neural correlate of consciousness, perception

Introduction

Ever since Baars (1988) argued for the possibility of investigating perceptual consciousness using contrastive analyses, perceptual consciousness has been investigated with a number of methods. For example, functional magnetic resonance imaging (fMRI) studies have identified activity in frontal (Dehaene et al. 2001; Lau and Passingham 2006) and occipital (Ffytche et al. 1998) areas as candidate neural correlate(s) of consciousness (NCC(s)). Electroencephalographic studies have shown that there are at least 3 event-related potentials (ERPs) of interest for perceptual consciousness, about 100, 200, and 400 ms after the onset of a stimulus with source reconstructions localizing them in the occipital lobes, occipito-temporal lobes, and fronto-parietal lobes, respectively (Sergent et al. 2005; Fahrenfort et al. 2007; Veser et al. 2008), which is in agreement with the above-mentioned fMRI findings. Magnetoencephalographic (MEG) studies have reported event-related fields (ERFs) corresponding to these ERPs (Vanni et al. 1996; Liu et al. 2012; Sandberg et al. 2013). In this study, we examine which spatial and temporal components of the MEG are the most predictive of graded levels of perceptual consciousness in a visual identification task. Before describing our study in greater detail, some theoretical distinctions must be made and some theories must be recounted.

The main distinctions relate to the definition of consciousness. Most importantly, it should be noted that we investigate conscious contents and not conscious states. Examples of differences in conscious states are differences between being awake, being asleep, being in a coma, etc. (Laureys et al. 2004). An example of a difference in conscious content is­ whether or not a briefly flashed stimulus was perceived. A further distinction can be made between becoming conscious of a stimulus and remaining conscious of that stimulus. Most commonly, stimuli are presented briefly or obscured in some manner in consciousness experiments, and the activity that predicts whether or not a stimulus was perceived consciously is examined. It is this “becoming conscious” of a stimulus that we examine in this study. Alternatively, one might examine the sustained activity related to consciously perceiving a stimulus for as long as it is presented. We do not examine this aspect as it would require a very different experimental paradigm.

A final distinction can be made between phenomenal consciousness, the experience of perceiving something, and access consciousness, the availability of these perceptions for action preparation, verbal report, etc. (Block 2005). Although conceptually important, this distinction is nevertheless very difficult to make experimentally as most studies, including the present, rely on participants' reports for separating trials into what degree stimuli were consciously perceived. For this reason, we do not interpret our findings in terms of access and phenomenal consciousness, and we do not make any claims as to whether our results reflect one or the other.

The experimental findings mentioned in the first paragraph are reflected in a number of theories about which specific activities correlate directly with perceptual consciousness. Crucially for the present study, these theories differ as to whether “early” differences in occipital activity around 130–300 ms (the N1 and N2 components) or “late” differences in frontal activity after 300–600 ms (the P3a) constitute the proper NCC (Aru, Bachmann, et al. 2012). There are also studies suggesting that changes in the P1 component (100 ms; Pins and Ffytche 2003; Veser et al. 2008) correlate with differences in perceptual consciousness, but these differences are reported much less consistently than for the N1, N2, and P3a components [for a review, see Koivisto and Revonsuo (2010)]. Evidence has also been reported for the P1 component correlating with differences in attention (Aru, Axmacher, et al. 2012) rather than perceptual consciousness per se. For these reasons, the P1 is generally not considered a main candidate for the correlate of perceptual consciousness, and the components of interest in the present study are thus the N1/N2 and the P3a.

In the Global Workspace Theory (GWT) of Baars (2005), the variation of it by Dehaene et al. (2006) and Dehaene (2014), and in the Higher-Order Thought (HOT) theory of consciousness (Lau and Rosenthal 2011), differences in late (P3a) frontal activity correlate with differences in perceptual consciousness. The frontal activity is theorized to reflect global broadcasting of perceptually integrated stimuli, and it is this broadcasting that makes it conscious according to the GWT. Sergent et al. (2005) argued that frontal components after 300 ms correlate with perceptual consciousness in a bimodal manner: Absent when participants are not conscious of a stimulus, and present when participants are conscious of a stimulus.

The P3a component has been observed to be bimodal in several experiments (Del Cul et al. 2007; Koivisto and Revonsuo 2010), including infant studies (Kouider et al. 2013), and to be absent in patients with prefrontal damage (Del Cul et al. 2009). According to GWT proponents, the bimodality of the proposed NCC suggests that perceptual consciousness is dichotomous: You either see something or you do not. We thus have one set of theories and studies, arguing that consciousness is dichotomous and related to the late P3a component and to activity (mainly) in frontal cortical areas.

In contrast, in the recurrent processing theory of Lamme (2006) and the research on the Visual Awareness Negativity (VAN) of Koivisto and Revonsuo (2010), differences in early occipital activity (N1/N2) are found to be the best correlate of differences in perceptual consciousness. Recurrent processing between higher and lower regions of the occipital lobes is theorized to be sufficient for perceptual consciousness in Lamme's theory (2006).

Furthermore, several behavioral studies have indicated that perceptual consciousness is better understood as graded with levels between conscious and unconscious (Overgaard et al. 2006, 2010; Sandberg et al. 2010; Nieuwenhuis and de Kleijn 2011; Wierzchoń et al. 2012).

An important consequence of the proposal that perceptual consciousness is graded is that more than one NCC may exist. Hypothetically, each grade of experience may be associated with activity in a different cortical area, or it may depend on different levels of activity in a single area. For instance, the Perceptual Awareness Scale (PAS; Ramsøy and Overgaard 2004), used in numerous studies (Ruzzoli et al. 2010; Melloni et al. 2011; Ludwig et al. 2013; Faivre and Koch 2014), has 4 qualitatively different ratings. The differences between the neighboring ratings can be summarized as follows: First and second ratings: the presence of subjective experience as such; second and third ratings: the presence of (unclear) content; third and fourth ratings: the presence of perceptually clear and unambiguous content. It is thus possible that we should not just be looking for one all-or-none component predicting perceptual consciousness, but instead several components or a graded modulation of a single component. The occipito-temporal N2 has been observed to vary in a graded manner (Sergent et al. 2005), and it may thus be argued that this component is in fact a more likely correlate of perceptual consciousness.

Recent electrophysiological studies also cast doubt on whether late frontal components specifically correlate with perceptual consciousness. Melloni et al. (2011) found that sensory expectations influence the amplitude of the late frontal component, but not that of the early occipital component. Pitts et al. (2012) found that the late frontal component disappeared for conscious percepts that were not task-associated. Even with no task association, the early occipital component still correlated with perceptual consciousness. Koivisto and Revonsuo (2007) found that the late frontal component interacted with selective attention, whereas the early occipital component did not. Sandberg et al. (2013) were able to decode which of 2 rivaling percepts the participant was conscious of, using only activity at occipital and temporal sources around 130–320 ms, and activity from these sources was more predictive than that at frontal sources at any time point.

Taken together, one set of theories and experimental findings argue in favor of a dichotomous, late frontal component being the main correlate of perceptual consciousness, and another set of theories and experimental findings argue in favor of graded, earlier occipital or occipito-temporal activity. We have previously suggested that one way of providing evidence relevant to this debate is to examine the predictive power of the components in question (Sandberg et al. 2014). Specifically, we have argued that the correlate of perceptual consciousness should be at least as predictive of reports on a perceptual consciousness scale as any process that is only a prerequisite of perceptual consciousness (which may sometimes lead to perceptual consciousness and sometimes not) or a potential consequence of consciousness (which may or may not occur consistently every time perceptual consciousness is present). For this reason, we conducted an MEG study of masked visual identification examining which spatial and temporal components of the MEG signal were the most predictive of perceptual consciousness. Specifically, we examined whether participants reported graded levels of perceptual consciousness, and whether these levels could be decoded from the MEG signal using multivariate classification algorithms trained and tested on data from a wide set of cortical sources primarily at the time windows of the P3a and the VAN (N1/N2).

Materials and Methods

Participants

Nineteen right-handed male participants with a normal or corrected-to-normal vision gave written informed consent to participate. Their age was 26.6 years on average (range: 21–37 years, SD: 4.4 years). The local ethics committee, De Videnskabsetiske Komitéer for Region Midtjylland, provided written confirmation that no ethical approval was required for the study according to the Danish law, specifically Komitéloven §7 and §8.1.

One participant misunderstood instructions and did not respond on the identification task when he had no experience of the target. With 2 other participants, there were problems with their Head Position Indicator (HPI) coils and head positions could thus not be monitored. Two participants reported claustrophobic reactions and did not complete the experiment. One participant did not use the “Almost Clear Experience” (ACE) rating at all (see definitions below) and could thus not be included in the analyses comparing the PAS ratings. Submission of the data to the MaxFilter (see below) of another participant returned an error that could not be resolved. Finally, one participant's contrasts were uniformly distributed among all possible contrasts, indicating that the staircase did not work for him. In summary, data from 8 participants were thus excluded before analyses.

Stimuli and Procedure

A visual masking paradigm was used (Fig. 1A). Participants were seated 60 cm from the screen. A Panasonic PT-D10000E projector was used with a resolution of 1200 × 1024 pixels and a frequency of 60 Hz. A fixation cross was presented for 500, 1000, or 1500 ms, followed by 1 of 2 target rectangles, presented for 33.3 ms (2 frames), which were rotated 45° relative to each other (size: 1.34 × 1.02° of visual angle; Fig. 1A). Presentation of the target was followed by a static random noise mask that was presented for 2000 ms. During these 2000 ms, participants were to identify the presented figure by a button press on a response box (ID box). Following the identification of the target, participants were to rate their conscious experience on the PAS using 1 of 4 categories (Table 1). No Experience (NE): Nothing at all was seen; Weak Glimpse (WG): A feeling of having seen something, which cannot be described further; Almost Clear Experience (ACE): An ambiguous experience of the stimulus, some aspects are experienced more clearly than others; Clear Experience (CE): An unambiguous and clear experience. Pressing the upper button of a second response box (PAS box) enabled participants to cycle through the 4 categories. The lower button was used to confirm the selection of the PAS category that the cursor was situated on. At the beginning of PAS selection, the cursor was not present on the screen. By the first press of the upper button, it would appear on the NE category. In the beginning of the experiment, the ID box was the box in the participant's right hand and the PAS box the one in the participant's left hand. In every 36 trials, the functions of the 2 boxes swapped such that the ID box became the one in the left hand and the PAS box the one in the right hand or vice versa.

Table 1

The Perceptual Awareness Scale (PAS)

Label	Description [from Ramsøy and Overgaard (2004)]
(1) No Experience (NE)	No impression of the stimulus. All answers are seen as mere guesses.
(2) Weak Glimpse (WG)	A feeling that something has been shown. Not characterized by any content, and this cannot be specified any further.
(3) Almost Clear Experience (ACE)	Ambiguous experience of the stimulus. Some stimulus aspects are experienced more vividly than others. A feeling of almost being certain about one's answer.
(4) Clear Experience (CE)	Non-ambiguous experience of the stimulus. No doubt in one's answer.

Note: Scale steps and their descriptions.

Figure 1.

Paradigm, stimuli, and behavioral results. (A) Paradigm and stimuli: First, a fixation cross was presented for either 500, 1000, or 1500 ms. Following that, the target (1 of 2 figures, rectangle or rotated rectangle) was presented for 33.3 ms. This was immediately followed by a static noise mask presented for 2000 ms. During these 2000 ms, participants reported the identity of the target by a button press with one hand. Finally, they indicated the clarity of their experience using the PAS (Table 1). A contrast staircase, a modified 2-up-1-down, was used throughout the experiment. In the lower left of A are the 2 target stimuli used throughout the experiment. (B) Definition of lobes: Lobes overlaid on inflated cortex (left hemisphere) of the fsaverage map. A lateral (left) and a medial (right) view is shown with the borders between the lobes highlighted (C) Behavioral results: Only responses that were within the time limit are plotted. Proportion correct and response times are shown for the identification task, which have been categorized according to the subsequently reported PAS rating. Mean proportion correct (left) for each PAS rating. Error bars are 95% confidence intervals. The punctured lines represent chance and ceiling. Response times (right) for each PAS rating with 95% confidence intervals.

Before participants were tested in the magnetically shielded room, they completed a short practice session of 32 trials of varying contrast. The purpose of the session was to accustom participants to the experimental procedure and to instruct them in how to use the PAS categories. During the practice session, participants received feedback about the correctness of their identifications. Participants were instructed to use “No Experience” (NE) when they had no conscious experience at all, “Weak Glimpse” (WG) when they had a conscious experience of a target appearing on the screen, but with no experience of its features, “Almost Clear Experience” (ACE) when they had a conscious experience of a target and some of its features, and “Clear Experience” (CE) when they had a conscious experience of a target and all of its features. Extra care was given in instructing participants in the difference between ACE and CE, because their names are semantically very close to one another. Participants were given as an example of an ACE contra a CE the clearer experiencing of the 2 lines that make up the upper left angle of a rectangle versus the equally clear experiencing of all 4 lines of a rectangle. All participants except one had used all categories after the practice session and reported that the 4 categories were experientially distinguishable to them. Finally, they were instructed that they were to describe the clarity of their experiences and not how confident they were in having made the correct identification.

In the magnetically shielded room, participants went through 1 practice block and 11 experimental blocks, each consisting of 72 trials. Participants received feedback on the identification task only during the practice trials. Between blocks, participants were encouraged to rest a little and move their limbs (not their heads). Furthermore, participants were notified by a message every 36 trials that the functions of the response boxes changed, in the manner explained earlier.

Because our planned statistical contrasts included the 4 levels of PAS ratings, a sufficient amount of responses for each PAS rating was necessary, and a contrast staircase was therefore used. All stimuli were white/gray on a black (RGB value of 0, 0, 0) background. The staircase had 26 contrast levels with the clearest level equivalent to a contrast of 77% (with 100% equivalent to an RGB value of 255, 255, 255) and the dimmest level equivalent to a contrast of 2% (with 0% equivalent to an RGB value of 0, 0, 0). All steps were of 3%. During the practice block and the first experimental block, 2 successive correct answers on the identification task resulted in going 2 levels down the staircase (making the stimulus dimmer), whereas 1 wrong answer resulted in going 1 level up the staircase (making the stimulus brighter). The contrast level was 14% or lower for all participants at the end of the practice trials. For each experimental block after the first, that is, blocks 2–11, the staircase adapted based on which PAS rating the participant had responded the least with throughout the experiment so far. If NE had been used the least number of times during a block, 3 levels were subtracted after 2 successive correct answers, and only 1 added for a wrong answer. If WG had been used the least number of times, 2 levels were subtracted and 1 added. For ACE, 1 level was subtracted and 2 added. Finally, for CE, 1 level was subtracted and 3 added.

Distributed pseudorandomly across the experiment, approximately 72 “catch trials” containing no stimulus were presented.

Magnetoencephalography

MEG data were recorded in a magnetically shielded room with an Elekta Neuromag Triux system with 102 magnetometers and 204 orthogonal planar gradiometers with a recording frequency of 1000 Hz. Offline, a Maxwell Filter was used to apply spatio-temporal Signal Space Separation (tSSS), which separates the brain signal from the external disturbances outside the sensor array, leaving only the brain signal. After applying tSSS, movement compensation was applied based on continuous HPI measurements with a step size of 30 ms. tSSS and movement compensation were both performed using the MaxFilter, version 2.2 (Elekta). Five HPI coils were used, one behind each ear, one on the left and right temples, respectively, and the final one on the forehead. Head shape was digitized using a Polhemus Fasttrack Digitizer (Colchester, Vermont, USA). The head shape of the participant was later used to create the forward model for each participant.

Data were analyzed using MNE-python (Gramfort et al. 2013). The data were bandpass-filtered (0.5–15 Hz, Butterworth) and epoched into epochs of −200 to –600 ms around the target and downsampled to 250 Hz. The upper boundary of 15 Hz was selected as both components of interest, the VAN and the P3a, have a frequency of approximately 7 Hz. Therefore, a filter removing frequencies above this will generate the greatest statistical power. Independent component analysis (Hyvärinen and Oja 2000) was used to remove eye blinks and eye movements by removing the component that correlated most with the horizontal and vertical electrooculograms.

Source Reconstruction

Source reconstruction was done using the minimum norm estimate (MNE) algorithm (Hämäläinen et al. 1993). MNE assumes minimal prior information, only that the source currents are spatially restricted. We aimed to model 8196 sources for each participant based on participant-specific cortical reconstructions and volumetric segmentations. The cortical reconstructions were modeled using FreeSurfer (http://surfer.nmr.mgh.harvard.edu/ [date last accessed; 11 May 2015]; Dale et al. 1999).

Dynamic statistical parametric mapping was used to overcome the superficial bias of MNE (Dale et al. 2000). We ran a separate source reconstruction for each of the 3 PAS comparisons. For each comparison, we used the shared maximum number of trials for each PAS rating. Because differences in stimuli contrasts can induce differences in a given NCC (Fisch et al. 2009), we only used trials with the same contrast level in our tests. Furthermore, we ensured that there was an equal amount of left-handed and right-handed responses to prevent the classifier from using motor activity associated with a perceptual state to classify trials (Sandberg et al. 2013).

Owing to individual anatomical differences, participants had different numbers of reconstructed sources in each lobe. The average amount of modeled sources in each participant was 8194 (min. = 8175, max. = 8196). For the frontal lobe, it was 2490 (min. = 2283, max. 2654); for the occipital lobe, it was 698 (min. = 585, max. = 821); for the parietal lobe, it was 2254 (min. = 2165, max. 2298); and for the temporal lobe, it was 1426 (min. = 1289, max. = 1539). The lobes were defined using the Desikan–Killiany Atlas (Desikan et al. 2006). See Figure 1B for the lobes displayed on the “fsaverage” template from FreeSurfer.

Multivariate Analyses (Within Participants)

We used a logistic regression classifier (Bishop 2006). We conducted 5 different runs with the classifier per PAS comparison (NE versus WG, WG versus ACE, and ACE versus CE), one with all sources included, and one with occipital, temporal, parietal, and frontal sources separately. The analyses were run within participants. We used stratified 5-fold cross-validation to ensure an equal amount of trials with left- and right-handed responses in each training set. Only correct trials were included, such that the influence of performance on decoding accuracy was minimized.

Thus, classification accuracy was calculated for each source group for each participant tested. The theoretical chance level was 50% since there was an equal number of trials for each comparison. We used L1-regularization, sparse weighting.

This classification analysis was run for an early range (VAN: 132–320 ms) and for a late range (P3a: 324–512 ms). These ranges were of equal duration (i.e., the number of temporal features was controlled) to ensure that they could be compared meaningfully (Sandberg et al. 2014). We thus specifically tested whether the early range or the late range was the more informative by comparing their classification accuracies to one another. For the 3 PAS comparisons, this resulted in the following median number of trials per participant: NE versus WG = 24, WG versus ACE = 34, and ACE versus CE = 30. It should be noted that the number of trials used in the analysis for a given participant did not predict classification accuracy (see control analysis reported in Fig. 5B).

Group-Level Analysis

The main objective of this analysis was to compare frontal and occipital lobes as to which was the better for classifying perceptual consciousness. We investigated this in the VAN time range and in the P3a time range since both these have been reported as correlating with perceptual consciousness. This was done for each of the 3 neighboring PAS comparisons, NE–WG, WG–ACE, and ACE–CE. More exploratively, the temporal and parietal lobes and the full brain were also investigated.

We fitted models with accuracy of the classifier as the dependent variable. Participants were modeled as having a unique intercept, that is, a random effect. Three fixed effects were of interest: PAS comparison (3 levels: NE–WG, WG–ACE, and ACE–CE), Lobe (5 levels: all, frontal, occipital, parietal, and temporal), and Time Range (2 levels: VAN and P3a; Fig. 3). We performed model comparisons between models that did or did not include the fixed effects and their interactions to find the best compromise between an explanatory and a parsimonious model. This was done using the log-likelihood ratio between the 2 models because this ratio approximates a χ² distribution. A χ² test can thus be used to assess whether 2 models differ significantly, where the test statistic is the log-likelihood ratio and the degrees of freedom is the difference in free parameters of the 2 models.

Results

Behavioral Results

The behavioral results showed differences in accuracy and response times for the 4 perceptual ratings. For the behavioral analyses, all data points were used, despite differences in contrasts. The analysis was performed to show the relationship between performance and perceptual clarity.

The proportion correct per PAS rating (4) was modeled using a logistic regression model. Each participant (11) was modeled with an individual intercept. Comparing this model with a null model, which assigns an identical proportion correct to each PAS rating, we found that the model including PAS ratings fitted proportion correct significantly better than the null model, χ²(3) = 1943.3, P < 0.001. This means that the accuracies differed significantly across PAS ratings. It can be seen from the confidence intervals of proportion correct (Fig. 1C) that performance was not significantly different from chance when participants reported NE. For the remaining PAS ratings, performance was significantly different from chance and different from one another in an ordered manner, that is, WG_ACC > NE_ACC, z = 19.18, P < 0.001; ACE_ACC > WG_ACC, z = 14.21, P < 0.001; and CE_ACC > ACE_ACC, z = 2.16, P = 0.031. CE_ACC > ACE_ACC was not significant when Bonferroni-corrected (3), P_BONF = 0.092.

Log response times per PAS rating (4) for the identification task were modeled factorially. Each participant (11) was modeled with an individual intercept. Comparing this model against a null model, we found that the model including PAS ratings explained significantly more than the null model, χ²(3) = 1818.8, P < 0.001. Response times decreased with clarity of experience: WG_RT < NE_RT, z = 2.14, P = 0.032; ACE_RT < WG_RT, z = 23.87, P < 0.001; CE_RT < ACE_RT, z = 11.11, P < 0.001. WG_RT < NE_RT was not significant when Bonferroni-corrected, P_BONF = 0.097. Overall, performance (measured using accuracy and response time) increased in relation to the clarity of perceptual consciousness.

The median number of trials per PAS rating used by a participant was: NE = 183; WG = 150; ACE = 177; CE = 121, indicating that participants used the scale in a graded manner.

Catch Trials

The median number of times participants used the 4 different PAS ratings on catch trials was: NE = 61; WG = 11; ACE = 0; and CE = 0.

This indicates that the sensory characteristics, figure-present versus figure-absent, correlates well with perceptual characteristics, that is, PAS rating, even though participants occasionally experienced a WG when no target was presented, indicating that some visual confabulation took place.

Illustrations of Components Found

We created grand averages for illustration (Fig. 2). In Figure 2A, the difference topographies for the differing neighboring comparisons are seen. In Figure 2B, an example of an ERF is shown from a temporal magnetometer. As can be seen, the components behind the VAN difference and the P3a difference are elicited. No formal statistics were done on these ERFs, since all statistics were done in source space using multivariate analyses. Nevertheless, it can be seen that the 2 components, VAN (130–320 ms) and P3a (320–510 ms), have comparable mean amplitude differences between conditions (Fig. 2A), indicating that any difference found using multivariate statistics reflects the consistency of information on the single trial level (Sandberg et al. 2014). The ratios (all close to 1) between peak differences for the VAN time range and the P3a time range for the 3 PAS comparisons indicated that they were indeed comparable. The peak differences over magnetometers were for NE–WG: VAN = 25.2 fT, P3a = 28.4 fT, ratio = 1.12; WG–ACE: VAN = 29.1 fT, P3a = 27.8 fT, ratio = 1.05; ACE–CE: VAN = 23.9 fT, P3a = 25.5 fT, ratio = 1.07.

Figure 2.

Sensor space data. (A) Topographic maps of the grand average difference waves between the neighboring PAS ratings. (B) Activity recorded at an example sensor (right temporal magnetometer) showcasing the components elicited, here exemplified by the WG–ACE comparison from the grand average over participants. The VAN difference (∼270 ms) and the P3a difference (∼440 ms) are both visible. (C) The position of the magnetometer on a participant.

Group-Level Analysis of Results From the Multivariate Decoding

To investigate which spatio-temporal features classified perceptual consciousness the best, the 3 effects of interest, PAS comparison (3), Lobe (5), and Time Range (2), and their interactions were modeled and evaluated for significance (the decoding accuracies are plotted in Fig. 3). Models including these effects were compared against a null model, which modeled accuracy as a constant. The Time Range model was not significantly different from the null model: χ²(1) = 1.1, P = 0.29. The PAS comparisons and Lobe models were, however, χ²(2) = 38.9, P < 0.001 and χ²(4) = 20.6, P < 0.001. None of the possible interactions between the fixed effects made a significant difference (for all tests, P > 0.30). Comparisons of the different levels of Lobe revealed that occipital sources were significantly better for classification than frontal sources, z = 4.46, P < 0.001. Occipital sources were also better for classification than temporal sources, z = 3.81, P < 0.001. These tests were also significant when Bonferroni-corrected for 10 comparisons. There was furthermore evidence of occipital sources classifying significantly better than parietal sources, z = 2.28, P = 0.023, and all sources together, z = 2.31, P = 0.021. Evidence of all sources together classifying better than frontal sources was also found, z = 2.15, P = 0.031. Finally, there was also evidence of parietal sources classifying better than frontal sources, z = 2.19, P = 0.029. These 4 comparisons were not significant when Bonferroni-corrected for multiple comparisons (10).

Figure 3.

Mean classification accuracies for each of the 5 lobes tested for the 3 PAS comparisons for each of the 2 ranges: The VAN range (132–312 ms) and the P3a range (324–512 ms). NE versus WG is the difference of a subjective experience as such. WG versus ACE is the experiential difference of content. ACE versus CE is the experiential difference of unambiguity. Of special importance is it that occipital sources can be used to classify all PAS comparisons significantly above chance. The error bars are 95% confidence intervals tested against chance, bootstrapped using 10 000 simulations, from a mixed model having Time Range (2) and PAS comparison (3), and Lobe (5) as fixed effects including all possible interactions. Participants (11) were modeled with individual intercepts (random effect).

Comparisons of the different levels of PAS revealed that both the WG–ACE comparison and the ACE–CE comparison were more accurate than the NE–WG comparison, z = 5.25, P < 0.001, and z = 6.11, P < 0.001, respectively. Both survived Bonferroni-correction for multiple comparisons (3).

Time Courses of the Classification

The analyses above focused on the classification accuracies over extended time periods. To investigate the earliest time post-stimulus that perceptual consciousness could be decoded (the time point at which no later information contributed to increased decoding accuracy), we performed classifications per time point in a cumulative manner as well (Fig. 4). For these analyses, the nth analysis included all time points up to and including the nth time point. The time range was from 200 ms pre-target to 600 ms post-target. With the downsampled frequency of 250 Hz, this resulted in 201 classification analyses being run for each of the 3 PAS comparisons. Only frontal and occipital lobes were tested, and all classifications were run within-participant.

Figure 4.

Mean cumulative time point classification accuracies (A) for the occipital lobe and (B) for the frontal lobe tested for the 3 PAS comparisons. The light gray indicates the VAN range (A) and the P3a range (B), respectively. Note that the largest increase in decoding accuracy occurred during the VAN at occipital sources. The darker gray area indicates 1 SEM. Curves have been smoothed by only plotting every 10th point. These points are based on the mean of the 9 samples that came before them. The SEM is calculated over 10 points as well.

The steepest rise in classification accuracy for the occipital sources (Fig. 4A) occurred in the VAN range, whereas the P3a range in the frontal sources did not seem to be associated with any change in classification accuracy (Fig. 4B). Paired t-tests corroborated this: for the occipital lobe, the difference in classification accuracy between 320 and 130 ms was significantly different from zero for all 3 PAS comparisons: NE–WG: t₍₁₀₎ = 3.06, P = 0.012; WG–ACE: t₍₁₀₎ = 2.97, P = 0.014; ACE–CE: t₍₁₀₎ = 2.67, P = 0.023, whereas for the frontal lobe the difference in classification accuracy between 510 and 320 ms was not significantly different from zero for any of the 3 PAS comparisons: NE–WG: t₍₁₀₎ = 0.45, P = 0.66; WG–ACE: t₍₁₀₎ = 1.74, P = 0.11; ACE–CE: t₍₁₀₎ = −0.075, P = 0.94.

Comparison Between the 2 Ranges

The multivariate analyses showed that only the occipital sources contain information for decoding all 3 PAS comparisons above chance (Fig. 3), and that only the VAN time range was associated with a significant increase in classification accuracy (Fig. 4A). Note that the temporal sources did not classify above chance for the NE–WG comparison. This fits well with the notion that temporal sources only start playing a role when the difference in experience is about content (Goodale and Milner 1992). However, it might be that all necessary processing takes place in the occipital lobe (e.g., V4), and that the temporal lobe is not necessary for an experience of content. This is entirely possible, especially because of evidence that V4 can process complex information such as shapes (Desimone and Schein 1987).

Analysis of Catch Trials

Six participants had enough catch trials rated NE and WG to do a classification between NE and WG. This number had to be greater than the number of folds (5). The trials were processed and analyzed in the same manner as described for the figure-present trials. Classifications were run for frontal and occipital sources in the VAN range and the P3a range. A mixed model was fitted with Lobe (2) and Time Range (2) as fixed effects and Participant (6) with random intercepts and with accuracy of the classifier as the dependent variable. The model with Lobe did not explain significantly more than a model with just an intercept, χ²(1) = 1.86, P = 0.17, but a model with Time Range did, χ²(1) = 6.20, P = 0.013. Adding the interaction between Time Range and Lobe did not explain significantly more, χ²(2) = 5.47, P = 0.065.

The effect of Time Range was driven by the P3a range, mean = 0.581, 95% CI [0.502; 0.660], classifying significantly better than the VAN range, mean = 0.435, 95% CI [0.356; 0.514]. The P3a range was thus marginally better than chance for classifying perceptual state. This was not a planned analysis, and the effect is marginal, but finding evidence for the P3a range being related to, and the VAN range unrelated to, illusory perception is interesting in its own right. An interpretation of this finding is that P3a reflects accumulation of internal evidence, veracious or not, resulting in a given report and does not reflect perceptual consciousness itself (Melloni et al. 2011). We will return to this discussion later.

Difference in Lobe Size

A potential confound of the present analysis is that the tested lobes differ in regard to the number of reconstructed sources they each contain. Specifically, it may be expected that given a fixed number of examples (trials), a very high number of spatial features (sources) could reduce the ability of the classifier to find an optimal border in the data to distinguish PAS responses. To address this potential issue, we trained new classifiers based on frontal and occipital lobes using 1) various fractions of the available sources and 2) different numbers of trials. To address the potential issue of the number of spatial features, we first randomly sampled one-tenth of the available sources for each lobe and each time range, VAN or P3a, respectively. This was repeated 100 times, each time with a new and independent sample. A multivariate analysis was run for each sampling, otherwise using the same parameters as in earlier analyses. This procedure was also run for the following fractions: two-, three-, four-, five-, six-, seven-, eight-, and nine-tenths. These analyses were run within-participant. For each range, VAN and P3a, the mean classification accuracy across participants was calculated for each lobe, frontal and occipital (Fig. 5A). We modeled Accuracy with PAS comparison (3) and Fraction as fixed effects and Participants (11) modeled as having a random intercept. No correlations were found between Fraction and Accuracy, and also no interaction between Fraction and PAS comparison, occipital Fs < 1 and frontal Fs < 0.01. There seemed to be an effect of PAS comparison, all Fs > 2.95 reflecting the results of the main analysis, but no formal test was done as this is unrelated to a test of the potential confound. Taken together, the analyses thus revealed that the difference in source number between the lobes could not explain the results of the main analysis as the multivariate model was indifferent to the fraction of sources used as long as one-tenth or more of the sources are used, corresponding to approximately 70 and approximately 240 spatial features for the occipital and frontal lobes, respectively. It should be noted that previous studies have found poor classification accuracy when a very low number of spatial features (below 20) was used (Haynes and Rees 2005; Sandberg et al. 2013), but the number of spatial features was significantly higher for all analyses in the present study.

Figure 5.

Control analyses: (A) Controlling for lobe size: Classification accuracies for the occipital lobe and for the frontal lobe collapsed over the 3 PAS comparisons. Separate lines are plotted for the VAN time range and the P3a range. All slopes are close to 0, ρ_min = −0.00070, ρ_max = 0.048, indicating that classification with the occipital lobe is best not simply because of differences in the number of reconstructed sources in the multivariate models across lobes. (B) Controlling for differences in the number of trials used for each classification: Individual observations, 3 for each participant, 1 for each PAS comparison, showing the relationship between classification accuracy for the occipital lobe for the VAN range and the number of trials used for classification. The linear regression line is drawn, ρ = 0.061. This indicates that the results are not a consequence of differences in the number of trials used for classification. (C) Controlling for differences in the number of trials with each stimulus used for classification: Individual observations showing the relationship between classification accuracy for the occipital lobe for the VAN time range and the difference between the number of rectangles and the number of rotated rectangles among the trials for that classification. Two linear regression lines are drawn, one with all observations, ρ = 0.21, and one with the rightmost outlying observation (encircled) removed, ρ = 0.056. This indicates that the results are not a consequence of differences in physical characteristics of the stimuli.

Number of Trials Used for Classification

The second potential confound mentioned earlier was that differing amount of trials were used for the classifications, and that this could be related to the accuracy of the classification. A linear regression (Fig. 5B) was run to investigate this relationship for the occipital sources in the VAN time range, ρ = 0.061, t₍₃₁₎ = 0.34, P = 0.73. The number of trials thus appears to be unrelated to the accuracy of the classifier.

Physical Characteristics Versus Perceptual Characteristics

A third potential confound was that some of the within-participant classification trial sets contained unequal amounts of trials with the 2 figures, a rectangle and a 45° rotated rectangle. It is possible that successful classification was based on decoding representations of physical properties, that is, orientation, rather than perceived clarity, that is, differences in perceptual consciousness. Therefore, we investigated the correlation between accuracy of the classification and how unequally the figures were distributed between the trials of that classification. Two linear regressions (Fig. 5C) were run for the occipital sources in the VAN time range, one with all data points, ρ = 0.21, t₍₃₁₎ = 1.17, P = 0.25, and one with the rightmost outlier removed, ρ = 0.056, t₍₃₀₎ = 0.31, P = 0.76. This indicates that there is no relation between variability of physical characteristics, that is, orientation of target stimuli, and the ability of the classifier to decode perceptual consciousness, that is, PAS ratings.

Discussion

In this study, we examined the neural activity related to becoming conscious of a visual stimulus. We found evidence that the MEG signal originating in the frontal lobe decoded graded differences in perceptual consciousness (measured using the PAS) significantly worse than the signal originating in the occipital lobe. Furthermore, the frontal activity could only be used to decode 1 out of the 3 PAS contrasts above chance (Fig. 3), and neither in the VAN nor P3a time ranges did frontal sources add to the predictive value of a classification algorithm (Fig. 4). While we found no mean difference in predictability of activity in the VAN and P3a range, only occipital sources in the VAN time range could be used to decode all 3 PAS comparisons (Fig. 3), and the greatest increase in predictive values was found during the VAN time range (Fig. 4). These results were unrelated to differences in the number of sources in the lobes (Fig. 5A), differences in the number of trials used to train the classifier (Fig. 5B), and differences in stimuli proportions (Fig. 5C).

Our results thus indicate that there are neural activations that systematically differ between experienced differences in perceptual consciousness. Taken together with previous behavioral experiments (Overgaard et al. 2006; Sandberg et al. 2010), this study provides evidence that perceptual consciousness is graded, and that differences between each gradation are best explained by the conglomerate activity of the neurons in the occipital lobe during the VAN time range, 130–320 ms. It should be noted that in this study, we do not distinguish between gradual/graded and partial awareness (Kouider et al. 2010), where gradual/graded awareness can be interpreted as meaning that the entire conscious percept is either more clear or less clear, whereas the partial awareness hypothesis states that the individual perceptual features are consciously perceived in an all-or-none manner.

Taken together, these results thus indicate that occipital activity seems a more likely candidate for a neural correlate of perceptual consciousness than does prefrontal activation. It should be noted that other studies (e.g., Sandberg et al. 2013) have found that other perceptual sources in, for instance, the temporal lobe are as predictive of conscious perception as occipital sources. It is likely that this difference is due to differences in stimuli. The simple stimuli used in the present experiment are expected to be processed mainly in relatively early visual areas, such as V4 (Pasupathy and Connor 2001). Had we used more complex stimuli, for example, tools or visual scenes, temporal sources might have been equally predictive. It is important to note that differences in activation patterns in perceptual areas across different conscious experiences do not, in themselves, imply a role for frontal areas. Based on the current findings, a likely explanation is that activity in perceptual areas is the main correlates of conscious perception of stimuli processed in those areas.

In HOT (Lau and Rosenthal 2011) and Information Integration Theory (Tononi 2004), it is proposed that consciousness is associated with prefrontal activations. The same is predicted by at least some versions of GWT (Baars 2005; Dehaene 2014). The present finding that sources in the frontal lobe decode differences in PAS levels significantly worse than those in the occipital lobe is thus not what one should expect seen from the vantage point of these theories. Furthermore, activity in the frontal sources could only be used to decode 1 out of the 3 PAS comparisons above chance (Fig. 3), and neither in the VAN or P3a time ranges did frontal sources seem to add a predictive value (Fig. 4B).

In contrast, the results of the experiment are consistent with theories that associate occipital activation with differences in perceptual consciousness such as Lamme's feedback theory (Lamme 2006), associating perceptual consciousness with recurrent processing within the occipital and temporal lobes, and the above-mentioned VAN proposed by Koivisto and Revonsuo (2010).

In the Neural GWT, the occipito-temporal VAN is often associated with construction of the percept, and the more frontal P3a is associated with becoming conscious of that content (Sergent et al. 2005). This interpretation appears somewhat inconsistent with the results of two of our main analyses. We have previously argued (Sandberg et al. 2014) that a prerequisite of perceptual consciousness (such as the construction of the percept) should not be more predictive of conscious perception than the actual correlate of conscious perception. As we found occipital activity to be more predictive than frontal activity, it thus appears unlikely that the occipital activity is only a prerequisite, especially given the similar mean differences between the 2 signals across the PAS comparisons (Fig. 2A). Additionally, in our spatio-temporal analyses (Fig. 4), frontal activity appeared completely unrelated to classification accuracy throughout the time range. If frontal activity in the P3a time range correlates with perceptual consciousness, we would instead have expected to see an increase in classification accuracy (Fig. 4B) around this time window. These results suggest that frontal activity may be related to processes typically (but somewhat inconsistently) occurring on trials with reported conscious perception. In an experimental context, these processes could be report, overt consideration, or memory consolidation. This interpretation is consistent with the studies mentioned in the introduction showing a decrease in the size of the P3a when a perceived stimulus is not task relevant (Pitts et al. 2012), and when it is expected (Melloni et al. 2011).

While our study did not provide evidence of a role for frontal sources, it is nevertheless not possible to conclusively dismiss a role of the late P3a time range as activity in this time range was not significantly less predictive than the VAN time range, although occipital sources were driving the accuracy in both ranges. One argument against the P3a being the key NCC is that it did not distinguish all PAS ratings as did the VAN (Fig. 3). Another argument is that no predictive value was added during the P3a time range as one might expect (Fig. 4), but proponents of GWT might argue that this is expected as all information is present in a preconscious state during the VAN range.

We also found evidence that activity during the P3a time range classified the NE–WG difference in catch trials better than that in the VAN time range, and we failed to reject the hypothesis that any cortical lobe performed better than the other. One interpretation of this finding, opposite to the general gist of our argument, is that P3a reflects perceptual consciousness better than VAN since it accounts for this illusory perception difference. However, if this were the case, it is surprising that the P3a was not more predictive of perceptual consciousness on veridical trials, and it is surprising that frontal sources could not be used to decode the NE–WG difference on such trials. Frontal sources in the P3a time range thus appeared to be predictive of the report of perception only when no stimulus was presented. For these reasons, an alternative explanation is that the P3a reflects differences in the accumulation of internal evidence for how to map perceptual consciousness onto PAS ratings (Melloni et al. 2011), and thus not perceptual consciousness itself. Based on the present experiment, however, it is not possible to conclude decisively about this matter.

In the NCC theories mentioned above, consciousness is often discussed and investigated by contrasting perceptual states dichotomously. PAS and other non-dichotomous scales allow for more options: One interpretation may be that there is more than one NCC, that is, one NCC per neighboring PAS comparison. If one maintains that there is only one proper NCC, then one has to decide on the defining feature of perceptual consciousness: that something is experienced at all (the difference between NE and WG), that one can specify the content of the experience (the difference between WG and ACE), or unambiguousness (the difference between ACE and CE). Assuming that for Lamme's theory, the defining feature of perceptual consciousness is an experience of content, it is noteworthy that temporal sources in the VAN time range can classify both the difference between ACE and WG and the difference between CE and ACE, providing some suggestive evidence for the involvement of temporal sources in the recurrent feedback to occipital sources. The present results also indicate that whether the dichotomously defining feature of perceptual consciousness is taken to be the difference between NE and WG or the difference between WG and ACE, there was no evidence for frontal sources decoding perceptual consciousness above chance (Fig. 3). It is of course possible that this is a question of statistical power, but importantly we found positive evidence for occipital sources classifying significantly better than frontal sources.

The REFCON model (Overgaard and Mogensen 2014) suggests that consciousness is related to information integrated in a “situational algorithmic strategy” (SAS) that is realized by a complex system of feed-forward and feed-backward mechanisms. Consciousness is seen as gradual, directly related to how integrated given information is in SAS, determined by its relevance, according to top-down expectations and evaluations. Thus, REFCON would predict that consciousness does not relate directly to one cortical structure, but rather, the structures that are “set up” in a given individual to realize particular functions. REFCON would predict that, in most cases, occipital regions are relatively more related to weak visual experiences than other cortical regions, but that more brain regions will be activated in an increasingly individual manner as a cascade as more information becomes available and the stimulus becomes more clearly experienced. Thus, REFCON does not assume that any correlation to a mental state is static but rather dynamic and may vary greatly between individuals. Such more theoretical aspects of REFCON are not directly reflected in the results of this experiment. However, its predictions related to gradual integration, the relatively stronger involvement of occipital regions for visual consciousness, and the increasing involvement of other cortical regions seem consistent with the results.

In summary, we found that participants reported differences in perceptual consciousness in a graded manner, and that occipital sources have the greatest predictive value for decoding these graded differences in perceptual consciousness, thus strengthening VAN and occipital lobe recurrent processing theories. The REFCON model is also compatible with the present results. In the context of this experiment, frontal activations did not appear directly related to perceptual consciousness, which is consistent with studies relating it to differences in, for example, attention and expectations. Using the Perceptual Awareness Scale made it possible to distinguish how different degrees of perceptual consciousness each are related to brain activity, highlighting an often neglected conceptual point that one needs to define a neural correlate of perceptual consciousness in order to find it—for example, as experience per se (as in the difference between NE and WG) or the experience of content (as in the difference between WG and ACE). Using the PAS is a possible way to approach and explore these different neural correlates of perceptual consciousness in more detail.

Funding

This work was supported by the European Research Council (Kristian Sandberg and Morten Overgaard).

Notes

Conflict of Interest: None declared.