Abstract

A multiplicity of sensory and cognitive functions has been attributed to the large cortical region at the temporo-parietal junction (TPJ). Using functional MRI, we report that a small region lateralized within the right TPJ responds robustly to certain simple visual stimuli (“vTPJ”). The vTPJ was found in all right hemispheres (n = 7), posterior to the auditory cortex. To manipulate stimuli and attention, subjects were presented with a mixture of visual and auditory stimuli in a concurrent block design in 2 experiments: (1) A simple visual stimulus (a grating pattern modulating in mean luminance) elicited robust responses in the vTPJ, whether or not the subject attended to vision and(2) a drifting low-contrast dartboard pattern of constant mean luminance evoked robust responses in the vTPJ when it was task-relevant (visual task), and smaller responses when it was not (auditory task). The results suggest a focal, visually responsive region within the right TPJ that is powerfully driven by certain visual stimuli (luminance fluctuations), and that can be driven by other visual stimuli when the subject is attending. The precise localization of this visually responsive region is helpful in segmenting the TPJ and to better understand its role in visual awareness and related disorders such as extinction and neglect.

Introduction

The human temporo-parietal junction (TPJ) is a very large cortical region. Its surface area is on the order of 50 cm2, comparable to the combined surface area of V1, V2, and V3. Researchers have attributed a multiplicity of functions to neurons within the TPJ (Bzdok et al. 2013; Carter and Huettel 2013), including social reasoning (Saxe and Kanwisher 2003), attentional reorienting (Corbetta et al. 2008), event timing (Battelli et al. 2007), detection of transitions between sensory modalities (Downar et al. 2000), and visual awareness (Karnath et al. 2003). It is likely that further experimental analyses will show that the TPJ comprises multiple regions that perform distinct functions, and that the region can be organized into distinct cortical systems.

One of the more remarkable claims connecting TPJ responses to visual awareness arises from electrocorticography. Electrical stimulation of occipital cortex can produce a visual phosphene, but this phosphene reaches consciousness only when the occipital stimulation elicits an accompanying signal in the TPJ (Beauchamp et al. 2012). A role for the TPJ in visual awareness is consistent with studies linking TPJ lesions to visual extinction, a deficit in detecting contralesional stimuli in the presence of ipsilesional stimuli (Karnath et al. 2003; Karnath and Rorden 2012). However, it is unknown whether the region of the TPJ in the electrocorticography study and in the lesion studies are the same, or even overlap.

Most of our understanding of the functional roles of TPJ derives from neurological case studies and neuroimaging. Identifying a specific region within the TPJ involved in a particular function is challenging. Because lesions are not confined by functional or anatomical boundaries, the precision with which an anatomical location can be associated with a functional deficit is limited, leading to uncertainty in the cortical substrates of extinction and related disorders. Secondly, neuroimaging studies often use paradigms that evoke only weak signals from the TPJ. To compensate, investigators typically align data from many subjects into a template space. This further reduces sensitivity to regions of interest (ROI) that do not align precisely in a template space (Glezer et al. 2009; Nieto-Castanon and Fedorenko 2012).

We examined the blood oxygenation level-dependent (BOLD) signal in individual subject responses to simple visual and auditory stimuli. We observed that temporal luminance modulations of a low spatial frequency grating pattern elicit robust responses in a region within the right TPJ, enabling us to make measurement in single subjects. A first experiment localized responses within the TPJ to a very salient flicker and tested how responses in this region depend on task and stimulus modality (visual vs. auditory). A second experiment measured the responses to a visible but less salient visual stimulus in this region.

We found a small (<1 cm2) visually responsive area in the anterior and inferior portion of the right TPJ (“vTPJ”) in all individuals (n = 7). The vTPJ is notable because it is (1) strongly responsive to visual stimuli and located adjacent to the auditory cortex, and (2) remote from identified visual field maps in occipital, temporal, and parietal cortices. In this study, our purpose is to characterize the location of this region, its task sensitivity, and its responsiveness to visual and auditory stimuli. The spatially resolved fMRI responses identify the position of this right TPJ subdivision with enough precision, so that the region can be identified using anatomical landmarks in individual subjects. Hence, future studies will be able to assess whether this region is the same location within right TPJ that plays a key role in visual awareness.

Materials and Methods

Subjects

Seven subjects participated in the experiments (age 29–62, all male), including the 3 authors. Informed consent was obtained from all 7 subjects. Stanford University's Institutional Review Board approved the experimental protocol.

Experiment 1

Experiment 1 (n = 7) measured BOLD responses to visual and auditory stimuli using a concurrent block design. In this method, the visual and auditory stimuli are presented in overlapping on- and off- blocks, but the auditory and visual blocks have different durations. Following a 12-s blank period (mean luminance), visual stimuli were presented in alternating “on” and “off” blocks, 12-s each, repeated 8 times per scan; auditory “on” and “off” blocks were 16-s each, repeated 6 per scan (total 204 s/scan). An experiment comprised 8 scans, 4 with an auditory, and 4 with a visual task, in alternating order.

The visual stimulus was a low spatial frequency grating (0.3 cpd), windowed within a rectangular aperture 12° in height by 20° in width. The grating modulated in mean luminance during 12-s “on” blocks and was static during “off” blocks. A single “on” block consisted of six 2-s epochs, each with 1.5 s of flicker and 0.5 s no-flicker gaps. The flicker rate was 4 Hz on 70% of the epochs and 3 Hz for 30% of epochs (“oddballs”).

Auditory stimuli were organized into 16-s “on” and “off” blocks. A single “on” block consisted of a series of 16 rising or falling tones (0.9 s tone and 0.1 s no-tone gaps). About 85% of tones rose from 100 to 500 Hz and 15% (oddballs) fell from 500 to 100 Hz. The frequency of auditory oddballs per scan was matched to that of visual oddballs; since auditory stimuli were twice as frequent, the oddball frequency per trial was half that of visual oddballs (15 vs. 30%). The sound level was 10 times threshold, measured during an EPI (echo planar imaging) scan immediately prior to the experiments. There were no tones during “off” blocks.

Subjects pressed a button to indicate a visual (4 scans) or auditory oddball (4 scans). Accuracy for oddball detection was high (94.2% visual and 97.7% auditory; mean across subjects).

Experiment 2

Experiment 2 (n = 5) was identical to Experiment 1 except for the visual stimulus, which was a low-contrast dartboard pattern (5% Michelson contrast) during “on” blocks and mean luminance during “off” blocks. The dartboard was windowed within a circular aperture with radius 6°. Alternate spokes of the dartboard drifted radially in or out, reversing in direction at pseudorandom times (average once per 10 s). The visual task was to detect a reversal in the motion direction; the auditory task was the same as Experiment 1.

Retinotopy and vTPJ Localizer

Population receptive field mapping procedures (Dumoulin and Wandell 2008) were used to identify retinotopic maps (n = 7) for each subject, and map boundaries were identified as in Winawer et al. (2010). ROI for V1, hV4, and TO-1 were limited to voxels whose receptive field centers were within 6 degrees of eccentricity, in order to match the stimulus extent.

A localizer experiment was conducted in separate scan sessions to independently identify the vTPJ ROI in 6 of 7 subjects. The purpose of the localizer was to separate the identification of the ROI from the measurement within the ROI, thereby avoiding the circularity of using the same measurement to select and to measure voxels. The localizer was identical to Experiment 1 except that it consisted of only the 4 scans with an auditory task. The purpose of using the auditory task and not the visual task for the localizer was to identify a region that is responsive to visual stimuli rather than visual attention. The localizer allowed us to identify a region in the right TPJ that was driven by visual responses even while attending auditory stimuli. For the seventh subject, a separate localizer session was not conducted; instead, the independent data set was derived by splitting data from Experiment 1 into 2 subsets comprising 2 scans each, 1 subset for localizing vTPJ, and 1 for subsequent analysis within the ROI.

Individually defined vTPJ ROIs were defined using the following criteria: (1) General linear model (GLM) variance explained exceeded a threshold (20%, except 10% for S4 and S6); (2) during an auditory task, beta weight for visual stimuli was greater than for auditory stimuli; and (3) location is posterior to planum temporale and dorsal/anterior to superior temporal gyrus.

GLM Analysis

Time series data were analyzed using a GLM implemented in vistasoft (https://github.com/vistalab/vistasoft). The GLM was solved in 2 ways. For the primary analysis, the GLM was solved on data averaged across repeated runs. The design included 4 regressors corresponding to each combination of stimulus and task (visual and auditory stimuli during a visual task; visual and auditory stimuli during an auditory task). Each of these regressors was a repeated boxcar (1 during stimulus blocks, otherwise 0) convolved with a fixed hemodynamic response function [difference in 2 gamma functions (Friston et al. 1998)]. In addition to these 4 regressors, there were 3 nuisance regressors to account for a slow drift in the BOLD signal. These regressors include polynomial terms up to second order (Kay et al. 2008).

A separate GLM was also conducted to account for responses to oddball stimuli. Because oddball stimuli occurred at random times throughout the scans, the data were not averaged across scans for this analysis. There were again 4 stimulus regressors corresponding to the 4 combinations of stimulus and task. In addition, there were 2 more regressors for visual and auditory oddball stimuli. These regressors were 1 for time points in which an oddball occurred during the appropriate task (e.g., visual oddball during visual task), and 0 for all other time points, convolved with the same fixed hemodynamic response function. The purpose of the oddball regressors was to capture any increase or decrease in the responses to oddballs beyond that captured by the boxcar for the entire stimulus blocks. Finally, there were 3 polynomial regressors for each scan. So if there were 8 scans, then there were 30 regressors (4 stimulus conditions, 2 oddballs, and 8 sets of 3 polynomials).

Stimulus Apparatus

Stimuli were controlled using the publicly available software (https://github.com/vistalab/vistadisp). Visual display devices were the same as reported in Kay et al. (2013). Auditory stimuli were presented via bone conduction using a Newmatic audio amplifier to drive piezo transducers, positioned on the cheeks over the zygomatic bone (http://cni.stanford.edu/wiki/MR_Hardware#Auditory_System).

MRI Data Acquisition

MRI data were collected at the Stanford Center for Cognitive and Neurobiological Imaging using a 3-T GE Signa MR750 scanner and a Nova 32-channel RF head coil. fMRI pulse sequence was single-shot, gradient-echo EPI. Thirty-two slices (2.5 mm, no gaps) were acquired roughly parallel to the calcarine sulcus. Scan parameters include: field-of-view 160 × 160 mm; phase-encode direction anterior–posterior (right-left in S3 and S6); matrix size 64 × 64; repetition time (TR) 2 s, echo time (TE) 28 ms, flip angle 68°, and nominal spatial resolution 2.5 × 2.5 × 2.5 mm3.

Data Preprocessing

Data preprocessing, including the discarding of initial frames, slice-timing and motion correction, field map-guided undistortion, and principal component analysis (PCA) denoising, was as reported in Kay et al. (2013), with one exception. Rather than using noise principal components as nuisance regressors in the GLM, the data were first denoised by PCA to render a new, denoised data set; this denoised data set was then used for the subsequent task-related GLM (https://github.com/kendrickkay/ Available from: URL GLMdenoise).

Results

A Visually Evoked Response Within the Right TPJ

To identify visually driven responses in the TPJ, we examined the responses to visual and auditory stimuli from Experiment 1, collapsing across visual and auditory detection tasks. As expected, visual responses were robust near the occipital pole, including primary visual cortex and extrastriate maps, and auditory responses were robust in the lateral temporal cortex in a region encompassing planum temporale (PT) and Heschl's gyrus (Fig. 1).

Figure 1.

Location of a visually responsive area in the right TPJ of subject S1. (A) Subjects were presented with a concurrent auditory and visual block design, at 2 different temporal frequencies, indicated by the red (visual) and blue (auditory) checkerboard in the lower left schematic. Responses to visual and auditory stimuli are shown on the smoothed right hemisphere (hot and cool colors, respectively). Just posterior to the auditory activation is a visually driven response within the TPJ (vTPJ: black arrow). The schematic in the inset shows the relationship between the auditory activation (blue), the visual activation (red), and the nearby major sulci (black) and gyri (light gray). PT, planum temporale; HG, Heschl's gyrus; STG, superior temporal gyrus; STS, superior temporal sulcus; LF, lateral fissure.

Figure 1.

Location of a visually responsive area in the right TPJ of subject S1. (A) Subjects were presented with a concurrent auditory and visual block design, at 2 different temporal frequencies, indicated by the red (visual) and blue (auditory) checkerboard in the lower left schematic. Responses to visual and auditory stimuli are shown on the smoothed right hemisphere (hot and cool colors, respectively). Just posterior to the auditory activation is a visually driven response within the TPJ (vTPJ: black arrow). The schematic in the inset shows the relationship between the auditory activation (blue), the visual activation (red), and the nearby major sulci (black) and gyri (light gray). PT, planum temporale; HG, Heschl's gyrus; STG, superior temporal gyrus; STS, superior temporal sulcus; LF, lateral fissure.

A focal, visually evoked response was identified within the right TPJ (Fig. 1, arrow). The activation was reproducible across measurements, as indicated by the agreement between the ROI definition and the measurements from Experiment 1 (Fig. 2). This response was found in the right TPJ in approximately the same anatomical location across subjects: in the temporal operculum, sandwiched by the planum temporale (PT; anterior to the vTPJ ) and the superior temporal gyrus (inferior/posterior to the vTPJ) (Figs 2 and 3; Supplementary Fig. 1). The PT was identified as an approximately triangular region which showed auditory responses, bounded on the anterior side by Heschl's gyrus, on the inferior side by the lip of the superior temporal gyrus, and posteriorly by the end of the auditory activation.

Figure 2.

Magnified views of the vTPJ and adjacent regions in 3 subjects. The activation maps show responses to visual and auditory stimuli in hot and cool colors, respectively. The black outline shows the location of the vTPJ as identified in a separate, independent localizer scan. The agreement between the color map and the ROI indicates the reliability across repeated measurements. The schematics below each map are as in Figure 1, showing the relationship between anatomical landmarks and functional activation. For each subject, the vTPJ is identified posterior to the PT.

Figure 2.

Magnified views of the vTPJ and adjacent regions in 3 subjects. The activation maps show responses to visual and auditory stimuli in hot and cool colors, respectively. The black outline shows the location of the vTPJ as identified in a separate, independent localizer scan. The agreement between the color map and the ROI indicates the reliability across repeated measurements. The schematics below each map are as in Figure 1, showing the relationship between anatomical landmarks and functional activation. For each subject, the vTPJ is identified posterior to the PT.

Figure 3.

Location of vTPJ in all 7 subjects. The location of the vTPJ ROI, as identified in a localizer scan, is shown in each of 7 subjects. The vTPJ (red) is posterior to the PT (blue) and Heschl's gyrus (black), superior to the superior temporal sulcus (yellow), and posterior to the lateral fissure (white). The location of the vTPJ relative to anatomical landmarks is similar across the 7 subjects. The location of the vTPJ relative to functional activations labeled “right TPJ” in the literature is shown in a template space (MNI) in the lower right. Activations from 85 studies reported in Appendix table of Decety and Lamm (2007) are plotted in MNI coordinates (black dots). More recent studies are not included. Studies reported in Talairach coordinates were converted to MNI coordinates via non-linear transform method. The 2 SD ellipsoid of the 3D locations is shaded in light green and encompasses a region of >30 mm along each of its 2 longest axes. The mean and SD of the MNI coordinates of vTPJ locations from the 7 subjects in this study are plotted as a red circle and black error bars. The vTPJ is within the region normally referred to as “TPJ” and is relatively anterior and inferior to the centroid of this region.

Figure 3.

Location of vTPJ in all 7 subjects. The location of the vTPJ ROI, as identified in a localizer scan, is shown in each of 7 subjects. The vTPJ (red) is posterior to the PT (blue) and Heschl's gyrus (black), superior to the superior temporal sulcus (yellow), and posterior to the lateral fissure (white). The location of the vTPJ relative to anatomical landmarks is similar across the 7 subjects. The location of the vTPJ relative to functional activations labeled “right TPJ” in the literature is shown in a template space (MNI) in the lower right. Activations from 85 studies reported in Appendix table of Decety and Lamm (2007) are plotted in MNI coordinates (black dots). More recent studies are not included. Studies reported in Talairach coordinates were converted to MNI coordinates via non-linear transform method. The 2 SD ellipsoid of the 3D locations is shaded in light green and encompasses a region of >30 mm along each of its 2 longest axes. The mean and SD of the MNI coordinates of vTPJ locations from the 7 subjects in this study are plotted as a red circle and black error bars. The vTPJ is within the region normally referred to as “TPJ” and is relatively anterior and inferior to the centroid of this region.

In addition to this visually evoked response, there were several nearby visual activations of lower amplitude that were not consistent across subjects: for example, anterior to the vTPJ near the supramarginal gyrus (S1 and S2), small patches within the STS (S1), and on the STG, posterior to the vTPJ (S3) (Fig. 2).

Visually Elicited Responses in the vTPJ Occur in the Absence of a Visual Task

Next, we separated the effects of modality (visual/auditory stimuli) and task (visual/auditory detection). We do so by separately examining the responses during the 2 tasks from Experiment 1 within the vTPJ ROI defined in the localizer.

The visual stimulus evokes a response in the vTPJ during both visual and auditory tasks. When the task pertained to the visual stimulus, the BOLD time series and amplitude spectrum show robust responses at 8 cycles per scan, the frequency of visual stimulation; there is only a small response at 6 cycles per scan, the frequency of auditory stimulation (Fig. 4A). When subjects perform the auditory task, the time series contains frequency components at both the visual and auditory stimulus frequencies. But even when subjects perform the auditory task, the visual response is larger than the auditory response (Fig. 4B). Hence, we consider the region visually dominant.

Figure 4.

Response in the vTPJ to luminance modulation and tones across 7 subjects. The ROI was defined in the localizer. Data are plotted from Experiment 1. (A) The mean BOLD responses to visual and auditory stimuli in the vTPJ during a visual task. The BOLD time series (mean, black solid line; SE, gray shading) shows a clear pattern of 8 cycles/scan (left panel); this pattern is reflected in the frequency domain as a peak at 8 cycles/scan (right panel). The middle inset shows sinusoids at the frequencies of the visual and auditory stimuli. (B) The BOLD response during the auditory task reflects both the visual stimulus (8 cycles/scan) and the auditory stimulus (6 cycles/scan); the largest amplitude response is at the frequency of the visual stimulus, indicated by the peak at 8 cycles in the spectral plot (right).

Figure 4.

Response in the vTPJ to luminance modulation and tones across 7 subjects. The ROI was defined in the localizer. Data are plotted from Experiment 1. (A) The mean BOLD responses to visual and auditory stimuli in the vTPJ during a visual task. The BOLD time series (mean, black solid line; SE, gray shading) shows a clear pattern of 8 cycles/scan (left panel); this pattern is reflected in the frequency domain as a peak at 8 cycles/scan (right panel). The middle inset shows sinusoids at the frequencies of the visual and auditory stimuli. (B) The BOLD response during the auditory task reflects both the visual stimulus (8 cycles/scan) and the auditory stimulus (6 cycles/scan); the largest amplitude response is at the frequency of the visual stimulus, indicated by the peak at 8 cycles in the spectral plot (right).

Stimulus Sensitivity Differs Between vTPJ and V1

For the luminance-modulated stimulus, the pattern of vTPJ responses is similar to the V1 pattern across the 7 subjects (Experiment 1). Both areas show a robust response to the flickering grating, and for both areas the response is slightly larger when subjects are engaged in a visual task compared with an auditory task (Fig. 5A, left panel). The response is stimulus driven: even in the absence of a visual task, luminance flicker elicits reliable responses in the vTPJ as well as V1.

Figure 5.

BOLD responses in the vTPJ and V1 depend on task and stimulus properties. (A) Plots show mean beta values ± SEM from a GLM fit to the BOLD response averaged across 7 subjects (luminance modulation experiment) and 5 subjects (drifting contrast experiment). The response to a luminance-modulated grating is similar in vTPJ and V1 (left panel). In each region, there is a reliable BOLD response during the visual task and a smaller but robust response during the auditory task. A drifting low-contrast pattern with constant mean luminance produces a different pattern of results (right panel). The response in V1 to the contrast pattern is much larger than that to the luminance pattern and is not modulated by task, whereas the vTPJ response is less for the contrast pattern than the luminance pattern. (B) The upper panel shows a frame of the luminance stimulus when the contrast is maximal. During “on” blocks, half of the bars were black and half modulated between black and white, creating a large modulation in mean luminance. The lower panel shows the low-contrast drifting dartboard pattern. The contrast is increased in one quadrant to enhance visibility in the figure. The arrows indicate direction of drifting motion.

Figure 5.

BOLD responses in the vTPJ and V1 depend on task and stimulus properties. (A) Plots show mean beta values ± SEM from a GLM fit to the BOLD response averaged across 7 subjects (luminance modulation experiment) and 5 subjects (drifting contrast experiment). The response to a luminance-modulated grating is similar in vTPJ and V1 (left panel). In each region, there is a reliable BOLD response during the visual task and a smaller but robust response during the auditory task. A drifting low-contrast pattern with constant mean luminance produces a different pattern of results (right panel). The response in V1 to the contrast pattern is much larger than that to the luminance pattern and is not modulated by task, whereas the vTPJ response is less for the contrast pattern than the luminance pattern. (B) The upper panel shows a frame of the luminance stimulus when the contrast is maximal. During “on” blocks, half of the bars were black and half modulated between black and white, creating a large modulation in mean luminance. The lower panel shows the low-contrast drifting dartboard pattern. The contrast is increased in one quadrant to enhance visibility in the figure. The arrows indicate direction of drifting motion.

When the visual stimulus changed to a drifting low-contrast pattern (Experiment 2), the vTPJ responses changed substantially (Fig. 5A, right panel). The biggest difference between the responses in the vTPJ to the 2 types of visual stimuli is during the auditory task. The response to the luminance stimulus is larger (0.5% BOLD increase) than that to the low-contrast pattern (0.2%). This response difference was assessed for statistical significance within each subject by bootstrapping; a GLM was solved 1000 times for each of the 2 stimuli by randomly sampling with replacement from the 4 scans of each type. For 4 of 5 subjects, the bootstrapped beta values were reliably higher for the luminance stimulus than the low-contrast stimulus (over 98% of comparisons), and for the fifth subject higher in 88% of comparisons. These results indicate that, in the absence of a visual task, a flickering luminance stimulus drives response in the vTPJ more effectively than a low-contrast drifting grating. In the presence of a visual task, both stimuli elicit 1% BOLD responses that are more than several standard deviations above baseline.

Because a modulating luminance pattern was used in the localizer scans to define the vTPJ ROIs, it might be the case that the larger response to this pattern compared with the drifting low-contrast pattern was due to biased voxel selection rather than a general stimulus preference in this region. To check for this possibility, we re-defined the vTPJ in the 5 subjects who participated in both experiments by concatenating the time series from Experiments 1 and 2, and contrasting the responses to the visual stimuli in both experiments against the auditory stimuli in both experiments (but only during the auditory task; GLM r2 > 20%, visual response > auditory response). This contrast results in largely similar ROI definitions, and a similar pattern of responses, with a larger response to the luminance modulation than the contrast pattern during the auditory task, confirming the prior analysis (Supplementary Fig. 2).

The V1 stimulus selectivity differs markedly from that of the vTPJ. In V1, the response to the drifting low-contrast pattern is much larger than that to the modulating luminance stimulus (2% signal change compared with 0.8%, averaging across the 2 tasks), which is opposite the pattern observed in the vTPJ. Moreover, unlike vTPJ, the V1 response to the drifting contrast pattern does not depend on task.

Responses in additional visual field maps, hV4 (a ventral map), and TO-1 [which overlaps area hMT+ (Amano et al. 2009)] show a pattern that is qualitatively similar to V1 (Supplementary Fig. 3). Like V1 and unlike vTPJ, the responses are larger for the drifting contrast pattern than the luminance modulation. Both areas show increased responses to visual stimuli during visual tasks compared with auditory tasks.

Oddball Detection in the vTPJ

The TPJ has been reported to be responsive to sensory oddballs, an infrequent stimulus presented within a train of repeated stimuli (Marois et al. 2000). The analyses reported above distinguished between stimulus “on” blocks and stimulus “off” blocks, but did not consider the effect of oddballs. Because oddballs only occurred during “on” blocks, it is possible that the greater response to “on” than to “off” blocks was driven by the oddball events only. To test this possibility, we reanalyzed the data from Experiment 1 (luminance modulation), using a GLM design that included 2 additional regressors, 1 for the oddball during visual tasks (a 3-Hz rather than 4-Hz flicker), and 1 for the oddball during the auditory task (falling rather than rising tone). Because these analyses also include regressors for the entire stimulus blocks, the weight on oddball regressors indicates the amount by which the response to oddballs differs from that to non-oddball stimuli. Unlike the analyses reported above, we did not average the time series across repeated scans, because the timing of the oddball stimuli was randomized across scans. Because oddballs occurred at random times within “on” blocks and did not occur in every “on” block, the oddball events are independent of the “on”–“off” block design.

The effect of oddball events is small (0.1 % for auditory oddballs and 0.2% for visual oddballs; Fig. 6). This modulation is significantly smaller than the response to visual stimuli irrespective of task (0.5% and 0.8% response to visual stimulus during the auditory and visual tasks, respectively), and is similar in amplitude to the response to auditory stimuli. Hence, we confirm that there is a reliable but small increase in the response to oddball stimuli compared with non-oddball stimuli. The largest portion of the response in the vTPJ is due to the visual stimulus rather than to the surprise arising from the oddball stimulus.

Figure 6.

Effect of stimulus oddball on BOLD responses in the vTPJ. Plots show the mean beta values across 7 subjects (±SEM) from a GLM fit to the BOLD time series in the vTPJ for Experiment 1. The largest responses are to the visual stimulus (open bars) regardless of task.

Figure 6.

Effect of stimulus oddball on BOLD responses in the vTPJ. Plots show the mean beta values across 7 subjects (±SEM) from a GLM fit to the BOLD time series in the vTPJ for Experiment 1. The largest responses are to the visual stimulus (open bars) regardless of task.

Discussion

Parcellating the TPJ

Simple visual stimuli evoke a robust response within the right TPJ (“vTPJ”). This region is <1 cm2, much smaller than the entire region referred to in the literature as TPJ (∼50 cm2; Fig. 1A). One recent report parcellates the TPJ into a more anterior and posterior region, corresponding to deployment of attention to external and internal stimuli, respectively (Bzdok et al. 2013). However, even these regions span many cm2 of cortex and likely comprise multiple functional subdivisions.

There are several experimental methods that enable us to identify this visually responsive subregion of the TPJ. First, fMRI analysis was conducted at the level of the individual subject, respecting the variation in sulcal and gyral patterns across individuals, as well as other anatomical differences. Had the data been analyzed in an anatomical template space, it is unlikely that the vTPJ would have manifested as a focal location reliably driven by visual stimuli. When we did co-register the individuals to an MNI template (Brett et al. 2002), we found considerable variability in the MNI coordinates of the center of the vTPJ across subjects: 53 ± 7, −35 ± 5, 20 ± 11 (mean ± SD, x, y, z). The standard deviation of the center position is large relative to the size of vTPJ. Because the vTPJ is located close to auditory cortex and to multimodal regions (Halgren et al. 1995; Downar et al. 2000; Ghazanfar and Schroeder 2006; Beauchamp et al. 2008), averaging the responses in a template anatomy would cause the visually driven responses to be intermingled with responses to other modalities, making the region look less visually selective than it is. In fact, the closest reported MNI coordinates to the mean coordinates in our study come from a multimodal attention paper (Downar et al. 2000; see Decety and Lamm (2007) for review of TPJ activations).

Secondly, measuring auditory responses provides an important functional landmark within each individual; identifying the vTPJ relative to auditory cortex increases our confidence that we have identified the corresponding region across the observers.

Thirdly, a block design provides high signal-to-noise, compared with event-related, designs (Liu et al. 2001).

Finally, the stimulus choice is likely important. For primary visual cortex, contrast patterns such as the moving dartboard used in Experiment 2 are much more efficient at driving BOLD responses compared with low spatial frequency flicker; however, the pattern in vTPJ was the opposite, with the luminance modulating stimulus producing the larger response.

Future work may show that the vTPJ itself may be parcellated further, if for example, it contains a retinotopic map. A suggestion of this possibility can be seen in one subject's data during attention conditions in retinotopic mapping studies (Saygin and Sereno 2008; Fig. 6, subj 9). Reliably identifying a retinotopic map in the vTPJ is challenging however, for several reasons. First, the area is small, so that there are few voxels to represent the different portions of the visual field. Secondly, the stimuli and task required for efficient mapping of this area likely differ from those used for identifying maps in occipital cortex. Thirdly, even if there is spatial tuning of the neurons in this region, they may not be arranged in a topographic map on the cortical surface. Many visual areas have been identified prior to the discovery that the areas contain spatial selectivity and/or retinotopic maps, including MT (Kuypers et al. 1965), the V4 complex (Meadows 1974; Lueck et al. 1989), the fusiform face area (Kanwisher et al. 1997), the parahippocampal place area (Epstein and Kanwisher 1998), and the visual word form area (Cohen et al. 2000).

TPJ in Visual Awareness

The vTPJ responses we observed may be part of a system that responds when the subject experiences visual awareness. A recent report using electrocorticography links the TPJ to visual awareness (Beauchamp et al. 2012), and our fMRI results are consistent with those findings. When subjects viewed a grating that flickered in mean luminance, a robust response was reliably observed in the vTPJ. Even when the subject's task was directed to another sensory modality (audition), the visual stimulus elicited a response in the vTPJ. Models of bottom-up visual salience indicate that luminance flicker is a powerful attentional cue (Carmi and Itti 2006). In our experiments, even when the luminance flicker is irrelevant to the task, it may reach visual awareness, in part by producing a response in the vTPJ.

However, not all visual stimuli elicit robust responses in the vTPJ. The response to the low-contrast grating (Experiment 2) was weak when it was not task relevant. Unlike the case of the luminance modulation, when the low-contrast pattern was not relevant for the task, it may fail to capture attention. When the contrast pattern was relevant for the task (visual oddball detection), the vTPJ response was large, indicating that the vTPJ is responsive to attended stimuli.

Future experiments designed to manipulate visual awareness are needed to test the hypothesis that the vTPJ identified here is the same region shown to support visual awareness in stimulation studies (Beauchamp et al. 2012).

Rethinking the Role of TPJ in Visual Attention

Our results differ from previous studies claiming that the right TPJ is involved in responses to visual stimuli only when they are relevant for a task (Corbetta et al. 2008). Had we run only the experiment with low-contrast gratings, we might have concluded that there is little response to task-irrelevant visual stimuli in the vTPJ, consistent with prior claims (de Fockert et al. 2004; Indovina and Macaluso 2007; Corbetta et al. 2008); however, the flickering luminance stimulus evokes a large, reliable response even when subjects were engaged in an auditory task. Moreover, the low-contrast stimulus evoked a powerful vTPJ response when the stimulus was task-relevant. Hence, it appears that the vTPJ can be strongly driven by salient visual stimuli and by attended stimuli. This vTPJ response pattern can be contrasted with primary visual cortex, where responses are strongly stimulus driven. The low-contrast dartboard pattern used in Experiment 2 produced large and reliable responses in V1 whether or not it was task-relevant.

TPJ in Extinction and Neglect

Consistent with the notion that regions within the TPJ are important for visual awareness are the findings that lesions in this region can create deficits of visual awareness including hemi-spatial neglect and visual extinction (Karnath and Rorden 2012). Lesion studies have limited precision in identifying the anatomical substrate of these conditions. One widely held view has been that the cortical substrate of neglect and extinction is a right posterior parietal lobe (Heilman et al. 1983; Vallar and Perani 1986); however, this view has been challenged, with the claim that the substrate for neglect lies in the temporal lobe and not in the parietal lobe (Karnath et al. 2001) or the junction between right temporal and parietal lobe (TPJ) (Karnath et al. 2003). The region we have identified, vTPJ, is located in the temporal operculum and not in the parietal lobe (Supplementary Fig. 1). If this region plays a critical role in visual awareness, then our results provide a possible explanation of why lesions to right superior temporal cortex would contribute to deficits in visual awareness of stimuli in the left hemifield. Further experiments are needed to directly test the spatial selectivity of the vTPJ and its role in visual awareness.

Conclusions

The vTPJ is a region within the right TPJ that responds powerfully to salient visual stimuli and to attended visual stimuli. Salient visual stimuli signal importance by their visual properties; attended visual stimuli are important because of their role in an ongoing task. The vTPJ responds to both and may play a role in generating awareness of these different types of significant visual stimuli. Using an individual localizer approach is critical for identifying this region, because it is small and its location in a standard template brain varies across individuals. The precise localization of visually evoked signals in the TPJ will be helpful in clarifying the role of the TPJ in visual awareness and associated disorders.

Authors' Contributions

Research was planned and designed by all authors and conducted by H.H. and J.W. The manuscript was written by H.H. and J.W. and edited by all the authors.

Supplementary Material

Supplementary material can be found at: http://www.cercor.oxfordjournals.org/.

Funding

This work was supported by JSPS Grant-in-Aid for Young Scientists (B) 26870605 (H.H.), NEI grant R01-EY03164 (B.A.W.), and NEI grant R00-EY022116 (J.W.).

Notes

We thank Anthony Wagner and Melina Uncapher for helpful discussion about functions of the TPJ, Hiromasa Takemura and Jason Yeatman for discussion about anatomy and white matter pathways, Franco Pestilli for suggestions about stimuli and task, Robert F. Dougherty for help in delivering auditory stimuli during scanning, Michael Perry for computer support, and Nathan Witthoft for reading an early draft of the manuscript, and Kendrick Kay for help with GLM and preprocessing analysis. Conflict of Interest: None declared.

References

Amano
K
Wandell
BA
Dumoulin
SO
.
2009
.
Visual field maps, population receptive field sizes, and visual field coverage in the human MT+ complex
.
J Neurophysiol
 .
102
:
2704
2718
.
Battelli
L
Pascual-Leone
A
Cavanagh
P
.
2007
.
The “when” pathway of the right parietal lobe
.
Trends Cogn Sci
 .
11
:
204
210
.
Beauchamp
MS
Sun
P
Baum
SH
Tolias
AS
Yoshor
D
.
2012
.
Electrocorticography links human temporoparietal junction to visual perception
.
Nat Neurosci
 .
15
:
957
959
.
Beauchamp
MS
Yasar
NE
Frye
RE
Ro
T
.
2008
.
Touch, sound and vision in human superior temporal sulcus
.
Neuroimage
 .
41
:
1011
1020
.
Brett
M
Johnsrude
IS
Owen
AM
.
2002
.
The problem of functional localization in the human brain
.
Nat Rev Neurosci
 .
3
:
243
249
.
Bzdok
D
Langner
R
Schilbach
L
Jakobs
O
Roski
C
Caspers
S
Laird
AR
Fox
PT
Zilles
K
Eickhoff
SB
.
2013
.
Characterization of the temporo-parietal junction by combining data-driven parcellation, complementary connectivity analyses, and functional decoding
.
Neuroimage
 .
81
:
381
392
.
Carmi
R
Itti
L
.
2006
.
Visual causes versus correlates of attentional selection in dynamic scenes
.
Vision Res
 .
46
:
4333
4345
.
Carter
RM
Huettel
SA
.
2013
.
A nexus model of the temporal-parietal junction
.
Trends Cogn Sci
 .
17
:
328
336
.
Cohen
L
Dehaene
S
Naccache
L
Lehericy
S
Dehaene-Lambertz
G
Henaff
MA
Michel
F
.
2000
.
The visual word form area: spatial and temporal characterization of an initial stage of reading in normal subjects and posterior split-brain patients
.
Brain
 .
123
(Pt 2)
:
291
307
.
Corbetta
M
Patel
G
Shulman
GL
.
2008
.
The reorienting system of the human brain: from environment to theory of mind
.
Neuron
 .
58
:
306
324
.
Decety
J
Lamm
C
.
2007
.
The role of the right temporoparietal junction in social interaction: how low-level computational processes contribute to meta-cognition
.
Neuroscientist
 .
13
:
580
593
.
de Fockert
J
Rees
G
Frith
C
Lavie
N
.
2004
.
Neural correlates of attentional capture in visual search
.
J Cogn Neurosci
 .
16
:
751
759
.
Downar
J
Crawley
AP
Mikulis
DJ
Davis
KD
.
2000
.
A multimodal cortical network for the detection of changes in the sensory environment
.
Nat Neurosci
 .
3
:
277
283
.
Dumoulin
SO
Wandell
BA
.
2008
.
Population receptive field estimates in human visual cortex
.
Neuroimage
 .
39
:
647
660
.
Epstein
R
Kanwisher
N
.
1998
.
A cortical representation of the local visual environment
.
Nature
 .
392
:
598
601
.
Friston
KJ
Fletcher
P
Josephs
O
Holmes
A
Rugg
MD
Turner
R
.
1998
.
Event-related fMRI: characterizing differential responses
.
Neuroimage
 .
7
:
30
40
.
Ghazanfar
AA
Schroeder
CE
.
2006
.
Is neocortex essentially multisensory?
Trends Cogn Sci
 .
10
:
278
285
.
Glezer
LS
Jiang
X
Riesenhuber
M
.
2009
.
Evidence for highly selective neuronal tuning to whole words in the “visual word form area”
.
Neuron
 .
62
:
199
204
.
Halgren
E
Baudena
P
Clarke
JM
Heit
G
Liegeois
C
Chauvel
P
Musolino
A
.
1995
.
Intracerebral potentials to rare target and distractor auditory and visual stimuli. I. Superior temporal plane and parietal lobe
.
Electroencephalogr Clin Neurophysiol
 .
94
:
191
220
.
Heilman
KM
Watson
RT
Valenstein
E
Damasio
AR
.
1983
.
Localization of lesions in neglect
. In:
Kertesz
A
, editor.
Localization in Neuropsychology
 .
New York
:
Academic Press
. pp.
471
492
.
Indovina
I
Macaluso
E
.
2007
.
Dissociation of stimulus relevance and saliency factors during shifts of visuospatial attention
.
Cereb Cortex
 .
17
:
1701
1711
.
Kanwisher
N
McDermott
J
Chun
MM
.
1997
.
The fusiform face area: a module in human extrastriate cortex specialized for face perception
.
J Neurosci
 .
17
:
4302
4311
.
Karnath
HO
Ferber
S
Himmelbach
M
.
2001
.
Spatial awareness is a function of the temporal not the posterior parietal lobe
.
Nature
 .
411
:
950
953
.
Karnath
HO
Himmelbach
M
Kuker
W
.
2003
.
The cortical substrate of visual extinction
.
Neuroreport
 .
14
:
437
442
.
Karnath
HO
Rorden
C
.
2012
.
The anatomy of spatial neglect
.
Neuropsychologia
 .
50
:
1010
1017
.
Kay
K
Winawer
J
Rokem
A
Mezer
A
Wandell
BA
.
2013
.
A two-stage cascade model of BOLD responses in human visual cortex
.
PLoS Comput Biol
 .
9
:
e1003079
.
Kay
KN
David
SV
Prenger
RJ
Hansen
KA
Gallant
JL
.
2008
.
Modeling low-frequency fluctuation and hemodynamic response timecourse in event-related fMRI
.
Hum Brain Mapp
 .
29
:
142
156
.
Kuypers
HG
Szwarcbart
MK
Mishkin
M
Rosvold
HE
.
1965
.
Occipitotemporal corticocortical connections in the rhesus monkey
.
Exp Neurol
 .
11
:
245
262
.
Liu
TT
Frank
LR
Wong
EC
Buxton
RB
.
2001
.
Detection power, estimation efficiency, and predictability in event-related fMRI
.
Neuroimage
 .
13
:
759
773
.
Lueck
CJ
Zeki
S
Friston
KJ
Deiber
MP
Cope
P
Cunningham
VJ
Lammertsma
AA
Kennard
C
Frackowiak
RS
.
1989
.
The colour centre in the cerebral cortex of man
.
Nature
 .
340
:
386
389
.
Marois
R
Leung
HC
Gore
JC
.
2000
.
A stimulus-driven approach to object identity and location processing in the human brain
.
Neuron
 .
25
:
717
728
.
Meadows
J
.
1974
.
Disturbed perception of colours associated with localized cerebral lesions
.
Brain
 .
97
:
615
632
.
Nieto-Castanon
A
Fedorenko
E
.
2012
.
Subject-specific functional localizers increase sensitivity and functional resolution of multi-subject analyses
.
Neuroimage
 .
63
:
1646
1669
.
Saxe
R
Kanwisher
N
.
2003
.
People thinking about thinking people. The role of the temporo-parietal junction in “theory of mind”
.
Neuroimage
 .
19
:
1835
1842
.
Saygin
AP
Sereno
MI
.
2008
.
Retinotopy and attention in human occipital, temporal, parietal, and frontal cortex
.
Cereb Cortex
 .
18
:
2158
2168
.
Vallar
G
Perani
D
.
1986
.
The anatomy of unilateral neglect after right-hemisphere stroke lesions. A clinical/CT-scan correlation study in man
.
Neuropsychologia
 .
24
:
609
622
.
Winawer
J
Horiguchi
H
Sayres
RA
Amano
K
Wandell
BA
.
2010
.
Mapping hV4 and ventral occipital cortex: the venous eclipse
.
J Vis
 .
10
(5)
:
1
22
.