Mugs and Plants: Object Semantic Knowledge Alters Perceptual Processing With Behavioral Ramifications

Abstract

Neural processing of objects with action associations recruits dorsal visual regions more than the neural processing of objects without such associations. We hypothesized that because the dorsal and ventral visual pathways have differing proportions of magno- and parvocellular input, there should be behavioral differences in perceptual tasks between manipulable and nonmanipulable objects. This hypothesis was tested in college-age adults across five experiments (Ns = 26, 26, 30, 25, and 25) using a gap-detection task, suited to the spatial resolution of parvocellular processing, and an object-flicker-discrimination task, suited to the temporal resolution of magnocellular processing. Directly predicted from the cellular composition of each pathway, a strong nonmanipulable-object advantage was observed in gap detection, and a small manipulable-object advantage was observed in flicker discrimination. Additionally, these effects were modulated by reducing object recognition through inversion and by suppressing magnocellular processing using red light. These results establish perceptual differences between objects dependent on semantic knowledge.

Keywords

object recognition perception semantic memory visual perception vision open data open materials

Object perception underlies effective engagement with our environment. When examining a potted plant, for example, precise access to long-term semantic memory may be needed to facilitate detailed identification of the plant to determine whether it needs water. However, when you reach for a mug containing your morning coffee, detailed identification may be less important (e.g., which one of your 10 favorite mugs it is), but computing a motor plan to bring the mug to your mouth without spilling its contents is pertinent. The visual object processing necessary to identify plants and manipulate mugs progresses along two distinct but interacting neural-processing pathways, each of which subserves different end goals (Kravitz et al., 2011; Mishkin & Ungerleider, 1982). Although the hypothesis of biased processing across the two pathways is widely accepted, it is unclear whether there are behavioral ramifications that directly relate to object processing being biased to different pathways. We provide evidence that object semantic knowledge evokes differences in object perception based on the recruited regions across the visual pathways. Importantly, we gained insight into neural processing by using purely behavioral methods that were custom tailored to the response properties of the neurons in both the dorsal pathway (higher temporal resolution) and ventral pathway (higher spatial resolution).

The ventral pathway, known as the “what” pathway, is characterized by object-feature selectivity and projects anteriorly from the occipital visual areas toward the hippocampus and the inferior temporal cortex (Kravitz et al., 2013). The dorsal pathway, colloquially known as the “where” or “how” pathway, courses from the occipital visual areas into the parietal cortex. The dorsal pathway is understood to subserve object processing for the purposes of reaching, grasping, and manipulation. Although both pathways contribute to visual perception, the demands of perceiving a specific object can differentially engage the two pathways such that the dorsal pathway provides greater contributions for visual processing of objects that are directly relevant for manipulation (e.g., hammer, plunger, or mug). Evidence for dynamic and biased recruitment of the visual pathways has been garnered from a wide range of techniques and paradigms. For example, when participants are presented with objects of various categories, including tools, places, animals, and faces, an increase in dorsal-pathway processing, specifically in the left ventral premotor and posterior parietal cortices, has been observed exclusively in response to tool presentation (Chao & Martin, 2000). Similar results evidencing a dorsal bias for manipulable objects are observed in functional MRI studies (Almeida et al., 2013; Chen et al., 2018; Mahon et al., 2007; Noppeney et al., 2006) and with various other paradigms, including interocular suppression (Fang & He, 2005) and continuous flash suppression (Almeida et al., 2008, 2010; but see Almeida et al., 2014; Sakuraba et al., 2012).

Whereas neurophysiological evidence has been convincing in showing selectivity for objects across the two streams, the impact of differential processing across the two pathways on object perception and subsequent behavior has not been characterized. Hypotheses regarding how dynamic recruitment of the visual pathways influences perception and behavior are based on an asymmetry in the cellular innervation of the two pathways that overlaps with the separate magnocellular and parvocellular channels identified in the anatomical architecture of visual processing (Baizer et al., 1991; Ferrera et al., 1992; Maunsell et al., 1990). The asymmetry in input endows each pathway with different response properties in accordance with cell stimulus preferences within the magno- and parvocellular channels. These channels originate from different types of ganglion cells within the retina and course separately through different layers of the lateral geniculate nucleus (Leventhal et al., 1981; Perry et al., 1984) to innervate separate layers of V1 (Blasdel & Lund, 1983). From V1, the parvocellular channel can be followed into V2 and V4 and then into the inferior temporal parietal cortex, whereas the magnocellular channel can be traced into different regions of V2 through V3d and middle temporal (MT) and into the posterior parietal cortices (DeYoe & van Essen, 1988; Livingstone & Hubel, 1988).

Differing innervation of the two visual pathways leads to different response properties that convey information of different spatial and temporal resolutions. The heavily myelinated magnocellular channel is derived from the parasol ganglion cells with relatively large receptive fields, spanning large regions of the retina (Maunsell et al., 1999; Nassi & Callaway, 2009). The features of the parasol retinal ganglion cells enable the dorsally biased magnocellular channel to encode information with higher temporal resolution than the parvocellular channel (Pokorny & Smith, 1997; but see Maunsell et al., 1999). Conversely, the parvocellular channel is derived from the midget retinal ganglion cells, which receive input primarily from cone receptors and have smaller receptive fields (Nassi & Callaway, 2009). The features of the midget retinal ganglion cells enable the ventrally biased parvocellular channel to encode information with a higher spatial resolution than the magnocellular channel (Derrington & Lennie, 1984; Leonova et al., 2003; McAnany & Alexander, 2008).

Statement of Relevance

It is commonly thought that what we know can change how we see the world. For a human, one very important thing we can know about an object is whether it is a tool. In the brain, when we perceive a tool, such as a mug, our parietal cortex is more strongly recruited than when we see a nontool, such as a potted plant. In this research, we found that the difference in the parts of the brain that are recruited when we see a tool or a nontool directly impacts our perception. If we see a tool, which recruits the parietal cortex, we perceive this object with higher temporal resolution. In other words, we see tools faster than we see nontools. Conversely, when we see a nontool, there is a benefit in spatial resolution, so we see the details better. This work suggests that what you know changes what and how you see.

The connections between the magnocellular channel and temporal resolution and the parvocellular channel and spatial resolution have been used to study the neural basis of several behavioral effects, ranging from spatial attention (Yeshurun, 2004) to the effects of fear on visual perception (Bocanegra & Zeelenberg, 2009). Most relevant, the asymmetric contributions of the magno- and parvocellular channels to the dorsal and ventral pathways have been proposed as a mechanism for near-hand effects on perception (Chan et al., 2013; Gozli et al., 2012). In one such study, it was hypothesized that when the participants’ hands were near the stimuli then processing would be biased to the dorsal pathway, which is more predominantly magnocellular, leading to enhanced sensitivity to temporal blinks. Conversely, when the participants’ hands were far from the stimuli then processing would be biased to the ventral pathway, which is more predominantly parvocellular, leading to an enhanced sensitivity to spatial gaps (Gozli et al., 2012). Consistent with differential engagement of two streams, it was observed that placing hands near the stimuli increased participants’ sensitivity to temporal blinks, whereas moving hands away from the screen increased sensitivity to spatial gaps.

Knowing that semantic knowledge of manipulability biases processing to the dorsal stream and that the dorsal and ventral streams have different response profiles because of differential innervation by the magnocellular and parvocellular channels, we hypothesized that semantic knowledge of object manipulability should have consequences for perceptual processing. Thus, because semantic knowledge of an object’s manipulability determines which pathway will be biased, we predicted that (a) manipulable objects that elicit a higher degree of magnocellularly biased dorsal processing would be processed with higher temporal resolution and (b) nonmanipulable objects that rely more on parvocellularly biased ventral processing would be processed with higher spatial resolution. Across five experiments, we tested these hypotheses by comparing spatial and temporal resolution across two object groups: manipulable and nonmanipulable objects.

Method

Participants

All participants were recruited from The George Washington University’s participant pool. All gave informed consent according to the university’s institutional review board, were naive to the purpose of the experiments, and reported normal or corrected-to-normal vision. In Experiment 4, which utilized color stimuli, no participant reported color blindness.

For each experiment, sample sizes were chosen on the basis of behavioral studies demonstrating similar effects (i.e., Gozli et al., 2012), and participants were recruited in batches of six to 10 until at least 25 participants with accuracy above chance were accumulated. For Experiment 1, 26 undergraduate students (19 female; average age = 19.2 years; six left-handed) were recruited. No participant was removed from the analyses. For Experiment 2, 27 undergraduate students were recruited. Twenty-six participants’ data (14 female; average age = 19.0 years; one left-handed) were analyzed; one participant was removed for below-chance accuracy in one of the conditions. For Experiment 3, 33 undergraduate students were recruited. Thirty participants’ data (26 female; average age = 19.07 years; three left-handed) were analyzed; three participants were removed for below-chance accuracy in at least one of the conditions. For Experiment 4, 40 undergraduate students were recruited. Twenty-five participants’ data (14 female; average age = 19.0 years; one left-handed) were analyzed; six participants were removed for not finishing the experiment, and nine were removed for having below-chance accuracy in at least one of the conditions. For Experiment 5, 26 undergraduate students were recruited. Twenty-five participants’ data (18 female; average age = 19.5 years; none left-handed) were analyzed; one participant was removed for chance accuracy in all conditions.

Apparatus and stimuli

All experiments were presented on a 24-in. Acer GN246 HL monitor with a refresh rate of 144 Hz, positioned at a distance of 60 cm from the viewer in a dark room. The experiment was generated and presented using PsychoPy (Version 1.82).

The stimuli in Experiments 1, 2, 4, and 5 consisted of line drawings of real-world, everyday objects obtained from The Noun Project, an online repository of object icons and clip art (https://thenounproject.com). The object stimuli consisted of 10 line drawings of manipulable objects and 10 line drawings of nonmanipulable objects. The manipulable objects were a snow shovel, handsaw, plunger, screwdriver, hammer, wrench, knife, bottle opener, spatula, and mug. The nonmanipulable objects were a fire hydrant, picture frame, window, toilet, candle, garbage can, water fountain, potted plant, fan, and lamp. The stimuli were displayed as large as possible in a 4° × 4° area, and all stimuli were presented in black (hue, saturation, value [HSV] = 0, 0, 0) on a dark-gray background (HSV = 0, 0, 50). All objects are displayed in Figure 1.

Fig. 1.

Stimulus sets used in the experimental paradigms. The stimuli in Experiments 1, 2, 4, and 5 (a) consisted of line drawings of manipulable and nonmanipulable objects. The stimuli in Experiment 3 (b) consisted of pictures of real-world manipulable and nonmanipulable objects. The numbers beneath each object correspond to the size of the spatial gap titrated to each object.

We controlled for low-level differences between objects by calculating the mean luminance, size, aspect ratio (i.e., measure of elongation), and spatial frequency (i.e., average distance from origin of the points calculated from a 2D fast Fourier transform) for each object. Independent-samples t tests confirmed that there were no mean differences between object groups in luminance, t(18) = −0.960, p = .350; mean size, t(18) = 1.043, p = .311; aspect ratio, t(18) = 1.209, p = .242; or spatial frequency, t(18) = −0.155, p = .879. The width of the bottom line was controlled for each object on which the gap appeared. There was no significant difference between manipulable and nonmanipulable objects in bottom-line width, independent-samples t test: t(18) = 0.922, p = .369.

For Experiment 3, the stimuli consisted of images of real-world, everyday objects obtained from Google image searches and manipulated in GIMP (Version 2.10.4). The object stimuli consisted of 10 images of the same manipulable objects and 10 images of the same nonmanipulable objects used in Experiments 1, 2, 4, and 5. The stimuli were displayed as large as possible in a 4° × 4° area, and all stimuli were presented in color on a dark-gray background (HSV = 0, 0, 50). All objects are displayed in Figure 1.

Additionally, we controlled for low-level differences between the colored objects used in Experiment 3 by calculating the mean size, hue, saturation, value, aspect ratio, and spatial frequency. Independent-samples t tests confirmed that there were no significant differences between manipulable and nonmanipulable objects for mean size, t(18) = −1.027, p = .318; hue, t(18) = −0.309, p = .761; saturation, t(18) = −0.031, p = .976; value, t(18) = 0.032, p = .975; aspect ratio, t(18) = −0.094, p = .927; or spatial frequency, t(18) = −1.466, p = .160. Additionally, we controlled the width of the bottom line of each object on which the gap would appear. There was no significant difference between manipulable and nonmanipulable objects in bottom-line width, independent-samples t test: t(18) = 0.557, p = .584.

Task

In Experiment 1, each trial began with a single central fixation point, which subtended a visual angle of 1° × 1°. The central fixation point was rendered in white (HSV = 0, 0, 100). After 1,000 ms of fixation presentation, a single object line drawing would appear 4° to the left or right of the fixation point. The side of presentation was counterbalanced such that each participant saw an equal number of stimuli on the left and on the right. The stimuli were displayed as large as possible within a 4° × 4° area, and all stimuli were presented in color on a dark-gray background (HSV = 0, 0, 50).

In one half of the experiment, the object appeared with or without a spatial gap in the center of the bottom line of the object. The objects had an equal probability of having a spatial gap or not having a spatial gap. The objects appeared for 100 ms, and participants were to report the presence of the spatial gap by pressing the right control button (present) or the left control button (absent) on the keyboard. Feedback was presented on incorrect trials only.

A staircase procedure was used to calibrate the size of the gap to each object to ensure that the gap was equally perceptible across each object regardless of each object’s individual characteristics (Pelli & Bex, 2013). Sixteen undergraduate students (13 female; average age = 18.8 years; four left-handed) from The George Washington University participated in exchange for course credit. In each trial, the object would be presented with or without the spatial gap, which would begin at a size of 0.025^o visual angle. If the gap was detected correctly for two consecutive trials, the gap decreased by one 0.005^o step. If the gap was missed, or if a false alarm was made to the absence of a gap, the gap was increased by one 0.005^o step for two consecutive trials. The gap was calibrated until 20 trials had been completed or until the staircase reversed direction (two correct followed by two incorrect, or vice versa) three times. The gap size for each object is displayed in Figure 1.

In the other half of the experiment, the object appeared with or without a temporal blink. The objects had an equal probability of having a temporal blink or not having a temporal blink. The blinks were 16 ms long. The object first appeared for 96 ms, followed by the temporal blink, and the object then appeared a second time for 32 ms. Participants were asked to report the presence of the temporal blink by pressing the right control key on the keyboard and to report the absence of the temporal blink by pressing the left control key. Feedback was presented on incorrect trials only.

The presentation of stimulus type was counterbalanced across participants such that half of the participants were first presented with spatial gaps and half of the participants were first presented with temporal blinks. Participants completed a total of 720 trials broken into six blocks, 360 trials of spatial gaps and 360 trials of temporal blinks.

Experiment 2 was identical to Experiment 1 except that each stimulus was presented upside down. The spatial gap was presented in the same place as in Experiment 1, relative to the object.

Experiment 3 was identical to Experiment 1 except that more realistic images were used instead of line drawings. A staircase procedure, similar to the one used with the line-drawing stimuli, was used to calibrate the size of the gap to each object to ensure that the gap would be equally perceptible across each object regardless of each object’s individual characteristics (Pelli & Bex, 2013). Twenty-three undergraduate students (18 female; average age = 18.9 years; four left-handed) from The George Washington University participated in exchange for course credit, and the procedure from Experiment 1 was used. The gap size for each object is displayed in Figure 1.

Experiment 4 was similar in trial structure to the temporal-blink condition of Experiment 1 except that instead of temporal-blink detection, each trial had a temporal blink that was either short or long in duration. The objects had an equal probability of having a short or a long temporal blink. In the short-blink condition, the object first appeared for 96 ms, followed by a 16-ms temporal blink, and then the object appeared a second time for 32 ms. In the long-blink condition, the object first appeared for 64 ms, followed by a 48-ms temporal blink, and then the object was presented a second time for 32 ms. Participants were to report a short temporal blink by pressing the “c” key on the keyboard and a long temporal blink by pressing the “m” key. Feedback was presented exclusively on incorrect trials. Participants completed a total of 720 trials broken into six blocks, 360 trials of short-blink duration and 360 trials of long-blink duration.

Experiment 5 was identical to the spatial-gap procedure of Experiment 1 except that the background color was either green (HSV = 110, 30, 90) or red (HSV = 5, 30, 90). The background color was counterbalanced across participants such that half of the participants would first be presented with the green background and half of the participants would first be presented with the red background. Participants completed a total of 720 trials broken into six blocks, 360 trials of green backgrounds and 360 trials of red backgrounds.

Results

Experiment 1

The spatial and temporal paradigms used d′ as a measure of perceptual sensitivity (Fig. 2). A two-way repeated measures analysis of variance (ANOVA) was conducted on d′ with object group (manipulable, nonmanipulable) and task type (gap, blink) as within-subjects variables (Fig. 2c, left). The ANOVA revealed no significant main effect of object group, F(1, 25) = 2.342, p = .138, η_p² = .086, but a significant main effect of task type, F(1, 25) = 50.953, p < .001, η_p² = .671; d′ sensitivity was higher for blinks than for gaps, M_gaps = 1.926, 95% confidence interval (CI) = [1.673, 2.179], M_blinks = 2.967, 95% CI = [2.680, 3.254]. Importantly, and consistent with the hypothesis of differential engagement of two pathways depending on object utility, results revealed a significant two-way interaction between object group and task type, F(1, 25) = 6.772, p = .015, η_p² = .213, driven by the difference between object groups in the gap condition, F(1, 25) = 9.888, p = .004, η_p² = .283; nonmanipulable objects had a higher d′ for gaps than manipulable objects (M_{nonmanipulable} = 2.021, 95% CI = [1.756, 2.285]; M_manipulable = 1.831, 95% CI = [1.589, 2.073]). The interaction effect and the driving simple main effect were consistent with the prediction that nonmanipulable objects, given their higher reliance on the ventral pathway, should yield higher sensitivity in the detection of spatial gaps than manipulable objects. Notably, the expected higher sensitivity to temporal gaps in the manipulable object set was not supported. The possibility of an advantage for manipulable objects in temporal sensitivity is further examined in Experiment 4.

Fig. 2.

Paradigms and results for Experiments 1 and 2. In both experiments, participants maintained fixation on the center cross. An object appeared to the left or right of fixation. For the spatial-gap paradigm (a), participants pressed a key to indicate whether the bottom line of the presented object contained a gap. For the temporal-blink paradigm (b), participants pressed a key to indicate whether the object flickered. At the end of every trial, participants were given feedback about whether their response was correct or incorrect. The graphs (c) show perceptual sensitivity (d′) for each combination of object group (manipulable, nonmanipulable) and task type (spatial gap, temporal blink), separately for Experiment 1 (left), in which object stimuli were presented upside up, and Experiment 2 (right), objects were presented upside down. Dots represent individual data, the bars show the means, and the error bars represent the standard error.

Experiment 2

To further probe the hypothesis that semantic knowledge of object utility (manipulable or nonmanipulable) biases the pathway that will ultimately process the object, we designed Experiment 2 to demonstrate that the hypothesized pathway biasing will not occur if the semantic identity of the objects is obscured (Firestone & Scholl, 2016). The same paradigm and objects from Experiment 1 were used, but the objects were inverted (upside down). The inversion preserved the low-level features of each object while impairing participants’ ability to rapidly recognize and access their semantic knowledge of the objects’ function. Inversion has been found to interfere with recognition of faces and objects (Diamond & Carey, 1986). Following the hypothesis that semantic knowledge of objects’ utility drives the perceptual difference between manipulable and nonmanipulable objects, we predicted that object inversion should reduce the difference between manipulable and nonmanipulable objects. A two-way repeated measures ANOVA was conducted on d′ with object group (manipulable, nonmanipulable) and task type (gap, blink) as within-subjects variables. The ANOVA revealed a significant main effect of object group, F(1, 25) = 5.860, p = .023, η_p² = .190; d′ was higher for nonmanipulable objects than for manipulable objects (M_{nonmanipulable} = 2.466, 95% CI = [2.174, 2.758]; M_manipulable = 2.340, 95% CI = [2.029, 2.650]; Fig. 2c, right). There was also a significant main effect of task type, F(1, 25) = 82.298, p < .001, η_p² = .767; d′ was higher for blinks than for gaps (M_gaps = 1.782, 95% CI = [1.476, 2.088]; M_blinks = 3.023, 95% CI = [2.723, 3.320]). As predicted, there was no significant interaction between object group and task type, F(1, 25) = 0.508, p = .483, η_p² = .020.

To assess the nonsignificant interaction effect, we conducted a Bayes factor (BF) analysis using the Bayesian repeated measures ANOVA in JASP (van den Bergh et al., 2020) comparing the posterior probability of a model with the main effects of task type and object group but no interaction term as the null hypothesis (H₀) to the posterior probability of a full model with the main effects and the interaction term as the alternative hypothesis (H₁). The BF analysis evaluating the nonsignificant interaction effect yielded a BF₀₁ of 4.83, indicating substantial evidence for the null hypothesis that does not include the interaction (Kass & Raftery, 1995).

Last, in order to statistically demonstrate that results of the inversion experiment are indeed different from those observed in Experiment 1, we subjected the data to a between-subjects ANOVA with object orientation (upright, inverted) as a between-subjects variable and object group and task type as within-subjects factors. The ANOVA revealed a significant three-way interaction between object group, task type, and orientation, F(1, 50) = 5.381, p = .024, η_p² = .097; as predicted, orientation significantly reduced the effect for the inverted objects.

Experiment 3

Although Experiment 1 provides evidence for biased engagement of the ventral pathway for nonmanipulable-object perception in a spatial task, it could be argued that despite careful low-level feature controls (e.g., luminance, size, elongation, spatial frequency), an uncontrolled low-level difference between manipulable and nonmanipulable objects is responsible for driving the manipulable versus nonmanipulable advantage in the spatial-gap task.

Experiment 3 used the same paradigm as Experiment 1 except that the line drawings were replaced by real-world images of corresponding objects (e.g., a line drawing of a candle was replaced by a picture of a candle; Fig. 1b). The prediction remained the same as in the original experiment. If object semantic knowledge determines which visual pathway object processing is biased toward, then nonmanipulable objects will be biased toward the ventral pathway, leading to higher sensitivity (as measured by d′) in the spatial-gap task. This would replicate the pattern of performance observed in Experiment 1. A two-way repeated measures ANOVA was conducted on d′ with object group (manipulable, nonmanipulable) and task type (gap, blink) as within-subjects variables. The ANOVA revealed a significant main effect of object group, F(1, 29) = 28.211, p < .001, η_p² = .493, with nonmanipulable objects having a higher sensitivity than manipulable objects (M_{nonmanipulable} = 2.361, 95% CI = [2.054, 2.668]; M_manipulable = 2.047, 95% CI = [1.773, 2.320]; Fig. 3). There was also a significant main effect of task type, F(1, 29) = 8.413, p = .007, η_p² = .225, with blinks having a higher average sensitivity than gaps (M_gaps = 1.998, 95% CI = [1.758, 2.238]; M_blinks = 2.410, 95% CI = [2.068, 2.751]). Importantly, a two-way interaction between object group and task type was also significant, F(1, 29) = 13.061, p = .001, η_p² = .311, driven by a simple main effect in the gap condition, F(1, 29) = 46.424, p < .001, η_p² = .616; nonmanipulable objects had a higher d′ for gaps than did the manipulable objects (M_{nonmanipulable} = 2.238, 95% CI = [1.982, 2.494]; M_manipulable = 1.758, 95% CI = [1.535, 1.981]). In addition to serving as a low-level control and a further test of the hypothesis, these results replicated those of Experiment 1, providing strong additional support for the hypothesis that the perceptual differences between manipulable objects and nonmanipulable objects are due to the semantic content of the objects.

Fig. 3.

Results from Experiment 3. Perceptual sensitivity (d′) is shown for each combination of object group (manipulable, nonmanipulable) and task type (spatial gap, temporal blink). Dots represent individual data, the bars show the means, and the error bars represent the standard error. Experiment 3 consisted of a control manipulation of Experiment 1 in which real object images were used instead of line drawings.

Experiment 4

Although the first three experiments yielded strong supporting evidence that gap detection is better on nonmanipulable objects than manipulable objects, perhaps resulting from bias toward the ventral pathway, manipulable objects did not elicit an advantage in temporal resolution, failing to provide evidence of a dorsal-pathway bias for manipulable objects. One explanation for the null effect is that the temporal-gap-detection task could also be construed as an abrupt-onset-detection task because of the way the object suddenly reappeared after the temporal blink. Abrupt onsets are known to be highly salient and easily detectable (Yantis & Jonides, 1984) and evoke strong activity in the lateral intraparietal area, a known attentional area with strong connections to the superior colliculus (Kusunoki et al., 2000). The strong connection between the lateral intraparietal area and the superior colliculus provides a mechanism by which abrupt onsets may bypass the parvo- and magnocellular visual-pathways framework that we aimed to test, thereby collapsing any differences in the d′ observed for manipulable and nonmanipulable objects. Another possible explanation for the null effect could be that the temporal-gap-detection task was too easy (ceiling effect), as evidenced by high d′ values in the temporal task. In order to have a task that is more difficult and is more specifically targeted to the high temporal resolution of the magnocellular channel, we redesigned the blink paradigm used in Experiments 1 through 3 as a discrimination task.¹ Objects were presented for 80 ms, removed from the screen for either a 16-ms or 48-ms blink, and redisplayed for 48 ms (Fig. 4). Participants’ task was to indicate whether the blink duration was short or long. We calculated d′ with short blinks considered as hits or misses and long blinks considered as correct rejections or false alarms. The prediction was that manipulable objects should have a higher d′ than nonmanipulable objects because of the increased magnocellular input, and therefore temporal resolution, of the dorsal pathway. A t test was used to analyze the difference between manipulable and nonmanipulable objects in d′ (M_{nonmanipulable} = 1.562, 95% CI = [1.354, 1.770]; M_manipulable = 1.620, 95% CI = [1.385, 1.855]), but no significant effect was found, t(48) = −0.362, p = .719.

Fig. 4.

Paradigm and results for Experiment 4. On each trial (a), participants maintained fixation on the center cross. An object would then appear to the left or right of fixation. The objects would flicker by being removed from the screen for 16 ms or 48 ms. Participants pressed a key to indicate whether the object’s blink duration was short or long, after which they were given feedback about whether their response was correct or incorrect. Reaction time (RT) is shown (b) for each combination of object group (manipulable, nonmanipulable) and blink duration (short, long). Dots represent individual data, the bars show the means, and the error bars represent the standard error.

Because of the temporal nature of the task, reaction time (RT) was also analyzed. We reasoned that any benefit in temporal resolution for manipulable objects might manifest as a speeded RT rather than a sensitivity benefit. A two-way repeated measures ANOVA was conducted on RT with object group (manipulable, nonmanipulable) and blink duration (short, long) as within-subjects variables. Only the RTs for correct responses were analyzed. The ANOVA revealed no significant main effect of object group, F(1, 24) = 4.080, p = .055, η_p² = .145 (Fig. 4). There was a significant main effect of blink duration, F(1, 24) = 4.850, p = .038, η_p² = .168; the short duration elicited higher RTs than did the long duration (M_short = 642.680, 95% CI = [604.400, 680.960]; M_long = 621.997, 95% CI = [581.794, 662.200]). The two-way interaction between object group and blink duration was significant, F(1, 24) = 9.011, p = .006, η_p² = .273, driven by a simple main effect in the short-blink condition, F(1, 24) = 11.710, p = .002, η_p² = .328; manipulable objects had a shorter RT for short blinks than did the nonmanipulable objects (M_{nonmanipulable} = 652.210, 95% CI = [612.360, 692.060]; M_manipulable = 633.150, 95% CI = [596.441, 669.859]). In accuracy, no effect was observed for object group, F(1, 24) = 3.666, p = .068, η_p² = .133; blink duration, F(1, 24) = 1.509, p = .231, η_p² = .059; or their interaction, F(1, 24) = 2.000, p = .170, η_p² = .077, suggesting that all significant effects were absorbed by the response time measure, and no speed/accuracy trade-offs were observed, indicating consistency with response time results.

These results—specifically, the simple main effect in the short-gap condition with manipulable objects having a shorter RT than nonmanipulable objects—lend some support to our hypothesis that manipulable objects would elicit higher temporal resolution than nonmanipulable-object processing. Although this result is not as strong as the observed benefit for nonmanipulable objects in spatial resolution, nor is it manifested in sensitivity, it does suggest that an advantage in temporal resolution is indeed present for manipulable objects. Investigating the exact nature of this temporal-resolution advantage may be a fruitful direction for future studies.

Experiment 5

The last test of our hypotheses was derived from the neurophysiological differences between the dorsal and ventral pathways. It was reasoned that if the differences in spatial-gap sensitivity for nonmanipulable objects are mechanistically derived from the magnocellular and parvocellular dichotomy of the two visual pathways, then the effect should be modulated through manipulation of the processing within the pathways. Ambient red light has been shown to suppress activity in the magnocellular channel because of the large number of visually responsive cells with red-inhibitory surrounds in their receptive fields (Wiesel & Hubel, 1966) and has been used to demonstrate the contribution of magnocellular processing in other behavioral effects, such as fear-processing and near-hand effects (Abrams & Weidler, 2014; West et al., 2010). To test our hypothesis that the spatial-resolution difference between manipulable and nonmanipulable objects observed in Experiments 1 and 3 was due to the differential input of the magnocellular channel to the two visual streams, we used the spatial-resolution paradigm from Experiment 1 and manipulated the background color to vary between green and red (Fig. 5a). Because of the suppression of the magnocellular channel by red light, it was predicted that the color of the background should modulate the perceptual difference in spatial gap detected in Experiment 1. A two-way repeated measures ANOVA was conducted on d′ with object group (manipulable, nonmanipulable) and background color (green, red) as within-subjects variables. The ANOVA revealed a significant main effect of object group, F(1, 25) = 17.171, p < .001, η_p² = .407; nonmanipulable objects had a higher average d′ than manipulable objects (M_{nonmanipulable} = 2.250, 95% CI = [1.941, 2.559]; M_manipulable = 2.022, 95% CI = [1.757, 2.287]; Fig. 5b). Note that this main effect is essentially a second replication of the spatial effect seen in Experiments 1 and 3. There was no significant main effect of background color, F(1, 25) = 2.138, p = .156, η_p² = .079, but, as predicted, there was a significant two-way interaction between object group and background color, F(1, 25) = 4.444, p = .045, η_p² = .151; the effect was increased with the red background. It was hypothesized that if the perceptual differences between manipulable and nonmanipulable objects are mechanistically derived from the magno- and parvocellular processing in the two visual pathways, then suppression of the magnocellular processing with red light should modulate the effect. The results of Experiment 5 supported our hypothesis by demonstrating an increase of the perceptual difference between manipulable and nonmanipulable objects with red light.

Fig. 5.

Paradigm and results of Experiment 5. On each trial (a), participants maintained fixation on the center cross. An object would then appear to the left or right of fixation. As in Experiment 1, participants were asked to detect the presence of a small gap in the bottom outline of the object, after which they were given feedback about whether their response was correct or incorrect. The background color (red or green) was manipulated. Perceptual sensitivity (d′) is shown for each combination of object group (manipulable, nonmanipulable) and background color (green, red). Dots represent individual data, the bars show the means, and the brackets show the standard error.

Discussion

We hypothesized that manipulable and nonmanipulable objects, because of differential recruitment of the visual pathways, would elicit perceptual differences stemming from the characteristic spatial and temporal resolutions associated with the magno- and parvocellular inputs. In five experiments, we found strong evidence in support of behavioral consequences driven by physiological and anatomical differences of the two visual pathways, and we argue that semantic knowledge of object manipulability guides processing along a particular pathway. If an object has a strong action association, processing is largely determined by activity in the dorsal pathway that is not found with objects that lack strong action association (Chao & Martin, 2000; Mahon et al., 2007; Noppeney et al., 2006). The increased level of activity in the parietal regions endows the perception of action-associated objects with greater access to the magnocellular channel that preferentially courses through the dorsal pathway. Without the enhanced dorsal-pathway activity evoked by action associations, object processing is more dependent on the ventral pathway, which has a higher ratio of parvocellular channel input than does the dorsal pathway (Baizer et al., 1991; Ferrera et al., 1992; Maunsell et al., 1990). Because of the differential magno- and parvocellular input to the two pathways, we predicted that the perception of objects with strong action associations would result in an increased temporal resolution and the perception of objects without such associations would result in an increased spatial resolution. In the experiments reported here, we found strong evidence that objects are perceived with different spatial resolutions and some evidence that objects are perceived with different temporal resolutions, depending on object semantic knowledge of manipulability. In two follow-up control experiments, further evidence was provided in support of the hypothesis that the difference between manipulable and nonmanipulable objects in the spatial task was driven by the semantic knowledge of the objects rather than possible low-level visual features. Namely, it was observed that the effect is curtailed by impeding the access of object semantic knowledge through inversion and that the perceptual differences between manipulable and nonmanipulable objects replicates with a separate set of more realistic object images (with different low-level properties). Last, to test whether the differing proportions of magno- and parvocellular input are responsible for the perceptual differences that were observed in our first four experiments, ambient red light was used to suppress activity of the magnocellular channel. It was observed that this increased the perceptual difference in the spatial task between manipulable and nonmanipulable objects.

It is important to be cognizant of the few limitations of our work in terms of generalizability, stemming from two sources: the participants and the stimuli. Our participants were all college-age adults, with an average age of 19, sampled from undergraduate psychology courses at The George Washington University. There is no reason to suspect that the observed findings are specific to this population, but further research will have to be done to directly probe this question. Additionally, our stimuli were chosen so as to have maximal experimental control while still using some variety of manipulable and nonmanipulable objects. Further research could examine whether the effects found in this research generalize not just to other exemplars of manipulable and nonmanipulable objects but also to objects with varying degrees of manipulability. Furthermore, except in Experiment 3, line drawings were used in all of our experiments. Although the findings in Experiment 3 do suggest that the results found in these experiments are not unique to line drawings and do generalize to more realistic images, it is yet unknown whether the results will generalize to real-world scenarios.

On the basis of the evidence provided, we argue that semantic knowledge of object manipulability, as defined by strong associations with an action appropriate for the item, generates the perceptual differences between manipulable and nonmanipulable objects. Previous studies have shown that manipulable objects evoke a larger degree of dorsal-pathway activity than do nonmanipulable objects (Almeida et al., 2013; Chao & Martin, 2000). However, the origin of this difference and of consequent perceptual differences between manipulable and nonmanipulable objects is poorly understood.

We propose that the differences we observe between manipulable and nonmanipulable objects derive directly from the near-hand effect reported by Gozli et al. (2012) in which stimuli presented proximally to the participants’ hands evoked a benefit in spatial resolution, similar to the nonmanipulable objects used in the experiments presented here, and stimuli presented distally from the participants’ hands evoked a benefit in temporal resolution. A possible mechanism supporting a connection between the near-hand effect and the manipulable versus nonmanipulable effect presented here derives from the bimodal cells in the anterior parietal cortex (Graziano & Gross, 1993). The bimodal cells have a somatosensory receptive field covering a part of the hand (a property possibly underlying the near-hand effect) and a visual receptive field corresponding to the visual field near the associated hand area. When an object is shown, if that object does not have a strong action association, then the activity of the bimodal cells is unaffected. After the organism has learned to manually manipulate the object, the bimodal cells respond to visual presentation of the object even without somatosensory input (Zhou & Fuster, 2000). Thus, the bimodal cells, with their responsiveness to hand location and their ability to become visually activated, may be a driver of the dorsal pathway’s object selectivity (Freud et al., 2016; Kastner et al., 2017; Vaziri-Pashkam & Xu, 2017), leading to the dorsal-pathway bias evoked by manipulable objects (Chao & Martin, 2000). The object-specific neural activity in the dorsal pathway, divorced from its need for somatosensory input and largely derived from magnocellular input, could then be read out during an identification task, leading to heightened temporal precision for manipulable objects.

Taken together, our results demonstrate that object semantic knowledge determines the processing bias of the object and evokes subsequent behavioral repercussions for perception and for action. This finding, in conjunction with the finding that manipulability has an influence on how attention interacts with object perception (Gomez et al., 2018), may point to manipulability, supported by a detailed neural mechanism, as an exemplar of cognitive penetrability. Additionally, our work underscores the need for careful consideration of object semantic knowledge, and its subsequent possible bias to either the dorsal or ventral pathway, when object images are used not only in psychological research but also in applied settings, such as display and product designs, environmental design, and the design of various cognitive assistants.

Footnotes

Transparency

Action Editor: Karen Rodrigue

Editor: Patricia J. Bauer

Author Contributions

D. Dubbelde and S. Shomstein designed the study. Testing, data collection, and data analysis were performed by D. Dubbelde under the supervision of S. Shomstein. D. Dubbelde and S. Shomstein wrote the manuscript. Both authors approved the final manuscript for submission.

Notes

References

Abrams

R. A.

Weidler

B. J.

(2014). Trade-offs in visual processing for stimuli near the hands. Attention, Perception, and Psychophysics, 76, 383–390. https://doi.org/10.3758/s13414-013-0583-1

Almeida

Fintzi

A. R.

Mahon

B. Z.

(2013). Tool manipulation knowledge is retrieved by way of the ventral visual object processing pathway. Cortex, 49(9), 2334–2344. https://doi.org/10.1016/j.cortex.2013.05.004

Almeida

Mahon

B. Z.

Caramazza

(2010). The role of the dorsal visual processing stream in tool identification. Psychological Science, 21(6), 772–778. https://doi.org/10.1177/0956797610371343

Almeida

Mahon

B. Z.

Nakayama

Caramazza

(2008). Unconscious processing dissociates along categorical lines. Proceedings of the National Academy of Sciences, USA, 105(39), 15214–15218. https://doi.org/10.1073/pnas.0805867105

Almeida

Mahon

B. Z.

Zapater-Raberov

Dziuba

Cabaço

Marques

Caramazza

(2014). Grasping with the eyes: The role of elongation in visual recognition of manipulable objects. Cognitive, Affective, & Behavioral Neuroscience, 14, 319–335. https://doi.org/10.3758/s13415-013-0208-0

Baizer

J. S.

Ungerleider

L. G.

Desimone

(1991). Organization of visual inputs to the inferior temporal and posterior parietal cortex in macaques. Journal of Neuroscience, 11(1), 168–190. https://doi.org/10.1523/jneurosci.11-01-00168.1991

Blasdel

G. G.

Lund

J. S.

(1983). Termination of afferent axons in macaque striate cortex. Journal of Neuroscience, 3(7), 1389–1413.

Bocanegra

B. R.

Zeelenberg

(2009). Emotion improves and impairs early vision. Psychological Science, 20(6), 707–713. https://doi.org/10.1111/j.1467-9280.2009.02354.x

Chan

Peterson

M. A.

Barense

M. D.

Pratt

(2013). How action influences object perception. Frontiers in Psychology, 4, Article 462. https://doi.org/10.3389/fpsyg.2013.00462

10.

Chao

L. L.

Martin

(2000). Representation of manipulable man-made objects in the dorsal stream. NeuroImage, 12(4), 478–484. https://doi.org/10.1006/nimg.2000.0635

11.

Chen

Snow

J. C.

Culham

J. C.

Goodale

M. A.

(2018). What role does “elongation” play in “tool-specific” activation and connectivity in the dorsal and ventral visual streams? Cerebral Cortex, 28(4), 1117–1131. https://doi.org/10.1093/cercor/bhx017

12.

Derrington

A. M.

Lennie

(1984). Spatial and temporal contrast sensitivities of neurones in lateral geniculate nucleus of macaque. The Journal of Physiology, 357(1), 219–240. https://doi.org/10.1113/jphysiol.1984.sp015498

13.

DeYoe

E. A.

van Essen

D. C.

(1988). Concurrent processing streams in monkey visual cortex. Trends in Neurosciences, 11(5), 219–226. https://doi.org/10.1016/0166-2236(88)90130-0

14.

Diamond

Carey

(1986). Why faces are and are not special. An effect of expertise. Journal of Experimental Psychology: General, 115(2), 107–117. https://doi.org/10.1037/0096-3445.115.2.107

15.

Fang

(2005). Cortical responses to invisible objects in the human dorsal and ventral pathways. Nature Neuroscience, 8(10), 1380–1385. https://doi.org/10.1038/nn1537

16.

Ferrera

V. P.

Nealey

T. A.

Maunsell

J. H. R.

(1992). Mixed parvocellular and magnocellular geniculate signals in visual area V4. Nature, 358(6389), 756–758. https://doi.org/10.1038/358756a0

17.

Firestone

Scholl

(2016). Cognition does not affect perception: Evaluating the evidence for “top-down” effects. Behavioral & Brain Sciences, 39, Article E229. https://doi.org/10.1017/S0140525X15000965

18.

Freud

Plaut

D. C.

Behrmann

(2016). “What” is happening in the dorsal visual pathway. Trends in Cognitive Sciences, 20(10), 773–784. https://doi.org/10.1016/j.tics.2016.08.003

19.

Gomez

M. A.

Skiba

R. M.

Snow

J. C.

(2018). Graspable objects grab attention more than images do. Psychological Science, 29(2), 206–218. https://doi.org/10.1177/0956797617730599

20.

Gozli

D. G.

West

G. L.

Pratt

(2012). Hand position alters vision by biasing processing through different visual pathways. Cognition, 124(2), 244–250. https://doi.org/10.1016/j.cognition.2012.04.008

21.

Graziano

M. S. A.

Gross

C. G.

(1993). A bimodal map of space: Somatosensory receptive fields in the macaque putamen with corresponding visual receptive fields. Experimental Brain Research, 97(1), 96–109. https://doi.org/10.1007/BF00228820

22.

Kass

Raftery

(1995). Bayes factors. Journal of the American Statistical Association, 90(430), 773–795. https://doi.org/10.1080/01621459.1995.10476572

23.

Kastner

Chen

Jeong

S. K.

Mruczek

R. E. B.

(2017). A brief comparative review of primate posterior parietal cortex: A novel hypothesis on the human toolmaker. Neuropsychologia, 105, 123–134. https://doi.org/10.1016/j.neuropsychologia.2017.01.034.A

24.

Kravitz

D. J.

Saleem

K. S.

Baker

C. I.

Mishkin

(2011). A new neural framework for visuospatial processing. Nature Reviews Neuroscience, 12(4), 217–230. https://doi.org/10.1038/nrn3008

25.

Kravitz

D. J.

Saleem

K. S.

Baker

C. I.

Ungerleider

L. G.

Mishkin

(2013). The ventral visual pathway: An expanded neural framework for the processing of object quality. Trends in Cognitive Sciences, 17(1), 26–49. https://doi.org/10.1016/j.tics.2012.10.011

26.

Kusunoki

Gottlieb

Goldberg

M. E.

(2000). The lateral intraparietal area as a salience map: The representation of abrupt onset, stimulus motion, and task relevance. Vision Research, 40(10), 1459–1468. https://doi.org/10.1016/S0042-6989(99)00212-6

27.

Leonova

Pokorny

Smith

V. C.

(2003). Spatial frequency processing in inferred PC- and MC-pathways. Vision Research, 43(20), 2133–2139. https://doi.org/10.1016/S0042-6989(03)00333-X

28.

Leventhal

A. G.

Rodieck

R. W.

Dreher

(1981). Retinal ganglion cell classes in the Old World monkey: Morphology and central projections. Science, 213(4512), 1142–1144. https://doi.org/10.1126/science.7268423

29.

Livingstone

M. S.

Hubel

D. H.

(1988). Do the relative mapping densities of the magno- and parvocellular systems vary with eccentricity? Journal of Neuroscience, 8(11), 4334–4339. https://doi.org/10.1523/jneurosci.08-11-04334.1988

30.

Mahon

B. Z.

Milleville

S. C.

Negri

G. A. L.

Rumiati

R. I.

Caramazza

Martin

(2007). Action-related properties shape object representations in the ventral stream. Neuron, 55(3), 507–520. https://doi.org/10.1016/j.neuron.2007.07.011

31.

Maunsell

J. H. R.

Ghose

G. M.

Assad

J. A.

McAdams

C. J.

Boudreau

C. E.

Noerager

B. D.

(1999). Visual response latencies of magnocellular and parvocellular LGN neurons in macaque monkeys. Visual Neuroscience, 16(1), 1–14. https://doi.org/10.1017/S0952523899156177

32.

Maunsell

J. H. R.

Nealey

T. A.

DePriest

D. D.

(1990). Magnocellular and parvocellular contributions to responses in the middle temporal visual area (MT) of the macaque monkey. Journal of Neuroscience, 10(10), 3323–3334. https://doi.org/10.1523/jneurosci.10-10-03323.1990

33.

McAnany

J. J.

Alexander

K. R.

(2008). Spatial frequencies used in Landolt C orientation judgments: Relation to inferred magnocellular and parvocellular pathways. Vision Research, 48(26), 2615–2624. https://doi.org/10.1016/j.visres.2008.02.012

34.

Mishkin

Ungerleider

L. G.

(1982). Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys. Behavioural Brain Research, 6(1), 57–77. https://doi.org/10.1016/0166-4328(82)90081-X

35.

Nassi

J. J.

Callaway

E. M.

(2009). Parallel processing strategies of the primate visual system. Nature Reviews Neuroscience, 10(5), 360–372. https://doi.org/10.1038/nrn2619

36.

Noppeney

Price

C. J.

Penny

W. D.

Friston

K. J.

(2006). Two distinct neural mechanisms for category-selective responses. Cerebral Cortex, 16(3), 437–445. https://doi.org/10.1093/cercor/bhi123

37.

Pelli

D. G.

Bex

(2013). Measuring contrast sensitivity. Vision Research, 90, 10–14.

38.

Perry

V. H.

Oehler

Cowey

(1984). Retinal ganglion cells that project to the dorsal lateral geniculate nucleus in the macaque monkey. Neuroscience, 12(4), 1101–1123. https://doi.org/10.1016/0306-4522(84)90006-X

39.

Pokorny

Smith

V. C.

(1997). Psychophysical signatures associated with magnocellular and parvocellular pathway contrast gain: Comment. Journal of the Optical Society of America A, 14(9), 2477–2486. https://doi.org/10.1364/josaa.15.002440

40.

Sakuraba

Sakai

Yamanaka

Yokosawa

Hirayama

(2012). Does the human dorsal stream really process a category for tools? Journal of Neuroscience, 32(11), 3949–3953. https://doi.org/10.1523/JNEUROSCI.3973-11.2012

41.

van den Bergh

van Doorn

Marsman

Draws

van Kesteren

E.-J.

Derks

Dablander

Gronau

Q. F.

Kucharský

Š.

Gupta

A. R. K. N.

Sarafoglou

Voelkel

J. G.

Stefan

Hinne

Matzke

Wagenmakers

E.-J.

(2020). A tutorial on conducting and interpreting a Bayesian ANOVA in JASP. L’Année Psychologique, 120(1), 73–96. https://doi.org/10.3917/anpsy1.201.0073

42.

Vaziri-Pashkam

(2017). Goal-directed visual processing differentially impacts human ventral and dorsal visual representations. Journal of Neuroscience, 37(36), 8767–8782. https://doi.org/10.1523/JNEUROSCI.3392-16.2017

43.

West

G. L.

Anderson

A. K.

Bedwell

J. S.

Pratt

(2010). Red diffuse light suppresses the accelerated perception of fear. Psychological Science, 21(7), 992–999. https://doi.org/10.1177/0956797610371966

44.

Wiesel

T. N.

Hubel

D. H.

(1966). Spatial and chromatic interactions in the lateral geniculate body of the rhesus monkey. Journal of Neurophysiology, 29(6), 1115–1156. https://doi.org/10.1152/jn.1966.29.6.1115

45.

Yantis

Jonides

(1984). Abrupt visual onsets and selective attention: Evidence from visual search. Journal of Experimental Psychology: Human Perception and Performance, 10(5), 601–621. https://doi.org/10.1037/0096-1523.10.5.601

46.

Yeshurun

(2004). Isoluminant stimuli and red background attenuate the effects of transient spatial attention on temporal resolution. Vision Research, 44(12), 1375–1387. https://doi.org/10.1016/j.visres.2003.12.016

47.

Zhou

Y.-D.

Fuster

J. M.

(2000). Visuo-tactile cross-modal associations in cortical somatosensory cells. Proceedings of the National Academy of Sciences, USA, 97(17), 9777–9782. https://doi.org/10.1073/pnas.97.17.9777