Newtonian Predictions Are Integrated With Sensory Information in 3D Motion Perception

Abstract

Because the motions of everyday objects obey Newtonian mechanics, perhaps these laws or approximations thereof are internalized by the brain to facilitate motion perception. Shepard’s seminal investigations of this hypothesis demonstrated that the visual system fills in missing information in a manner consistent with kinematic constraints. Here, we show that perception relies on internalized regularities not only when filling in missing information but also when available motion information is inconsistent with the expected outcome of a physical event. When healthy adult participants (Ns = 11, 11, 12, respectively, in Experiments 1, 2, and 3) viewed 3D billiard-ball collisions demonstrating varying degrees of consistency with Newtonian mechanics, their perceptual judgments of postcollision trajectories were biased toward the Newtonian outcome. These results were consistent with a maximum-likelihood model of sensory integration in which perceived target motion following a collision is a reliability-weighted average of a sensory estimate and an internal prediction consistent with Newtonian mechanics.

Keywords

causality motion perception vision visual perception visual tracking open data open materials

A hallmark of human intelligence is the ability to predict the behavior of moving objects under complex physical constraints. For example, billiards players make precise judgments of the ball trajectories that will unfold after a given shot of the cue ball, and they leverage this knowledge to sink their own balls and strategically place their opponent’s in difficult positions. Ideally, when presented with precollision conditions (e.g., relative positions and velocities), humans would make judgments of postcollision trajectories that are perfectly consistent with Newtonian mechanics, the physical laws that govern human-scale object motion.

One way that a nervous system could promote physical consistency in explicit judgments, as well as in perception and motor control, is by internalizing the regularities that hold between task variables in order to direct and constrain the processing of limited and noisy sensory information. This internalization hypothesis is often attributed to Shepard (1984), who demonstrated that the perceptual system fills in missing information according to kinematic constraints: In simple two-frame apparent-motion sequences, the displayed sequence is experienced as a rigid motion in three dimensions rather than as two distinct objects. However, although Shepard originally argued for internalized kinematic constraints on perception, he subsequently denied that dynamic variables are internalized to the same extent (Shepard, 1994), an argument that was consistent with contemporaneous experimental findings. For instance, when prompted to indicate the future flight paths of projectiles, observers’ explicit judgments deviated from Newtonian mechanics (Hecht & Bertamini, 2000; McCloskey, Caramazza, & Green, 1980). Non-Newtonian biases were also found when observers were asked to visually estimate the relative masses of two objects in a collision event (Gilden & Proffitt, 1989; Runeson & Vedeler, 1993; Todd & Warren, 1982).

In recent years, however, the internalization hypothesis has resurfaced as a hotly debated topic. Proponents point to evidence suggesting that the visual system is indeed sensitive to the latent physical quantities involved in Newtonian mechanics, such as mass distribution (Battaglia, Hamrick, & Tenenbaum, 2013; Firestone & Keil, 2016; Firestone & Scholl, 2014), force (Freyd, Pantzer, & Cheng, 1988; Hubbard, 2013), momentum (Freyd, 1983; Hubbard, 2019), and gravity (Hubbard, 2020; McIntyre, Zago, Berthoz, & Lacquaniti, 2001). Additionally, optimal-inference models have successfully accounted for human judgments in complex physical scenes by combining sensory observations with internal models that accurately represent Newtonian mechanics (Battaglia et al., 2013; Sanborn, Mansinghka, & Griffiths, 2013). By explicitly acknowledging the presence of noise in both sensory information and in additional internal sources of information (i.e., priors and predictions), such models show how apparently non-Newtonian biases can be consistent with accurate internalization. For example, Sanborn et al. (2013) demonstrated that the bias toward judging the object that moves faster prior to a collision as the heavier object can be explained by a shift in a decision boundary that reflects accurate knowledge of collision dynamics. This shift is produced when the prior over relative precollision velocities is more precise than the prior over relative postcollision velocities. Likewise, Battaglia et al. (2013) explain stability illusions (i.e., stable configurations of stacked objects that appear extremely unstable) as the result of noise in the predicted object trajectories generated by an “intuitive physics engine” (p. 18327).

In this work, we aimed to conduct a clear and direct test of the influence of internalized Newtonian regularities in visual perception while maintaining the principles of an optimal-inference approach. We reasoned that if the displayed object motions after a collision were experimentally manipulated to violate Newtonian mechanics, then the hypothesized internal predictive signal should bias the perceived motions toward the Newtonian outcome. Additionally, this bias should grow stronger as the reliability of the displayed visual information is reduced. We will refer to the internal predictive signal as the Newtonian prediction of the deflection angle, but our model makes no explicit claims regarding the generative mechanism. Our hypothesis is compatible with both explicit Newtonian simulation, as in the intuitive-physics-engine approach (Battaglia et al., 2013), and with physically consistent predictions produced by visual heuristics, which avoid explicit representation of Newton’s laws (Gilden & Proffitt, 1994; Runeson, 1977; Siegler, Bardy, & Warren, 2010; Todd & Warren, 1982).

Statement of Relevance

Our everyday manipulation and perception of object motion tends to be quite accurate, suggesting that the human nervous system may have accurately internalized physical regularities. In a direct test of this hypothesis, we looked for evidence of a bias in visual-motion perception in which perceived object trajectories after a collision are biased toward the direction predicted by Newtonian mechanics. Our findings suggest that the visual system generates physically accurate predictions and integrates those predictions with available sensory information. When sensory information is inconsistent with the Newtonian prediction, this internally generated predictive signal biases perception in the direction of a physically consistent event. Additionally, the influence of this prediction is inversely proportional to the noise in the available sensory information. These results are consistent with an internal model that simulates the collision event according to Newton’s laws or a simple heuristic that generates comparable predictions.

To test our hypothesis, we presented observers with short videos of billiard-ball collisions and asked them to report the direction along which they saw the launched ball travel after the collision. Related work on perceived causality has provided a rich basis for the design of collision displays (Badler, Lefevre, & Missal, 2010; Michotte, 1946/1963; Scholl & Tremoulet, 2000; White, 2012) and suggests that the impression of causality may be important to the generation of internal predictions. By independently controlling the Newtonian prediction (given by the relative positions of the balls at impact) and the displayed postcollision motion, we directly measured the extent to which visual perception of object motion integrates Newtonian predictions.

Maximum-Likelihood Model of Motion Perception in Billiard-Ball Collisions

In our model, we assumed that the visual system infers the target ball’s deflection angle θ on the basis of two sources of information: the sensory estimate θ_s and a Newtonian prediction θ_N (Fig. 1). Optimal inference involves computing the most likely value of θ, which corresponds with maximizing the likelihood L(θ _s , θ_N | θ). We assumed that θ_s and θ_N are (a) normally distributed, (b) conditionally independent given θ, and (c) unbiased with respect to the displayed movement angle and Newtonian ground truth. Under these assumptions, the maximum-likelihood estimate of the deflection angle θ^*, which we take as the perceived (reported) deflection angle, can be computed exactly as a reliability-weighted linear combination (Clark & Yuille, 1990):

Fig. 1.

Maximum-likelihood model. The cue ball (white) strikes a glancing blow on one side of the target ball (black), which is launched away. In this example, the displayed deflection angle of the target ball is to the left (blue line), inconsistent with the Newtonian prediction for this collision (red line). These two independent sources of information (sensory estimate θ_s and Newtonian prediction θ_N) are combined into an optimal estimate of the target ball’s deflection angle θ* (purple line), which is the value of θ that maximizes the joint likelihood L(θ_s, θ_N | θ), indicated by the purple distribution. The joint likelihood is the product of the two component likelihoods L(θ_s | θ), indicated by the blue distribution, and L(θ_N | θ), indicated by the red distribution. We take θ* to be the perceived (reported) postcollision motion of the target ball. In this model, θ* can be computed directly as a reliability-weighted linear combination of θ_N and θ_s (see Equation 1).

θ^{*} = w_{N} θ_{N} + w_{s} θ_{s}

(1a)

w_{N} = σ_{N}^{- 2} / (σ_{N}^{- 2} + σ_{s}^{- 2})

(1b)

w_{s} = 1 - w_{N}

(1c)

This maximum-likelihood model produces the key predictions of the present experiments: When the displayed deflection angle θ_s is inconsistent with the Newtonian prediction θ_N, the resulting percept will be biased toward the Newtonian prediction (Equation 1a), and the magnitude of this bias will depend on the sensory noise σ_s (Equation 1b).

Experiment 1

Our main experiment tested whether precollision information, which determines the Newtonian prediction, would bias perception of the target ball’s postcollision trajectory. On each trial (Fig. 2), participants viewed a short 3D simulation in which a cue ball collided on the left or right side of a target ball, which then moved along one of 13 possible deflection angles. Thus, in the context of our model, the precollision information determined the Newtonian prediction, whereas the postcollision motion determined the sensory estimate. To modulate the reliability of the sensory estimate, we varied the duration of the postcollision trajectory between 40 ms and 80 ms. Our model predicted that the influence of the Newtonian prediction on perception would be stronger in the 40-ms condition. After the postcollision duration elapsed, both balls disappeared, and a response line appeared in the same plane in which the balls had moved. Participants adjusted the orientation of the response line to report the perceived deflection angle of the target ball.

Fig. 2.

Collision displays and task design. The experimental apparatus is shown in (a). The stereoscopic image on the monitor was reflected so that the 3D scene appeared beyond the mirror. An aerial view of a single-trial time course is illustrated in (b). Two spheres were presented just below eye level, separated in depth by 100 mm. The cue ball moved toward the target ball along a leftward or rightward trajectory (aiming angle). The balls collided 130 ms after movement onset. The cue ball stopped in place, and the target ball began moving in one of 13 directions (delineated in c). Postcollision movement was displayed for either 40 ms or 80 ms, and then both balls disappeared. Following a 130-ms delay, a 3D response line was presented. Participants adjusted the orientation of the line to indicate the direction in which they saw the target ball move after the collision. The 26 simulated collision displays presented in Experiment 1 (2 aiming angles × 13 deflection angles) are shown in (c). For each aiming angle, the Newtonian prediction is shown in a corresponding color (red or blue). The other 12 deflection angles deviated from the Newtonian prediction to varying degrees.

Method

Participants

Eleven participants were recruited for Experiment 1. This sample size was determined by power analysis of a pilot study. All participants were right-handed, had normal or corrected-to-normal vision, and were paid $12 per hour as compensation. Written informed consent was obtained from all participants in accordance with the Declaration of Helsinki and following protocol approved by the Brown University Institutional Review Board.

Apparatus

With their head position fixed by a chin rest, participants viewed stereoscopic renderings of 3D objects by looking into a half-silvered mirror arranged at a 45° angle to a 19-in. CRT monitor directly to the left of the mirror. The mirror reflected the image displayed on the monitor so that the rendered objects appeared to be floating in space beyond the mirror (Fig. 2a). The distance from the eyes to the reflected image of the monitor screen was 400 mm. Stereoscopic presentation of the simulated 3D scene was achieved using a frame-interlacing technique and a pair of 3D Vision 2 wireless glasses (NVIDIA, Santa Clara, CA).

Stimuli

Participants viewed stereoscopic 3D simulations of billiard-ball collisions (Fig. 2b). A cue ball and a target ball were rendered as red spheres, 32 mm in diameter. Diffuse shading cues to 3D shape were created by including an overhead directional light source in the simulations. Additionally, the objects’ apparent sizes decreased linearly with their distance from the participant, in accordance with accurate perspective projection. In a right-handed coordinate system centered at the cyclopean eye, the initial positions of the cue ball and target ball were, respectively, 0, −75, and −300 mm and 0, −75, and −400 mm. In other words, the balls appeared in the same horizontal plane, with the target ball initially 100 mm beyond the cue ball. The cue ball moved at a constant speed (0.52 meters per second) toward the target ball over 130 ms and collided on either the right side or the left side. Because the duration of the precollision cue-ball trajectory was constant across trials, we also assume that the reliability of the Newtonian prediction was constant.

The cue ball’s aiming angle (i.e., its angular deviation from straight ahead) was either −10° or +10°. Under Newtonian mechanics, these two aiming angles lead to collisions that would cause the target ball to be deflected at angles of +23° and −23° (Fig. 2c, red and blue traces) and the cue ball to be deflected at −67° and +67°, respectively. However, in our displays, the cue ball stopped in place on collision, and the target ball began moving along one of 13 possible displayed deflection angles, independently of the cue ball’s aiming angle. The cue ball stopped in place in order to remove a potential confound: Across the 13 target-ball deflection angles, there would have been different relative motions of the cue ball and target ball. We confirmed in pilot testing that stopping the cue ball did not appear unnatural or jarring. The postcollision movement of the target ball was displayed for either 40 ms or 80 ms, and then both balls disappeared. The target ball speed was 0.52 meters per second, identical to the cue ball speed. As a result, in trials with an 80-ms postcollision duration, the target ball traveled farther than in trials with a 40-ms duration. Because postcollision duration and trajectory length are confounded given a constant velocity, we cannot determine how much each of these variables contributed to the sensory noise; however, for the purposes of testing our model, this distinction is not necessary.

Following a 130-ms delay, a response line was displayed on the screen. The slight delay was introduced to mitigate apparent motion between the target ball and response line. The response line was a 20-mm line segment lying flat in the horizontal plane, 75 mm below eye level, with its near end anchored at the initial location of the target ball. The location of the far end depended on the current orientation of the response line. Given that stereo, perspective, and motion cues were available, there was sufficient information to perceive the response line as pointing outward in depth, in the same plane where the balls moved. No participant expressed any confusion about the orientation of the response line.

Procedure

After viewing the simulated collision event, participants used key presses to adjust the orientation of the response line to match the previously perceived trajectory of the target ball. The line could be adjusted from −90° to 90° (0° was straight ahead). The initial line orientation was randomly selected from this range. The factorial combination of two aiming angles (i.e., Newtonian predictions), 13 displayed deflection angles, and two postcollision durations yielded 52 distinct trial types. These stimuli were presented in pseudorandom order in five 52-trial blocks for a total of 260 trials.

Results

As depicted in Figure 3, judgments were accurate when the displayed deflection angle matched the Newtonian prediction (open circles) but became increasingly biased toward the Newtonian prediction as the difference between these two quantities grew in either direction.

Fig. 3.

Experiment 1 results. Perceived deflection angle is shown as a function of displayed deflection angle. In (a), results are color coded according to the Newtonian prediction, separately for the 40-ms and 80-ms postcollision durations. If perception were determined entirely by the displayed motion, the data would lie along the dashed unity line. The influence of the Newtonian prediction is indicated by the bend toward the horizontal red and blue dashed lines. The unity line and the horizontal red and blue lines intersect at the points where the displayed deflection angle is consistent with the Newtonian prediction; the observed data points for these veridical collision stimuli are filled in white. Error bars represent ±1 SEM. In (b), data from the leftward (blue) and rightward (red) Newtonian-prediction conditions were combined (separately for each postcollision duration). The response (y-axis) and deflection angle (x-axis) values for the leftward Newtonian-prediction condition (aiming angle = +10°) were multiplied by −1, which aligned the Newtonian predictions for both conditions at +23°, doubling the number of trials per depicted data point.

Note that a simple bias toward the mean deflection angle (0°) does not explain the pattern of results, as the bias is centered on the Newtonian prediction (23°). In particular, notice that responses for displayed deflection angles between 0° and 23° are pulled toward 23°. A three-way analysis of variance (ANOVA) with the factors displayed deflection angle, Newtonian prediction, and postcollision duration revealed significant main effects of displayed deflection angle, F(12, 120) = 131.00, p < .001, η_p² = .93, 90% confidence interval (CI) = [.90, .94], and Newtonian prediction, F(1, 10) = 121.72, p < .001, η_p² = .92, 90% CI = [.78, .95], as well as significant interactions of displayed deflection angle and postcollision duration, F(12, 120) = 21.72, p < .001, η_p² = .68, 90% CI = [.57, .71]; Newtonian prediction and postcollision duration, F(1, 10) = 6.089, p = .033, η_p² = .38, 90% CI = [.019, .61]; and displayed deflection angle and Newtonian prediction, F(12, 120) = 9.61, p < .001, η_p² = .49, 90% CI = [.33, .53].

Discussion

In light of our model, the two interactions with postcollision duration can be understood as complementary effects (see Equations 1b and 1c): The effect of sensory information (displayed deflection angle) becomes weaker at shorter postcollision durations because of increased sensory noise (i.e., less time to accumulate sensory evidence), thereby increasing the influence of the internal Newtonian prediction. Alongside the main effect of Newtonian prediction, these two interactions support our model. The remaining interaction of displayed deflection angle with Newtonian prediction captures the fact that the effect of Newtonian prediction is stronger when the displayed deflection angle is more oblique, a finding that will be addressed in Experiment 2.

Experiment 2

Experiment 1 qualitatively demonstrated that perceived motion in collision events reflects a reliability-weighted combination of a sensory estimate (the displayed deflection angle) and a Newtonian prediction based on precollision information. As mentioned above, the magnitude of the Newtonian bias was found to increase as the displayed deflection angle became more oblique. In our model, the influence of the Newtonian prediction is inversely related to the reliability of the sensory estimate (Equation 1). Thus, we predicted that purely sensory estimates of target-ball movement would be noisier at more oblique deflection angles. Experiment 2 tested this prediction by measuring the noise in participants’ judgments under conditions designed to minimize the influence of Newtonian predictions, thereby isolating the sensory estimate. To do this, we removed the dynamic causal element that created the impression of causality in the collision displays of Experiment 1 (Michotte, 1946/1963): Instead of the cue ball moving toward the target ball, the balls were initially rendered side by side, and after a short delay, the target ball moved away on its own.

Method

Eleven new participants were recruited for Experiment 2. The design and procedure were identical to those of Experiment 1, except that we altered the displays to remove the collision event. Instead of the cue ball traveling toward the target ball, the two balls were initially shown sitting side by side, in the positions where they appeared at the moment of contact in Experiment 1. Specifically, the cue ball was displayed at ±12, −75, and −370 mm in a right-handed coordinate system centered at the cyclopean eye; the positive or negative sign of the x-coordinate corresponds to the two aiming angles in Experiment 1. After a 1,200-ms period in which both objects were stationary, intended to suppress any percept of causality, the target ball began moving, apparently self-propelled, along one of the 13 deflection angles.

Results

When the collision event was removed, perception of the target ball’s displayed movement direction was more accurate than in Experiment 1 (cf. Figs. 4a and 4c), t(19.96) = 2.44, p = .024, Cohen’s d = 1.04, consistent with a predictive mechanism that is gated by the impression of causality. A three-way ANOVA similar to the analysis in Experiment 1 (Displayed Deflection Angle × Cue-Ball Position × Postcollision Duration) revealed significant main effects of displayed deflection angle, F(12, 120) = 210.57, p < .001, η_p² = .95, 90% CI = [.94, .96], and cue-ball position, F(1, 10) = 51.36, p < .001, η_p² = .84, 90% CI = [.57, .90], as well as significant interactions of displayed deflection angle and cue-ball position, F(12, 120) = 2.40, p = .0080, η_p² = .19, 90% CI = [.029, .22], and displayed deflection angle and postcollision duration, F(12, 120) = 4.82, p < .001, η_p² = .33, 90% CI = [.15, .37].

Fig. 4.

Experiment 2 results and Experiment 1 model fit. Data from the leftward (negative values) and rightward (positive values) Newtonian-prediction conditions were combined (separately for each postcollision duration). The response (y-axis) and deflection angle (x-axis) values for the leftward Newtonian-prediction condition (aiming angle = +10°) were multiplied by −1, which aligned the Newtonian predictions for both conditions at +23°, doubling the number of trials per depicted data point. Perceived deflection angle (a) is shown as a function of displayed deflection angle and postcollision duration. Average within-subjects standard deviations of responses (b) are shown as a function of displayed deflection angle and postcollision duration. Solid lines indicate the point estimates of σ_s used to fit the model in (c). The dashed horizontal line shows the value of σ_N obtained from the model fit. Results of model fit to Experiment 1 data are shown in (c), using the same axis values as in (a) and the empirical estimates of sensory noise displayed in (b). Model predictions are represented by the curved lines, with the relevant empirical data from Experiment 1 replotted from Fig. 3b. Error bands indicate ±1 SEM. For each of our 26 displays, the empirically measured bias toward the Newtonian prediction in Experiment 1, θ* − θ_s, is plotted (d) against the bias predicted by our model, w_N(θ_N − θ_s). In all panels, the dashed diagonal line represents unity, and error bars represent ±1 SEM.

By removing the precollision movement of the cue ball, we aimed to fully suppress any internal prediction, forcing participants to rely entirely on sensory information so we could estimate the sensory noise alone. However, Figure 4a depicts a slightly sigmoidal curvature of the response data, suggesting that a small bias may have persisted in Experiment 2. In a control experiment (Experiment 4; see the Supplemental Material available online), we ruled out the possibility that this bias was attributable to the sensory estimate of 3D motion (Welchman, Lam, & Bülthoff, 2008; Welchman, Tuck, & Harris, 2004), which would conflict with one of our model assumptions. In this control experiment, we removed the cue ball entirely to eliminate any impression of a launching event and found that judgments were completely unbiased. Two key differences with earlier studies showing biases were that (a) our displays involved trajectories moving away from the participant in depth below eye level, and (b) our response line was rendered in 3D in the same spatial-coordinate frame as the observed motion. Thus, it seems that the bias in Experiment 2 was caused by the placement of the static cue ball next to the target ball, which is supported by the main effect of cue-ball position in the ANOVA results. However, the interaction of cue-ball position and postcollision duration was not significant in Experiment 2, F(1, 10) = 0.21, p = .66, η_p² = .021, 90% CI = [.00, .26]), despite the fact that response variability was greatly increased at shorter display times (Fig. 4b).

We turned next to these standard-deviation data, finding a V-shaped pattern (depicted in Fig. 4b) showing that sensory noise increased approximately linearly as the displayed deflection angle became more oblique. Further, as mentioned above, it was greater for the 40-ms postcollision duration than for the 80-ms postcollision duration. A linear model with the continuous predictor eccentricity (i.e., unsigned displayed deflection angle) and the categorical predictor postcollision duration showed significant effects of both factors—eccentricity: F(1, 10) = 24.51, p < .001, η_p² = .86, 90% CI = [.81, .88]; postcollision duration: F(1, 10) = 28.67, p < .001, η_p² = .90, 90% CI = [.73, .94] (solid lines in Fig. 4b). Both of these effects are qualitatively consistent with our model’s explanation of the pattern of Newtonian biases observed in Experiment 1: The influence of the Newtonian prediction was proportional to sensory noise, with the weakest biases observed when the deflection angle was near 0°, where purely sensory judgments are most reliable.

Discussion

The slight bias observed in Experiment 2 suggests that participants occasionally perceived the no-collision displays as causal and integrated a Newtonian prediction into perception (see also Experiment 4 in the Supplemental Material). However, the relative reliability of these predictions was clearly reduced to such an extent that their contribution to the response standard deviations would be negligible. Therefore, the standard-deviation data obtained in Experiment 2 still provide a reasonably accurate empirical estimate of sensory noise.

Modeling

Next, we used the predicted sensory variances from the linear model described above (solid lines in Fig. 4b) as estimates of σ_s in our maximum-likelihood model. This enabled us to fit the model to the average data of Experiment 1 using ordinary least squares with a single free parameter, the variance of the Newtonian prediction σ_N².

Method

We fitted our model to the average data from Experiment 1 as follows. First, as described in the Experiment 2 results, we calculated the average within-subjects standard deviations of the Experiment 2 responses and fitted these data with a linear model using unsigned displayed deflection angle (i.e., eccentricity) and display duration as predictors (Fig. 4b). This linear model provided an estimate of σ_s, the standard deviation of the sensory information, for each of the 26 displays used in Experiment 1. We used these estimates in conjunction with the average Experiment 1 data to fit σ_N², the variance of the Newtonian prediction, by ordinary least squares on the following linear model, obtained by rearranging Equation 1:

σ_{s}^{2} (θ_{N} - θ^{*}) = σ_{N}^{2} (θ^{*} - θ_{s}) .

(2)

Results

The fit to the Experiment 1 data involved only one free parameter, σ_N², the variance of the Newtonian prediction (mean σ_N = 12.7°, SEM = 3°). As shown in Figure 4c, the model fit quantitatively captures the pattern of increasing bias at more oblique deflection angles and at the shorter postcollision duration. In Figure 4d, we demonstrate the excellent fit of the model by plotting the empirically observed biases toward the Newtonian prediction θ^* − θ_s against the biases predicted by the model given the estimated Newtonian weight w_N(θ _N − θ_s). Although there are some minor deviations for some stimuli, it is clear that this relatively simple cue-integration model, with the noise in the Newtonian prediction as its only free parameter, provides a very close fit to the observed biases across 26 different stimulus displays.

Experiment 3

In Experiment 3, we aimed to verify that the Newtonian bias is truly perceptual in origin. Alternatively, the observed biases could be the result of (a) response corrections based on explicit knowledge or (b) schema-driven revisions introduced during maintenance in short-term memory. To minimize the possibility of these alternatives, we designed a variant of the task in which participants judged whether the target ball’s trajectory was divergent or convergent with the trajectory of an additional probe ball that moved in tandem with the target ball. We expected explicit corrections to be minimized because participants were focused on parallelism, a property that is unrelated to the collision. Moreover, by using an immediate-response paradigm, we minimized effects of short-term memory while maximizing perceptual contributions.

Method

The design and procedure were similar to those of Experiment 1, with a few key alterations described below.

Participants

Twelve new participants were recruited for Experiment 3.

Stimuli

In addition to the cue and target balls, an identical third ball (the probe ball) was displayed at an initial position 40 mm to the right or the left of the target ball. Simultaneous with the target-ball launch, the probe ball also moved along a reference trajectory. Because of the lengthier psychophysical procedure used to determine the biases in this experiment (see the Procedure section), we fixed the postcollision display duration at 60 ms and presented only three displayed deflection angles for each aiming angle: −23°, 0°, and 23°. Even in this limited set, one displayed trajectory was consistent with the Newtonian prediction, one deviated with low sensory noise, and one deviated with high sensory noise, which was sufficient to test the key predictions of our model. In pilot testing, we found that 40 ms of postcollision motion was too brief to form a reliable judgment of parallelism, so we slightly increased the duration to 60 ms.

Procedure

The movement angle of the probe ball (reference trajectory) was determined according to a psychometric procedure to establish the angle that appeared parallel to the displayed target-ball trajectories. In each trial, participants indicated whether the probe ball and target ball were converging or diverging by pressing one of two keys. This response was used to adjust the reference-movement angle of the probe ball according to a one-up, one-down staircase procedure that terminated after 16 inversions. Factorial combination of two aiming angles (i.e., Newtonian predictions), three displayed deflection angles, and two probe-ball positions (left or right of the visual field) yielded 12 distinct trial types. One staircase was completed for each of 12 trial types. To obtain an estimate of the point of subjective parallelism, we averaged the probe-ball movement angles at the final four reversals.

Results

Perception of the displayed deflection angle of the target ball was highly accurate when it was consistent with the Newtonian prediction and also when it was straight ahead (Fig. 5). However, a two-way ANOVA revealed significant main effects of Newtonian prediction, F(1, 11) = 20.68, p < .001, η_p² = .65, 90% CI = [.27, .78], and displayed deflection angle, F(2, 22) = 488.32, p < .001, η_p² = .977, 90% CI = [.96, .98]. A follow-up t test revealed that the main effect of Newtonian prediction was driven by the Newtonian bias that emerged when displayed deflection angles were highly oblique and in the opposite direction of the Newtonian prediction. For these displays (i.e., a −23° displayed deflection angle with a 23° Newtonian prediction, and vice versa), we found a significant Newtonian bias, 95% CI = [0.60°, 5.60°], t(11) = 2.73, p = .019. However, this bias was weaker than in the comparable displays from Experiment 1, 95% CI = [−∞, −4.38°], t(13.64) = −5.31, p < .001.

Fig. 5.

Design and results of Experiment 3. We included a probe ball (gray) to the right or left of the target ball (black), which also began moving after the collision event (a). Here, the displayed deflection angle of the target ball (blue line) is inconsistent with the Newtonian prediction (red line). The displayed movement angle of the probe ball was adjusted according to a psychometric procedure to converge on the point at which the trajectories appeared parallel (gray line). The graph (b) shows probe angle at perceived parallel as a function of displayed deflection angle, separately for Experiment 3 and a subset of the Experiment 1 results (average of 40-ms and 80-ms conditions). In (b), data from the leftward (negative values) and rightward (positive values) Newtonian-prediction conditions were combined (separately for each experiment). The response (y-axis) and deflection angle (x-axis) values for the leftward Newtonian-prediction condition (aiming angle = +10°) were multiplied by −1, which aligned the Newtonian predictions for both conditions at +23°, doubling the number of trials per depicted data point.

Discussion

Overall, Experiment 3 reinforced our claim that the Newtonian bias is perceptual in origin. Although the bias was weaker than in Experiment 1, this is actually consistent with our model because the parallelism displays involved relative rather than absolute judgments of motion direction. Although absolute judgments of angular motion are typically inaccurate and noisy (Welchman et al., 2004), perceptual acuity is often greatly increased for relative spatial judgments (Lappin & Craft, 2000). In the Experiment 3 displays, we therefore expected the presence of two spatially related object motions to reduce the noise in sensory estimates of each component motion. Thus, it is encouraging that we obtained a clear Newtonian bias despite the fact that our method to obtain truly perceptual responses also inadvertently reduced sensory noise.

General Discussion

We have described evidence from three experiments testing whether 3D visual perception of target motion following a billiard-ball collision is biased toward the trajectory that is consistent with Newtonian mechanics. Additionally, the Supplemental Material contains two control experiments that can be summarized as follows. First, as mentioned previously, Experiment 2 raised the possibility that sensory estimates of 3D motion are biased even in the absence of collisions; however, Experiment 4 confirmed that this was not the case for our displays. Second, we noticed a potential confound in the design of Experiment 1: The greatest biases were observed for displays in which target-ball motion was more occluded by the cue ball. However, we obtained an identical pattern of results in Experiment 5, in which a transparent cue ball made the target-ball trajectory highly visible in all directions.

Overall, these results constitute strong evidence of a visual mechanism that generates internal predictions of object motion during perceived causal interactions, such as collision events, and integrates these predictions with incoming sensory inputs as if they were an additional source of visual information. In particular, we showed that our results are neatly explained by a well-established maximum-likelihood model. This model integrates multiple sensory signals on the basis of their relative reliabilities in order to resolve sensory ambiguities and enable more precise perceptual judgments (Jacobs & Fine, 1999; Knill & Richards, 1996). This type of reliability-weighted cue-combination model was highly effective in accounting for our empirical data, indicating that the observed perceptual phenomenon is consistent with a normative inference. The findings of the present study are quite different from past investigations of reliability-weighted cue combination because they suggest that internal predictions are treated as an additional source of information, as if they were noisy observations of the actual event. Thus, our findings extend the current conception of perceptual-cue combination to include concurrent visual signals as well as predictive signals.

A key limitation of the present work is that it does not show exactly how Newtonian predictive signals are generated. As a result, we remain agnostic about the nature of the predictive mechanism. In particular, the observed phenomenon does not necessitate a detailed internal model of Newtonian mechanics (e.g., Battaglia et al., 2013) but instead could be the result of learned contingencies between perceived events in a Newtonian environment (Hubbard, 2020; Shepard & Chipman, 1970). Setting aside this larger debate, some of our data suggest that the predictive mechanism is activated only when the collision event is perceived as causal (Michotte, 1946/1963). Although we were initially surprised to find a small Newtonian bias when precollision motion of the cue ball was removed in Experiment 2, this finding is consistent with causality-gated predictions, as these displays may not have completely extinguished impressions of causality.

Last, it should be noted that although we describe a bias in the laboratory conditions of our experiments, the proposed mechanism would have clear adaptive value in the real world, as noted in previous discussions of internalization (Hubbard, 2005; Runeson & Vedeler, 1993). Objects can change course dramatically after a collision, and the collision itself could introduce additional distracting sensory stimuli that might reduce the fidelity of motion signals from a target object, especially in the first 100 ms. Incorporating an internal prediction about what is happening during that period directly into perception would thus be extremely useful in maintaining perceptual continuity and in limiting the effect of potential distractors. The present study is, to our knowledge, the first to demonstrate that these predictions can directly modulate perceptual experience.

Supplemental Material

sj-docx-1-pss-10.1177_0956797620966785 – Supplemental material for Newtonian Predictions Are Integrated With Sensory Information in 3D Motion Perception

Supplemental material, sj-docx-1-pss-10.1177_0956797620966785 for Newtonian Predictions Are Integrated With Sensory Information in 3D Motion Perception by Abdul-Rahim Deeb, Evan Cesanek and Fulvio Domini in Psychological Science

Footnotes

Transparency

Action Editor: Marc J. Buehner

Editor: Patricia J. Bauer

Author Contributions

A.-R. Deeb and E. Cesanek contributed equally to this work. A.-R. Deeb developed the study concept in collaboration with E. Cesanek and F. Domini. All authors contributed to the study design. Testing and data collection were performed by A.-R. Deeb. Data were analyzed and interpreted by A.-R. Deeb under the supervision of E. Cesanek and F. Domini. The model was developed by F. Domini and computationally executed by E. Cesanek. A.-R. Deeb and E. Cesanek drafted the manuscript, and F. Domini provided critical revisions. All authors approved the final version of the manuscript for submission.

Supplemental Material

Additional supporting information can be found at

References

Badler

Lefevre

Missal

(2010). Causality attribution biases oculomotor responses. The Journal of Neuroscience, 30, 10517–10525. doi:10.1523/JNEUROSCI.1733-10.2010

Battaglia

P. W.

Hamrick

J. B.

Tenenbaum

J. B.

(2013). Simulation as an engine of physical scene understanding. Proceedings of the National Academy of Sciences, USA, 110, 18327–18332. doi:10.1073/pnas.1306572110

Clark

J. J.

Yuille

A. L.

(1990). Data fusion for sensory information processing systems. Norwell, MA: Kluwer. doi:10.1007/978-1-4757-2076-1

Firestone

Keil

F. C.

(2016). Seeing the tipping point: Balance perception and visual shape. Journal of Experimental Psychology: General, 145, 872–881. doi:10.1037/xge0000151

Firestone

Scholl

B. J.

(2014). “Please tap the shape, anywhere you like”: Shape skeletons in human vision revealed by an exceedingly simple measure. Psychological Science, 25, 377–386. doi:10.1177/0956797613507584

Freyd

J. J.

(1983). The mental representation of movement when static stimuli are viewed. Perception & Psychophysics, 33, 575–581. doi:10.3758/BF03202940

Freyd

J. J.

Pantzer

T. M.

Cheng

J. L.

(1988). Representing statics as forces in equilibrium. Journal of Experimental Psychology: General, 117, 395–407. doi:10.1037/0096-3445.117.4.395

Gilden

D. L.

Proffitt

D. R.

(1989). Understanding collision dynamics. Journal of Experimental Psychology: Human Perception and Performance, 15, 372–383. doi:10.1037/0096-1523.15.2.372

Gilden

D. L.

Proffitt

D. R.

(1994). Heuristic judgment of mass ratio in two-body collisions. Perception & Psychophysics, 56, 708–720. doi:10.3758/BF03208364

10.

Hecht

Bertamini

(2000). Understanding projectile acceleration. Journal of Experimental Psychology: Human Perception and Performance, 26, 730–746. doi:10.1037//0096-1523.26.2.730

11.

Hubbard

T. L.

(2005). Representational momentum and related displacements in spatial memory: A review of the findings. Psychonomic Bulletin & Review, 12, 822–851. doi:10.3758/BF03196775

12.

Hubbard

T. L.

(2013). Launching, entraining, and representational momentum: Evidence consistent with an impetus heuristic in perception of causality. Axiomathes, 23, 633–643. doi:10.1007/s10516-012-9186-z

13.

Hubbard

T. L.

(2019). Momentum-like effects and the dynamics of perception, cognition, and action. Attention, Perception, and Psychophysics, 81, 2155–2170. doi:10.3758/s13414-019-01770-z

14.

Hubbard

T. L.

(2020). Representational gravity: Empirical findings and theoretical implications. Psychonomic Bulletin and Review, 27, 36–55. doi:10.3758/s13423-019-01660-3

15.

Jacobs

R. A.

Fine

(1999). Experience-dependent integration of texture and motion cues to depth. Vision Research, 39, 4062–4075. doi:10.1016/S0042-6989(99)00120-0

16.

Knill

D. C.

Richards

(Eds.). (1996). Perception as Bayesian inference. Cambridge, England: Cambridge University Press. doi:10.1017/cbo9780511984037

17.

Lappin

J. S.

Craft

W. D.

(2000). Foundations of spatial vision: From retinal images to perceived shapes. Psychological Review, 107, 6–38. doi:10.1037/0033-295X.107.1.6

18.

McCloskey

Caramazza

Green

(1980). Curvilinear motion in the absence of external forces: Naïve beliefs about the motion of objects. Science, 210, 1139–1141. doi:10.1126/science.210.4474.1139

19.

McIntyre

Zago

Berthoz

Lacquaniti

(2001). Does the brain model Newton’s laws? Nature Neuroscience, 4, 693–694. doi:10.1038/89477

20.

Michotte

(1963). The perception of causality ( Miles

T. R.

Miles

, Trans.). New York, NY: Basic Books. doi:10.4324/9781315519050. (Original work published 1946)

21.

Runeson

(1977). On the possibility of “smart” perceptual mechanisms. Scandinavian Journal of Psychology, 18, 172–179. doi:10.1111/j.1467-9450.1977.tb00274.x

22.

Runeson

Vedeler

(1993). The indispensability of precollision kinematics in the visual perception of relative mass. Perception & Psychophysics, 53, 617–632. doi:10.3758/BF03211738

23.

Sanborn

A. N.

Mansinghka

V. K.

Griffiths

T. L.

(2013). Reconciling intuitive physics and Newtonian mechanics for colliding objects. Psychological Review, 120, 411–437. doi:10.1037/a0031912

24.

Scholl

B. J.

Tremoulet

P. D.

(2000). Perceptual causality and animacy. Trends in Cognitive Sciences, 4, 299–309. doi:10.1016/S1364-6613(00)01506-0

25.

Shepard

R. N.

(1984). Ecological constraints on internal representation: Resonant kinematics of perceiving, imagining, thinking, and dreaming. Psychological Review, 91, 417–447. doi:10.1037/0033-295X.91.4.417

26.

Shepard

R. N.

(1994). Perceptual cognitive universals as reflections of the world. Psychonomic Bulletin & Review, 1, 2–28.

27.

Shepard

R. N.

Chipman

(1970). Second-order isomorphism of internal representations: Shapes of states. Cognitive Psychology, 1, 1–17. doi:10.1016/0010-0285(70)90002-2

28.

Siegler

I. A.

Bardy

B. G.

Warren

W. H.

(2010). Passive vs. active control of rhythmic ball bouncing: The role of visual information. Journal of Experimental Psychology: Human Perception and Performance, 36, 729–750. doi:10.1037/a0016462

29.

Todd

J. T.

Warren

W. H.

(1982). Visual perception of relative mass in dynamic events. Perception, 11, 325–335. doi:10.1068/p110325

30.

Welchman

A. E.

Lam

J. M.

Bülthoff

H. H.

(2008). Bayesian motion estimation accounts for a surprising bias in 3D vision. Proceedings of the National Academy of Sciences, USA, 105, 12087–12092. doi:10.1073/pnas.0804378105

31.

Welchman

A. E.

Tuck

V. L.

Harris

J. M.

(2004). Human observers are biased in judging the angular approach of a projectile. Vision Research, 44, 2027–2042. doi:10.1016/j.visres.2004.03.014

32.

White

P. A.

(2012). Visual impressions of causality: Effects of manipulating the direction of the target object’s motion in a collision event. Visual Cognition, 20, 121–142. doi:10.1080/13506285.2011.653418

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.53 MB