The effect of modality and order presentation of emotional stimuli on time perception

Abstract

Despite human accuracy in perceiving time, many factors can modulate the subjective experience of time. For example, it is widely reported that emotion can expand or shrink our perception of time and that temporal intervals are perceived as longer when marked by auditory stimuli than by visual stimuli. In the present study, we aimed at investigating whether the influence of emotion on time perception can be altered by the order in which emotional stimuli are presented and the sensory modality in which they are presented. Participants were asked to complete a time bisection task in which emotional stimuli were presented either acoustically or visually, and either before or after interval to be estimated. We observed a main effect of modality (longer perceived duration and lower variability in the auditory than in the visual modality) as well as a main effect of emotion (temporal overestimation for negative stimuli compared to neutral). Importantly, the effects of modality and emotion interacted with the order of presentation of the emotional stimuli. In the visual condition, when emotional stimuli were presented after the temporal intervals, participants overestimated time, but no differences between negative and neutral stimuli were observed when emotional stimuli were presented first. In the auditory condition, no significant effect of emotion on perceived duration was found. Results suggest that negative emotions affect our perception of durations acting at the decision-making stage rather than at the pacemaker one. No effect on time perception was observed for emotional auditory stimuli.

Keywords

Time perception temporal information processing time bisection task emotions sensory modalities

Introduction

The ability to accurately perceive time is invaluable to humans. It is important in everyday activities to understand when in time and for how long an event occurred, or how soon in the future an event might occur. These different aspects of time allow us to move through the world and perform necessary actions on time. It is necessary, then, that we understand the conditions under which time perception can be altered and the inaccuracies that result.

One of the most influential models of time, the pacemaker-accumulator model of time (Gibbon et al., 1984; Treisman, 1963), posits that the subjective experience of time depends on the number of pulses stored in the accumulator. More specifically, the pacemaker-accumulator model includes three stages: the clock, memory, and decision stages. The clock stage includes a pacemaker that produces pulses; the onset of a to-be-timed event closes a switch-like mechanism and starts the period of accumulation of pulses. The entering of pulses in the accumulator stops at the offset of the to-be-timed event. The memory stage includes the storing system that contains a long-term representation of the number of pulses accumulated across prior temporal experiences. The final stage is related to decision-making, in which the number of pulses accumulated during a given trial is compared with those stored in reference memory.

Despite being very accurate in estimating time, we have all experienced a mismatch between the objective time, measured by our watches, and the subjective experience of it. Indeed, subjective time expands or shrinks depending on different factors (Grondin, 2010, 2020; Matthews, 2011), such as body temperature (Hancock, 1993; Mioni et al., 2021; Wearden & Penton-Voak, 1995), psychological state of anxiety or depression (Mioni et al., 2016), or drug consumption (Wittmann et al., 2007; Zhang et al., 2019). The present work focuses on two factors, emotions (negative or neutral emotional stimuli) and sensory modality (auditory vs. visual). These two factors, effects of emotions on time (Lake, 2016; Lake et al., 2016) and on effects of modality on time (Azari et al., 2020; Grondin, 2014; Van Wassenhove et al., 2008), have been largely studied in different lines of research and have progressed almost independently, but were never combined within the same experimental procedure to have a complete understanding of how visual or auditory emotional stimuli influence subjective time.

Effects of emotional stimuli on time perception

The literature that investigates the effects of emotional stimuli on subjective time is rich, and the results show that, depending on the stimulus employed, we can either overestimate or underestimate time (Lake, 2016; Lake et al., 2016). The majority of previous studies have hypothesised that the temporal distortion induced by emotional stimuli acted at the level of the pacemaker: an increase in the level of arousal, induced by the presentation of emotional stimuli, increases the speed of the pacemaker. For a given amount of time, if the pacemaker runs faster, more pulses are collected in the accumulator, and stimulus duration is judged to be longer. It has been found, indeed, that emotional pictures generating high arousal led to greater overestimation of time than emotional stimuli generating less arousal (Angrilli et al., 1997; Droit-Volet et al., 2004; Droit-Volet & Gil, 2009; Gil & Droit-Volet, 2011; Grondin et al., 2015; Lee et al., 2011; Noulhiane et al., 2007; Tipples, 2008). However, not all studies reported temporal overestimation during emotional manipulations. Other scholars have instead revealed the potential involvement of the attentional system in emotion-related time estimates (Lake et al., 2016). For example, facial expressions of shame lead to an underestimation of time (Droit-Volet & Gil, 2009; Grondin et al., 2014, 2015; Mioni et al., 2016). This may be attributed to the nature of shame as a secondary emotion that develops later through social interactions and the internalisation of social norms (Lagattuta & Thompson, 2007; Tangney & Dearing, 2002); it is often regarded as a self-conscious emotion involving self-reflection (Haidt, 2003), leading some researchers to propose that facial expressions of shame capture attentional resources resulting in an underestimation of time (Gil & Droit-Volet, 2011). Also, temporal underestimation is observed when images of rotten food are presented (Gagnon et al., 2018; Gil et al., 2009; Mioni et al., 2021), which is generally explained on the basis of a deviation of attentional resources from time processing. Indeed, according to the Attentional Gate Model (Zakay & Block, 1995), when attention is not fully allocated to time, less temporal information is stored in the accumulator and the temporal interval is judged to be shorter than the objective duration (Droit-Volet & Meck, 2007; Gil & Droit-Volet, 2011). Changes in attention due to emotional stimuli can indeed alter the distribution of cognitive resources to, or away from, a timing task thus affecting subjective time (Lake et al., 2016).

Overall, previous studies indicated an effect of emotional manipulation on subjective time, but distinguishing between arousal and/or attentional effects remains difficult; indeed both can lead to either identical or different behavioural effects (Burle & Casini, 2001; Lake et al., 2016). One possible difficulty in disentangling the effects of emotion on time (pacemaker vs. attentional hypothesis) comes from the use of emotional stimuli to mark time. In this way, not only the presentation of the emotional stimulus is strictly related to the duration to be processed, but it does not allow to outline in which of the stages of the pacemaker-accumulator model emotions act.

To overpass this methodological limitation, some studies have manipulated the position of the emotional stimulus either before or after the temporal interval to be timed. Lui and colleagues (2011) presented an emotional stimulus between two temporal intervals (first temporal interval = S1 and second temporal interval = S2); participants were asked to judge if S2 was longer or shorter than S1. The results indicated that when an emotional stimulus (Negative: Experiments 1, 2, & 4 or Positive: Experiment 3) was introduced, participants tended to underestimate S2 compared with S1.

Later, Vallet et al., (2019) asked participants to compare a target duration with a standard duration previously learned. Two coloured circles were presented at the onset and offset of the target duration, with the colour signalling the valence of the emotional stimulus presented after the response. Results revealed that the expectation of a negative stimulus led to a lengthening of the perceived duration compared with positive and neutral stimuli. In addition, the researchers recorded electroencephalography (EEG) activity and observed a larger N150 component, indicating increased attentional involvement when participants anticipated a negative-valenced stimulus. Moreover, they found a larger negative deflection of the N200 component, suggesting an effect on the decision-making stage of the pacemaker-accumulator process when participants expected a negative stimulus. Finally, a larger late positive potential (LPP) was observed when participants anticipated a negative stimulus than when they anticipated neutral stimuli.

Finally, Gladhill and colleagues (2022) asked participants to complete a time bisection task in which each temporal stimulus was preceded and followed by a negative or neutral face arranged in three conditions: neutral—neutral, neutral—negative, and negative—neutral. The authors observed that, compared with the other conditions, there was a temporal overestimation in a condition where the negative stimulus preceded the temporal one. Furthermore, the EEG activity was recorded and the results showed that contingent negative variation (CNV) signal was significantly larger in the emotional conditions. Moreover, a larger N170 amplitude for negative faces compared with the neutral ones was observed suggesting that negative stimuli are perceived as more arousing. At the same time, an ampler N1 when the temporal stimulus was preceded by a negative one suggests an increased attention towards time. Finally, the authors found a change in the late positive component of time (LPCt) in the emotional conditions for a short duration; this finding suggests the presence of a decision-making bias (Gladhill et al., 2022).

The studies mentioned so far have, however, employed only visual emotional stimuli. Very few studies have been conducted using auditory emotional stimuli. Mella et al. (2011), Noulhiane et al. (2007), and Wackermann et al. (2014) used sounds from the International Affective Digitalized Sounds System (IADS-2; Bradley & Lang, 2007). In Noulhiane et al. (2007), the sounds used were pleasant high-arousal, pleasant low-arousal, unpleasant high-arousal, unpleasant low-arousal, and neutral; Wackermann et al. (2014) used pleasant, unpleasant, and neutral sounds; Mella et al. (2011) used only negative (high and low arousal) and neutral sounds. Overall, the results from these studies seem to confirm the one observed with visual emotional stimuli showing that arousing sounds are judged to be longer than neutral (Mella et al., 2011; Wackermann et al., 2014).

Effects of modality on timing

Another factor that can alter the perception of time is the modality in which temporal intervals are marked. Indeed, temporal intervals marked by auditory stimuli are perceived as longer than temporal intervals of the same length marked by visual stimuli (Wearden & Jones, 2021). Moreover, it is also known that when time is marked aurally, participants show a higher sensitivity to time compared with when time is marked visually (Azari et al., 2020; Grondin, 1993, 2003; Grondin et al., 1998; see also Killeen & Grondin, 2022). The superiority of auditory modality in the temporal domain was investigated using different temporal tasks (Wearden & Jones, 2021), across different temporal ranges (Rammsayer et al., 2015), and in different age groups (Droit-Volet et al., 2004). Rammsayer and colleagues (2015) investigated modality-specific differences by asking their participants to perform duration discrimination of brief (50 ms) and long (1,000 ms) duration and found superior discrimination performance with auditory stimuli than with visual ones, this finding applying to both brief and long temporal intervals. As for age-related differences, Droit-Volet and colleagues (2004) showed that 5-year-old participants reported perceiving the same duration as shorter when presented in the visual modality than when presented aurally. Also, in this age group, the sensitivity to time was higher in the auditory condition than in the visual condition. Penney (2003) posits that there are two plausible accounts for those results. One is that the pacemaker speeds at a faster rate for auditory signals (Wearden et al., 1998), and the other is that there is a difference between auditory and visual signals in the efficiency of maintaining the switch kept closed. In this way, according to Penney (2003), there is a much higher probability that visual stimuli trigger a flickering state into the switch, causing a higher loss of pulses in the visual than in the auditory condition.

Effects of modality on emotion

The study of emotional recognition has shown that the way stimuli are presented affects their perception.¹ Bänziger and colleagues (2009) explored the differences in how emotions are perceived. In their study, participants were presented with 10 emotions portrayed by actors in four modalities: audio only, video only, audio-video, and still pictures. The results revealed a significant effect of the presentation modality. Dynamic facial expressions of emotion (audio-video or video only) were perceived differently compared with acoustic-only and still picture stimuli. In another study by Paulmann and Pell (2011), participants were asked to recognise emotions presented through different modalities: face, semantic, and prosody. The emotions were presented unimodally (one modality), bimodally (two modalities combined), and multimodally (all three modalities combined). The results showed that the more channels employed to display emotional stimuli, the better the participants were at recognising the emotions. It is important to note, however, that the stimuli used in the aforementioned studies refer to highly informative emotional stimuli (prosody, faces, semantics). When referring to emotional stimuli that may not necessarily contain a high amount of information, similar patterns of activations have been found between auditory and visual modalities. For instance, Bradley and Lang (2000) conducted a study comparing emotional reactions to acoustic and visual stimuli. They found that the International Affective Picture System (IAPS) and International Affective Digitized Sounds (IADS) stimuli were comparable in terms of their emotional impact and physiological responses.

Taken together, previous studies demonstrated that our experience of time could be altered by emotional stimuli that serve as temporal markers, as well as the modality through which these stimuli are presented. The present work aims at investigating the contribution of emotions and modality on timing. Participants will be instructed to perform a time bisection task under auditory and visual conditions; auditory emotional stimuli were selected from the IADS-2 database (Bradley & Lang, 2007) and visual emotional stimuli from the IAPS (Lang et al., 2005). Bradley and Lang (2000) observed comparable response modifications, as well as skin conductance and heart-rate responses, to auditory (IADS) and visual (IAPS) emotional stimuli; they concluded that participants’ responses to these auditory and visual emotional stimuli were equivalent.

Regarding the simple effect of modality on time perception we expect higher accuracy and lower temporal variability for temporal intervals marked by auditory rather than visual stimuli in accordance with previous studies that have shown superiority in the perception of temporal duration when those were marked acoustically rather than visually (Grondin, 2003; Wearden & Jones, 2021; Wearden et al., 1998).

The second aim of the present study is to test the effect of emotional stimuli on subjective time and to assess whether this effect is modality-dependent. We selected negative and neutral stimuli for both visual and auditory modalities and expected that, compared with the neutral conditions, using negative emotional stimuli would lead to longer temporal estimation. Furthermore, taking into account the modality in which the emotional and temporal stimuli are presented, we expect that the greater the sensitivity towards time will be, the greater will be the interference of emotional stimuli.

Finally, to better understand whether the effect of emotional stimuli on subjective time perception is caused by a modification at the level of the clock stage or at the level of attention/decision-making stage, we manipulated the presentation of the emotional stimulus by presenting it before or after the to-be-timed interval. If the effect of emotion works at the level of the clock stage, we expect an effect on time perception when the emotional stimulus is presented before the to-be-timed one. On the contrary, if emotional stimulus influences subjective perception of time by diverting attentional resources or modulating the decision process, we expect a greater difference when the emotional stimulus is presented after the to-be-timed one.

Method

Participants and procedure

One hundred university students were included in the study (M_age of 23.5 [2.64] years). Participants were recruited and tested at the Department of General Psychology (University of Padova, Italy) during one experimental session lasting approximately 40 min. The dimension of the sample was obtained via a power analysis computed with G*Power software (Faul et al., 2009).

For an F test (repeated measures ANOVA), with alpha = .05, an expected power of .8, and an effect size (d) of 0.2 (Cohen, 1988), the projected sample size needed was N = 100. Participants were randomly assigned to one of four experimental conditions: two modalities of the emotional stimulus (visual vs. auditory) by two presentations of the emotional stimulus (before vs. after the to-be-timed interval). The study was approved by the Ethics Committee of the Department of General Psychology, University of Padova (Italy; protocol number: 4490), and in accordance with the Declaration of Helsinki. The experimental protocol was carefully explained to each participant and written informed consent was obtained.

Timing task

We used a time bisection task in which participants first learned two standard intervals (short standard and long standard) and then they were instructed to judge new temporal intervals as more similar to the short (“S” key on the keyboard) or to the long standard (“L” key on the keyboard). The two standard intervals were presented 10 times each and lasted 400 (short standard) and 1,000 ms (long standard). During the testing phase participants underwent two blocks of 84 trials each. In each block participants were presented with 7 target intervals of 400, 500, 600, 700, 800, 900, and 1,000 ms; each interval was repeated 12 times in each block (6 negative and 6 neutral emotional stimuli marked to-be-timed interval).

We used two conditions in which the emotional stimuli (either negative or neutral) were presented before or after the timing interval (Figure 1a and b). For example, in the condition “emotional stimuli first” (Figure 1a), each trial in the testing phase started with the presentation of a fixation cross (500 ms), followed by the presentation of the emotional stimulus (negative or neutral), followed by a blank screen (500 ms); then the temporal interval was presented (between 400 and 1,000 ms). In the condition “emotional stimuli second,” the order of presentation of emotional and temporal intervals was reversed (Figure 1b).

Figure 1.

Graphical representation of the experimental procedure in both presentation orders. The emotional stimulus is presented (a) first or (b) second.

A grey circle was used as the visual timing stimulus in the visual task whereas a pink noise, which was synthesised at a sample rate of 44.1 kHz and a 16-bit resolution delivered from computer speakers, was used for the auditory timing stimulus. Participants were seated approximately 60 cm from the computer screen. The visual emotional stimuli were selected from the IAPS database (Lang et al., 2005) and always lasted 700 ms. Four images were selected as negative-high arousing stimuli and there were four neutral images (Table 1). The auditory stimuli were selected from the IADS-2 database (Bradley & Lang, 2007). We selected 16 stimuli, 8 negative, and 8 neutral. Original stimuli lasted for a few seconds; to be consistent with the visual condition we selected auditory stimuli based on the original level of arousal and valence to be negative and neutral and cut the auditory traces to make sure these stimuli would also last 700 ms. Twenty university students (range 21–26 years old) from the Department of General Psychology, University of Padova (Italy), were recruited for the pilot study, and they were asked to evaluate the 16 auditory traces and judge the level of arousal and valence using the Self-Assessment Manikin (SAM, Bradley & Lang, 1994). The SAM is a non-verbal pictorial assessment technique that directly measures the valence and arousal associated with a person’s affective reaction to the presented stimuli. This scale requires participants to express their subjectively perceived levels of activation and pleasantness after being exposed to a stimulus. The scale in which participant had to respond was a 9-item scale ranging from the negative pole (less pleasant and less activated) to the positive one (more pleasant and more activated). Eight auditory traces were finally selected, the levels of arousal and valence associated with those stimuli are listed in Table 1.

Table 1.

Visual and auditory stimuli employed in the study.

	Visual			Auditory
	St. ID	Valence	Arousal	St. ID	Valence	Arousal
Negative	2,800	1.78	5.49	275	2.2	6.1
	3,051	2.30	5.62	277	1.7	7.1
	9,040	1.67	5.82	286	3.2	8.5
	9,253	2.00	5.53	286	3.3	7.9
Neutral	7,035	4.98	2.66	152	6.9	3.6
	7,052	5.33	3.01	170	5.2	3.3
	7,053	5.22	2.95	171	5.3	2.8
	7,055	4.90	3.02	206	6.1	3.1

Statistical analyses

We calculated two indices: one for the perceived duration and one for sensitivity. The first was the bisection point (BP)—that is, the stimulus duration at which the participants responded “short” or “long” with equal frequency. A psychometric function was plotted for each participant in each condition and each psychometric function was adjusted to a cumulative Gaussian function through a maximum likelihood estimation procedure. A shift of the BP for the different emotional stimuli presented can be interpreted as an indicator of differences in these conditions, with smaller BP (left shift of the psychometric function) values meaning longer perceived durations. The second dependent variable was the Weber ratio (WR); here it is defined as the standard deviation from the psychometric function divided by the mean duration (700 ms) in order to avoid biases due to a smaller BP in the auditory condition (Wearden & Jones, 2021) and have a more consistent index across modalities. With this measure of sensitivity, smaller values indicate higher sensitivity and less variable timing.

Two separate repeated-measures ANOVAs were conducted on BP and WR as dependent variables with Emotion (negative vs. neutral) as within-subjects factor and Order (emotion first vs. emotion second) and Modality (visual vs. auditory) as between-subjects factors. All significant effects were followed by post hoc analyses performed with a Bonferroni correction to reduce Type I error rate, and the effect size was estimated with partial eta squared (η²_p).

Results

Bisection point

When data were analysed in terms of BP, we observed the main effects of Emotion (F[1,96] = 6.30, p = .014, η²_p = .06), Order (F[1,96] = 7.64, p = .007, η²_p = .07), and Modality (F[1,96] = 8.63, p = .004, η²_p = .08) indicating that participants overestimated time when exposed to negative stimuli compared with neutral ones, when the emotional stimuli were presented before the timing interval and in the auditory compared with the visual condition.

We also observed significant interactions between Emotion × Order (F[1,96] = 4.87, p = .030, η²_p = .05), Emotion × Modality (F[1,96] = 5.49 p = .021, η²_p = .05), and Order × Modality (F[1,96] = 5.21, p = .025, η²_p = .05). Also, the interaction Emotion × Order × Modality (F[1,96] = 4.56, p = .035, η²_p = .04) was significant (Figure 2). To better understand our results, we decided to separate the file based on modality and conduct two separate ANOVAs with Emotion as a within-subjects factor and Order as a between-subjects factor. Making this decision was supported by the fact that, although a fully interactive model can give more precise and robust results, and is preferable, a sample-divided approach yields approximately the same estimates (Franzese & Kam, 2009).

Figure 2.

Mean bisection point. (a) Visual and (b) auditory conditions when the emotional stimuli were presented before (First) or after (Second) the to-be-timed interval. Error bars indicate standard errors.

Modality: visual

We observed significant main effects of Emotion (F[1,48] = 7.78, p = .008, η²_p = .02) and Order (F[1,48] = 11.50, p = .001, η²_p = .16) as well as a significant interaction between Emotion × Order (F[1,48] = 6.23, p = .016, η²_p = .02) (Figure 2a). Post hoc analyses indicated that when the emotional stimuli were presented after the to-be-timed interval, participants responded “long” more often when the emotional stimulus was negative compared with when it was a neutral stimulus (t = 3.74, p = .003); there was no such difference between negative and neutral stimuli when the emotional stimuli were presented before the to-be-timed interval (t = .21, p > .05). When the emotional stimulus was negative, no difference between conditions was observed (t = 2.07, p > .05) but participants underestimated time when the neutral stimulus was presented after the temporal interval compared with when it was presented before the temporal interval (t = 4.12, p < .001).

Modality: auditory

We observed no significant main effect of Emotion (p = .866, η²_p < .01) or Order (p = .720, η²_p < .01) and no significant interaction between Emotion × Order (p = .943, η²_p < .01) (Figure 2b).

Weber ratio

When data were analysed in terms of WR, only the main effect of Modality (F[1,96] = 22.62, p < .001, η²_p = .19) was significant, indicating that participants were more variable when temporal intervals were presented in the visual modality than when presented in the auditory modality. Neither the main effect of Emotion (p = .175, η²_p = .02) or Order (p = .053, η²_p < .04) nor the interaction between these variables (all ps > .069, η²_p = .03) were significant (Figure 3a and b).

Figure 3.

Weber ratio. (a) Visual and (b) auditory conditions when the emotional stimuli were presented before (First) or after (Second) the to-be-timed interval. Error bars indicate standard errors.

Discussion

Despite the human ability to correctly perceive time, many factors alter the subjective perception of it and emotion is one of those factors. There is, indeed, abundant evidence in the literature supporting the hypothesis that emotionally relevant situations can expand or shrink our perception of time (Droit-Volet, 2019). However, support is lacking concerning whether the effect of emotions on time perception takes place at the clock level, if it diverts attentional resources from time, or if it acts at the decision-making stage of the internal clock model (Gibbon et al., 1984; Lake et al., 2016). Moreover, fewer studies have been conducted to investigate if the temporal distortion induced by emotional stimuli might be modality-dependent. The present study aimed at providing responses to the above-mentioned issues; to do so, we asked participants to complete a time bisection task in which we manipulated the presentation of emotional stimuli, presenting them before or after the interval to-be-timed, and the modality (visual vs. auditory) of the stimuli.

First, we hypothesised that participants would perform with higher accuracy and lower temporal variability in the auditory condition compared with the visual one. Our results confirm this hypothesis. Analysing the BP, indeed, we observed a significant effect of modality on time perception with participants reporting a temporal overestimation when temporal intervals were marked by auditory rather than visual stimuli. A possible explanation comes from the superiority of the auditory over the visual modality for time perception, a result that is consistent with the high sensitivity of the auditory system to rapidly changing signals (i.e., speech or music perception Tallal et al., 1993). Specifically for timing, it is critical to reliably detect the beginning and the ends of the timekeeping activity. Less variance in the time taken to detect sensory signals marking an interval, and in the sensory trace left by such signals, would allow a more precise internal representation of intervals to be judged. It has been demonstrated that auditory signals are more readily processed than visual signals; indeed, the response time for an auditory stimulus is sensibly shorter than that for a visual stimulus (Jaśkowski et al., 1990). Discussing the modality effect on timing according to the internal clock model, a possible explanation of the modality effect is that an internal clock runs at a faster rate for auditory than for visual signals (Wearden et al., 1998). Therefore, the accumulated clock value for a given duration will be larger when the signal is auditory than when it is visual (the auditory signal will seem longer). On the contrary, the effect of modality on timing can be explained by a more stable openness of the switch for auditory (Penney, 2003); compared with visual stimuli, acoustic ones are indeed less likely to cause a flickering state of the switch, allowing a larger number of pulses to reach the accumulator. Furthermore, analysing the WR as an index of sensitivity, we observed that acoustic stimuli were associated with higher sensitivity, a finding that is consistent with the literature (Azari et al., 2020; Grondin, 2003).

A second aim of the present study was to test the effect of emotions on timing and, in particular, if the effects of emotions acted at the level of the clock stage or at the level of attention/decision-making stage. To do so, we manipulated the presentation of the emotional stimuli by placing them before or after the to-be-timed interval. Overall, we observed a main effect of emotion indicating that participants tended to respond “long” more often when they were exposed to negative stimuli over neutral ones. The effect of emotion on timing was modulated by the modality and presentation order, to better understand the effect of emotion on timing based on the presentation order we decided to run separate analyses based on modality.

When the emotional stimulus was visual, we observed an effect on timing (temporal overestimation) when it was presented after the temporal interval, a result that seems to be in line with the hypothesis that emotions act on the decision process rather than at the clock level; a result that differs from what is commonly reported in the literature (Droit-Volet, 2019; Droit-Volet & Gil, 2009). Indeed, most of the previous studies investigating the effect of emotional stimuli on timing claimed that the temporal overestimation observed when participants are exposed to negative and arousing stimuli is explained by a speed-up of the internal clock (Droit-Volet, 2019; Droit-Volet & Gil, 2009). It is important to note that here we presented the emotional stimuli before or after the timing interval, whereas in previous studies the emotional stimulus marked the interval to be timed. In such situations, it can be possible that the effect of emotion on timing occurs through an online process during the storing of the temporal stimulus, an effect that can reasonably act at the clock stage.

In line with this assumption, the separation of the emotional stimulus from the temporal one should allow us to detect possible emotional interference at different stages of the internal clock model. For instance, Lui and colleagues (2011) carried out a series of experiments in which the to-be-timed stimulus was separated from the emotional one and found a temporal underestimation of the temporal intervals when preceded by a negative stimulus, challenging the idea that the effect of emotion on time perception occur merely influencing the pacemaker rate. Specifically, Lui and colleagues (2011) asked participants to compare a standard duration with a target one with an emotional distracter placed between the two temporal stimuli. In their experiments the authors manipulated the modality in which temporal stimulus was presented, the valence of the emotion presented (negative/positive vs. neutral), the delay that occurred between the offset of the emotional distracter and the onset of the target duration, and the task employed. Similar results are not in line with the hypothesis that an arousing stimulus speeds up the pacemaker rate enlarging the perceived duration of the temporal interval; rather, the authors assume that the effects found can be due to attentional influences. In this case, indeed, attentional resources can be deviated from the to-be-timed duration leading participants to perceive duration as shorter. Another important study that is worth mentioning and that excludes a possible bias when storing the temporal stimulus is the one authored by Vallet and colleagues (2019). The authors used an EEG approach to test the effect of emotional expectation on time perception. Their task required participants to compare the duration of two temporal intervals that were followed by an emotional (positive vs. neutral vs. negative) stimulus. The results showed that the expectation of a negative stimulus enlarged the perception of the duration in the participants. Moreover, the larger amplitude in the negative condition of N15O, N200, LPP, and CNV, associated with attention capturing and cognitive control, suggest that below the role of the pacemaker other cognitive processes such as attention and decision-making/strategy adjustment can be involved in the perception of time.

Based on the evidence reported above, it seems that when the emotional stimulus is released from the temporal one, it is possible to study at which level of the internal clock model the distortion of time occurs and, by looking at our results, it seems that the presence of a negative stimulus after the to-be-timed event affected the decision process by biasing participants against a “choose short” effect, the tendency to respond shorter when participants are uncertain (Wiener & Thompson, 2015).

A similar result has also been described in a recent work by Gladhill and colleagues (2022).

In their study, indeed, the authors employed both behavioural and electrophysiological paradigms to investigate the underlying mechanism involved in the emotional distortion of time perception. The experimental procedure included only a visual time bisection task in which the temporal stimulus was included between two emotional stimuli generating three different conditions: one in which the temporal stimulus was framed by two neutral stimuli, one in which the temporal stimulus was preceded by a negative stimulus and followed by a neutral one, and one in which the temporal stimulus was preceded by a neural stimulus and followed by a negative one. Their EEG data showed that the LPCt, which is generally associated with decision-making and difficulty in temporal discrimination, increased significantly when the emotional stimulus was presented after the temporal one, suggesting a bias in the decision-making process against choosing “short” as a response option (Gladhill et al., 2022).

When the emotional stimulus was, instead, auditory we did not observe any significant effect on timing, although previous findings reported an effect. Studies such as the one conducted by Mella and colleagues (2011) found that when participants were attending to the emotional content of a stimulus, they perceived its duration as longer. Moreover, Noulhiane and colleagues (2007), as well as Wackermann and colleagues (2014), found that when emotional auditory stimuli, both positive and negative, were used to mark time, participants reported longer perceived duration compared with neutral conditions. The absence of an effect of emotion on timing in the auditory condition in our study may be due to the fact that the range of target durations employed in the present study (400–1,000 ms) was significantly shorter than those employed in other works (1 s–6 min), and that emotional stimuli presented in the auditory modality require a longer time to be accurately processed and recognised (Paulmann & Pell, 2011).

Taken together, our results support the effect of modality on time perception indicating longer perceived duration and less variability with auditory compared with the visual modality. We expected this effect to be observed also when emotional auditory stimuli were used; however, our results indicated that only emotional visual stimuli influenced subjective timing, emphasising the need to use longer durations. Moreover, our results support the findings that negative emotional stimuli expand our perception of time, but that the effect of emotion on time depends on the presentation order (emotional stimuli presented after the interval to be timed) of the emotional stimuli. In conditions where the order of presentation of the emotional stimulus is manipulated, negative emotions affect our perception of durations acting at the decision-making stage rather than at the pacemaker one.

Footnotes

Acknowledgements

The information in this manuscript and the manuscript itself has never been published either electronically on in print.

Author contributions

Conceptualisation: G.M., S.G.; data curation: G.M., L.M.; formal analysis: G.M., L.M.; investigation: G.M.; methodology: G.M.; project administration: G.M.; resources: G.M.; software: G.M., L.M.; supervision: G.M., S.G.; roles/writing—original draft: G.M., L.M., S.G.; writing—review & editing: G.M., L.M., S.G.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Giovanna Mioni

Data accessibility statement

Data are available upon request.

Notes

References

Angrilli

Cherubini

Pavese

Manfredini

(1997). The influence of affective factors on time perception. Perception & Psychophysics, 59(6), 972–982.

Azari

Mioni

Rousseau

Grondin

(2020). An analysis of the processing of intramodal and intermodal time intervals. Attention, Perception, & Psychophysics, 82(3), 1473–1487.

Bänziger

Grandjean

Scherer

K. R.

(2009). Emotion recognition from expressions in face, voice, and body: the Multimodal Emotion Recognition Test (MERT). Emotion, 9(5), 691.

Bradley

M. M.

Lang

P. J.

(1994). Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59.

Bradley

M. M.

Lang

P. J.

(2000). Affective reactions to acoustic stimuli. Psychophysiology, 37, 204–215.

Bradley

M. M.

Lang

P. J.

(2007). International affective digitized sounds (IADS-2): Affective ratings of sounds and instruction manual (Technical Report No. B-3). NIMH Center for the Study of Emotion and Attention, University of Florida.

Burle

Casini

(2001). Dissociation between activation and attention effects in time estimation: Implications for internal clock models. Journal of Experimental Psychology: Human Perception and Performance, 27(1), 195–205.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.), Erlbaum, Hillsdale, NJ.

Droit-Volet

(2019). The temporal dynamic of emotion effects on judgment of durations. In Arstila

Bardon

Power

S. E.

Vatakis

(Eds.), The illusions of time (pp. 103–125). Palgrave Macmillan.

10.

Droit-Volet

Gil

(2009). The time–emotion paradox. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1525), 1943–1953.

11.

Droit-Volet

Meck

W. H.

(2007). How emotions colour our perception of time. Trends in Cognitive Sciences, 11(12), 504–513.

12.

Droit-Volet

Tourret

Wearden

(2004). Perception of the duration of auditory and visual stimuli in children and adults. The Quarterly Journal of Experimental Psychology Section A, 57(5), 797–818.

13.

Faul

Erdfelder

Buchner

Lang

A. G.

(2009). Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160.

14.

Franzese

Kam

(2009). Modeling and interpreting interactive hypotheses in regression analysis. University of Michigan Press.

15.

Gagnon

Begin

Laflamme

Grondin

(2018). Temporal processing of joyful and disgusting food pictures by women with an eating disorder. Frontiers in Human Neurosciences, 12, Article 129.

16.

Gerdes

A. B.

Wieser

M. J.

Alpers

G. W.

(2014). Emotional pictures and sounds: a review of multimodal interactions of emotion cues in multiple domains. Frontiers in Psychology, 5, 1351.

17.

Gibbon

Church

R. M.

Meck

W. H.

(1984). Scalar timing in memory. Annals of the New York Academy of Sciences, 423, 52–77.

18.

Gil

Droit-Volet

(2011). How do emotional facial expressions influence our perception of time? In Masmoudi

Yan Dai

Naceur

(Eds.), Attention, representation, and human performance: Integration of cognition, emotion and motivation. Psychology Press, Taylor & Francis.

19.

Gil

Rousset

Droit-Volet

(2009). How liked and disliked foods affect time perception. Emotion, 9, 457–463.

20.

Gladhill

K. A.

Mioni

Wiener

(2022). Dissociable effects of emotional stimuli on electrophysiological indices of time and decision-making. PLOS ONE, 17(11), Article e0276200.

21.

Grondin

(1993). Duration discrimination of empty and filled intervals marked by auditory and visual signals. Perception & Psychophysics, 54, 383–394.

22.

Grondin

(2003). Sensory modalities and temporal processing. In Helfrich

(Ed.), Time and mind II: Information processing perspectives (pp. 75–92). Hogrefe & Huber.

23.

Grondin

(2010). Timing and time perception: A review of recent behavioral and neuroscience findings and theoretical directions. Attention, Perception, & Psychophysics, 72(3), 561–582.

24.

Grondin

(2014). Why studying intermodal duration discrimination matters. Frontiers in Psychology, 5, Article 628.

25.

Grondin

(2020). The perception of time—Your questions answered. Routledge.

26.

Grondin

Laflamme

Bienvenue

Labonté

Roy

M.-L.

(2015). Sex effect in the temporal perception of faces expressing anger and shame. International Journal of Comparative Psychology, 28, 27200.

27.

Grondin

Laflamme

Gontier

. (2014). Effect on perceived duration and sensitivity to time when observing disgusted faces and disgusting mutilation pictures. Attention, Perception, & Psychophysics, 76(6), 1522–1534.

28.

Grondin

Meilleur-Wells

Ouellette

Macar

(1998). Sensory effects on judgments of short-time intervals. Psychological Research, 61, 261–268.

29.

Haidt

(2003). The moral emotions. In Davidson

R. J.

Scherer

K. R.

Goldsmith

H. H.

(Eds.), Handbook of affective sciences (pp. 852–870). Oxford University Press.

30.

Hancock

P. A.

(1993). Body temperature influence on time perception. Journal of General Psychology, 120, 197–215.

31.

Jaśkowski

Jaroszyk

Hojan-Jezierska

(1990). Temporal-order judgments and reaction time for stimuli of different modalities. Psychological Research, 52(1), 35–38.

32.

Killeen

P. R.

Grondin

(2022). A trace theory of time perception. Psychological Review, 129, 603–639. https://doi.org/10.1037/rev0000308

33.

Lagattuta

K. H.

Thompson

R. A.

(2007). The development of self-conscious emotions: Cognitive processes and social influences. In Tracy

J. L.

Robins

R. W.

Tangney

J. P.

(Eds.), The self-conscious emotions: Theory and research (pp. 91–113). Guilford.

34.

Lake

J. I.

(2016). Recent advances in understanding emotion-driven temporal distortions. Current Opinion in Behavioral Sciences, 8, 214–219.

35.

Lake

J. I.

LaBar

K. S.

Meck

W. H.

(2016). Emotional modulation of interval timing and time perception. Neuroscience & Biobehavioral Reviews, 64, 403–420.

36.

Lang

P. J.

Bradley

M. M.

Cuthbert

B. N.

(2005). International affective picture system (IAPS): Affective ratings of pictures and instruction manual (pp. A-8). Gainesville, FL: NIMH, Center for the Study of Emotion & Attention.

37.

Lee

K. H.

Seelam

O’Brien

(2011). The relativity of time perception produced by facial emotion stimuli. Cognition & Emotion, 25(8), 1471–1480.

38.

Lui

M. A.

Penney

T. B.

Schirmer

(2011). Emotion effects on timing: Attention versus pacemaker accounts. PLOSONE, 6(7), Article e21829.

39.

Matthews

W. J.

(2011). Can we use verbal estimation to dissect the internal clock? Differentiating the effects of pacemaker rate, switch latencies, and judgment processes. Behavioural Processes, 86(1), 68–74.

40.

Mella

Conty

Pouthas

(2011). The role of physiological arousal in time perception: Psychophysiological evidence from an emotion regulation paradigm. Brain and Cognition, 75(2), 182–187.

41.

Mioni

Grondin

Stablum

(2021). Do I dislike what you dislike? Investigating the effect of disgust on time processing. Psychological Research, 85(7), 2742–2754.

42.

Mioni

Stablum

Prunetti

Grondin

(2016). Time perception in anxious and depressed patients: A comparison between time reproduction and time production tasks. Journal of Affective Disorders, 196, 154–163.

43.

Noulhiane

Mella

Samson

Ragot

Pouthas

(2007). How emotional auditory stimuli modulate time perception. Emotion, 7, 697–704.

44.

Paulmann

Pell

M. D.

(2011). Is there an advantage for recognizing multi-modal emotional stimuli? Motivation and Emotion, 35(2), 192–201.

45.

Penney

T. B.

(2003). Modality differences in interval timing: Attention, clock speed, and memory. In Meck

W. H.

(Ed.), Functional and neural mechanisms of interval timing (pp. 209–233). CRC Press/Routledge/Taylor & Francis Group.

46.

Rammsayer

T. H.

Borter

Troche

S. J.

(2015). Visual-auditory differences in duration discrimination of intervals in the subsecond and second range. Frontiers in Psychology, 6, Article 1626.

47.

Tallal

Galaburda

A. M.

Llinás

R. R.

von Euler

(Eds.). (1993). Temporal information processing in the nervous system: Special reference to dyslexia and dysphasia. New York Academy of Sciences.

48.

Tangney

J. P.

Dearing

R. L.

(2002). Shame and guilt. Guilford Press.

49.

Tipples

(2008). Negative emotionality influences the effects of emotion on time perception. Emotion, 8(1), 127–131.

50.

Treisman

(1963). Temporal discrimination and the indifference interval: Implications for a model of the “internal clock.” Psychological Monographs: General and Applied, 77, 1–31.

51.

Vallet

Laflamme

Grondin

(2019). An EEG investigation of the mechanisms involved in the perception of time when expecting emotional stimuli. Biological Psychology, 148, 107777.

52.

van Wassenhove

Buonomano

D. V.

Shimojo

Shams

. (2008). Distortions of subjective time perception within and across senses. PLOS ONE, 3(1), Article e1437.

53.

Wackermann

Meissner

Tankersley

Wittmann

(2014). Effects of emotional valence and arousal on acoustic duration reproduction assessed via the “dual klepsydra model.” Frontiers in Neurorobotics, 8, Article 11.

54.

Wearden

J. H.

Edwards

Fakhri

Percival

(1998). Why “sounds are judged longer than lights”: Application of a model of the internal clock in humans. The Quarterly Journal of Experimental Psychology: Section B, 51(2), 97–120.

55.

Wearden

J. H.

Jones

L. A.

(2021). Judgements of the duration of auditory and visual stimuli. Timing & Time Perception, 9(2), 199–224.

56.

Wearden

J. H.

Penton-Voak

I. S.

(1995). Feeling the heat: Body temperature and the rate of subjective time revisited. Quarterly Journal of Experimental Psychology Section B, 48, 129–141.

57.

Wiener

Thompson

J. C.

(2015). Repetition enhancement and memory effects for duration. Neuroimage, 113, 268–278.

58.

Wittmann

Leland

D. S.

Churan

Paulus

M. P.

(2007). Impaired time perception and motor timing in stimulant-dependent subjects. Drug and Alcohol Dependence, 90(2–3), 183–192.

59.

Zakay

Block

R. A.

(1995). An attentional-gate model of prospective time estimation. Time and the Dynamic Control of Behavior, 5, 167–178.

60.

Zhang

Zhao

Zhang

Cao

Yin

Liu

Yuan

T. F.

Luo

(2019). Time perception deficits and its dose-dependent effect in methamphetamine dependents with short-term abstinence. Science Advances, 5, Article eaax6916.