Dissociable Codes in Motor Working Memory

Abstract

Working memory has been comprehensively studied in sensory domains, like vision, but little attention has been paid to how motor information (e.g., kinematics of recent movements) is maintained and manipulated in working memory. “Motor working memory” (MWM) is important for short-term behavioral control and skill learning. Here, we employed tasks that required participants to encode and recall reaching movements over short timescales. We conducted three experiments (N = 65 undergraduates) to examine MWM under varying cognitive loads, delays, and degrees of interference. The results support a model of MWM that includes an abstract code that flexibly transfers across effectors, and an effector-specific code vulnerable to interfering movements, even when interfering movements are irrelevant to the task. Neither code was disrupted by increasing visuospatial working memory load. These results echo distinctions between representational formats in other domains, suggesting that MWM shares a basic computational structure with other working memory subsystems.

Keywords

working memory motor action visuospatial executive function open data

Imagine learning a new dance move—you have just done something right and you want to do it again. Recalling your recent successful action requires the use of working memory. Working memory is a form of short-term memory in which encoded information is actively maintained and/or manipulated to serve the needs of an ongoing task or goal (Atkinson & Shiffrin, 1971; Luck & Vogel, 2013). Our capacity to remember recent movements in “motor working memory” (MWM) is likely critical for early stages of motor learning (Krakauer et al., 1999; Posner & Konick, 1966) and for flexible control across a range of tasks (Allen et al., 2023; Raw et al., 2019).

Despite its everyday importance, a distinct MWM system has been largely overlooked in models of both working memory and motor control. Some studies have demonstrated that people can retain information about the felt distance of a recent reaching movement without visual information, but this work has primarily focused on people recalling somatosensory information from the encoding limb (Jones & Henriques, 2010; Keele & Ells, 1972; Posner & Konick, 1966; Smyth & Pendleton, 1989; Vaillancourt & Russell, 2002). This type of somatosensory memory resource may be modality specific, as it is not significantly affected by visual memory load (Seemüller et al., 2011). Furthermore, work by Smyth et al. (1988) showed that complex novel gestures can be held in memory even with concurrent articulatory and spatial interference. Somatosensory memory of recent movements is thought to improve motor learning, suggesting that “bottom-up,” effector-specific information can aid in the acquisition of new motor behaviors (Bernardi et al., 2015; Sidarta et al., 2018). There is also evidence that a distinct memory store for motor plans can aid in the maintenance of sequential action instructions (Allen et al., 2023; Jaroslawska et al., 2018; Li et al., 2022).

Returning to our dance example, re-creating or executing a variation of your new dance move may require more than somatosensory memory alone. One could imagine recalling a more effector-independent, abstract representation of a trajectory or spatial goal (Cohen et al., 2005; Scheidt & Ghez, 2007; Wong et al., 2019). Previous work related to MWM has typically not explored these more abstract codes. Here we propose that both of these forms of information—effector specific and effector independent—can simultaneously contribute to MWM.

Models of short-term memory that include multiple distinct codes are common in the memory literature. For example, in visual short-term memory, iconic memory is distinct from visual working memory (Sperling, 1960). Within working memory itself, phonological working memory can be dissociated from visuospatial working memory (Baddeley, 2003). In the motor domain, there is substantial evidence for multiple representations (e.g., reference frames) in long-term memory, demonstrated by motor learning research that points to both intrinsic (body-referenced) and extrinsic (world-referenced) long-term memory formats (Bapi et al., 2000; Bays & Wolpert, 2006; Brayanov et al., 2012; Liu et al., 2020; Wiestler et al., 2014). Whether multiple codes are present in MWM, when long-term learning is not relevant, is, to our knowledge, unknown. Moreover, if multiple MWM codes do exist, it is not clear what form they take. Here we propose that in addition to maintaining an effector-specific code, MWM may also represent movement information in a more flexible, abstract format that is distinct from conventional visuospatial working memory. We employed a novel working memory paradigm designed to dissociate these putative motor memory codes. Our experimental logic is centered around the notion that a more abstract MWM code is transferable across effectors (effector independent), whereas an effector-specific MWM code should be especially vulnerable to sensory interference. Here we test our hypotheses in three behavioral experiments.

Open Practices Statement

We have made all data and analysis used in this article available in a GitHub repository located at github.com/hhillman231/Dissociable_Codes_in_MWM. The experiments were not preregistered.

Method

Participants

All participants were recruited via Yale’s student psychology pool (Experiment [Exp.] 1: N = 22; 13 female, 1 prefer not to say; mean age = 19.38; Exp. 2: N = 26; 19 female, 1 prefer not to say; mean age = 20.04; Exp. 3: N = 17; 8 female; mean age = 19.71). Participants received class credit for participation, and there was no financial incentive to participate. Approval for this research was granted by Yale University’s institutional review board, and each participant signed a consent form prior to participation, in accordance with these approved protocols.

Statement of Relevance

Remembering movement information over short timescales plays an important role in a wide range of tasks, from learning a new phrase on the piano, to relearning movements after a stroke. In this study, we used a novel paradigm in which participants were required to maintain movement information with no visual input and recall it after short delays. Beyond demonstrating classic working memory load and interference effects, we provide evidence for two distinct motor working memory codes: one that is bound to the recently moved limb (effector specific) and a more abstract representation that generalizes across limbs (effector independent). These findings advance our understanding of an important but understudied working memory subsystem and make novel mechanistic predictions with psychological, neural, and clinical implications.

The Edinburgh Handedness Inventory (Oldfield, 1971) was used to determine handedness. Seven participants were excluded from analysis: Two were excluded due to experimenter error (Exp. 1), and five (two in Exp. 2, three in Exp. 3) were excluded prior to data analysis for not adhering to the instructions. Sample sizes were based on recent psychophysical studies on motor memory that revealed robust, replicable memory effects (Bays & Wolpert, 2006; Brayanov et al., 2012; McDougle & Taylor, 2019).

General procedure

Participants were monitored by the experimenter throughout the session. They were seated in front of a robotic manipulandum (KINARM End-Point, Ontario, CA; Fig. 1A), which has a low-friction two-joint robotic arm with a cylindrical handle for participants to grasp. Participants can move or be moved by the robotic arm, which tracks their position in a two-dimensional workspace (sampling frequency: 1 kHz). Participants sat in an adjustable chair, and its height was adjusted for each participant so that their forehead aligned with a soft padded headrest. The task chair was centered so that the resting place of the manipulandum was centered at the participant’s midline, and the chair wheels were locked into place. Participants kept their feet on a mat below the table that supported the robot. Participants were explicitly instructed to try and maintain the same posture throughout the experiment and were closely monitored for visible deviations from that posture. In Exp. 1 and Exp. 2, the hand not currently holding the robot handle rested on the support table in a prone position. In Exp. 3, whichever hand was not holding the robot handle held onto a joystick (Fig. 3A). One of the participant’s hands held the robot handle at all times, except for occasional brief moments when they were instructed to switch hands, as described later.

Fig. 1.

Task design. (A) Schematic of the participant setup with the Kinarm robot. Participants received no visual feedback except for the occasional text instructions on the monitor. The hand and arm were occluded by an opaque screen, and a bib further blocked their ability to see anything below the neck. (B) Experiment 1: Participants encoded one or four movements in a sequence. During the maintenance period, they were either asked to switch hands or received no instruction and kept the encoding hand on the robotic handle. If participants encoded one movement (set size = 1), they would always be asked to recall that movement. If participants encoded four movements (set size = 4), they would be asked to recall one of the four encoded movements via a number cue. An example trial of Experiment 1 is on the right, showing set size = 4 and “switch” conditions. The black boxes with text show what the participant would have seen. (C) Experiment 2: Participants were instructed to encode one movement. During maintenance, participants experienced one or three extra “irrelevant” movements they did not need to remember, or a short or long period of delay that was equivalent to the amount of time it took for one or three extra movements (3.7 or 11.1 s, respectively). Following one of these four conditions, participants were either asked to switch hands or not. Regardless of participants experiencing any extra movements, they were tasked with recalling the single encoded movement.

Crucially, the participant’s vision of their upper limbs was fully occluded by both the horizontally mounted LCD display screen of the robotic device and an opaque bib worn around their neck, which was fastened to the horizontal monitor (Fig. 1A). Unless otherwise noted (i.e., Exp. 3), visuals in the task were limited to task instructions (i.e., text prompts), which were reflected from the display onto a visible semisilvered mirror positioned above the workspace. Thus, at no point in the experiment did participants receive any visual information regarding movement target locations, their limb position, or any errors in their memory reports, making it a fully kinesthetic task. Text prompts appeared at the top of the workspace screen (i.e., out of reach of the movement workspace) such that participants could not use text stimuli as potential visual anchor points.

We conducted three MWM experiments that adhered to the same basic design. Before each trial began, participants were instructed to grasp the manipulandum handle with their right or left hand, with the hand cued by a text prompt (“Grab with [right/left] hand”). The trial did not begin until the robotic handle’s built-in sensor registered the participant’s grasp. All movements by the robot or subject stemmed from a central “home” position centered on the midline ~40 cm from their chest.

We hypothesized that by switching hands between encoding and recall (“switch” condition), participants could no longer directly rely on active effector-specific information held in working memory to inform their recall and could thus perform the memory task only on the basis of some form of recoded, effector-independent information. Alternatively, in the “no-switch” condition, participants could use both effector-specific and effector-independent information. Our logic is subtractive, such that subtracting the performance in the no-switch condition from the switch condition (i.e., isolating the hand “switch cost”) can reveal the role of effector-specific memory in MWM recall performance.

Experiment 1: set size manipulation

Exp. 1 was broken into four 24-trial blocks as well as one eight-trial practice block. No practice block data were included in our analysis. Between blocks, participants could take a moment to rest. Participants were encouraged to ask questions during the practice block if they had any. Each participant was tested on a different trial schedule during each block, and the schedules were designed to equally balance the number of switch and no-switch trials, the initial encoding hand (left or right), and set size 1 versus 4, which refers to the number of movements encoded and maintained on a given trial (henceforth, “SS1” and “SS4,” respectively). In SS4, trials were counterbalanced for which of four successively encoded movements they were asked to recall (movement 1, 2, 3, or 4). Target locations were pseudorandomized within and across blocks and conditions and were drawn from invisible arcs subtending 150° at three distances (8, 12, and 16 cm) from the starting position.

During the encoding phase, the manipulandum guided participants to an invisible target(s). Each movement started in the same location and moved outward in a straight line, paused briefly, and returned to the starting position. Each encoding movement took ~800 ms. Participants were naive to the set size they would experience until they either began a second movement or did not. At the same time that the movement began, a number would appear on the screen explicitly indicating the current movement number (e.g., “#1”) and remained there until the movement was terminated. This cue appeared regardless of set size. During the maintenance phase, subjects were first either cued to switch hands or not. Regardless of the switch condition, the maintenance phase lasted for 4 s. On switch trials, hand switching had to be completed in less than those 4 s; this allowed us to match the total length of the maintenance period between switch and no-switch conditions. Across all trials, 2.0% were excluded from analysis because of timing out (not switching hands quickly enough). Finally, in the recall phase, subjects were cued to recall either the single encoded movement (in SS1 trials) or one of the four encoded movements (in SS4 trials).

Experiment 2: movement interference and delay

Exp. 2 was similar in design to Exp. 1, though all trials were SS1 trials (i.e., only one movement was encoded for later recall). Crucially, in the experimental conditions of Exp. 2, trials involved either one or three “irrelevant” passive movements that followed the single encoded movement. That is, participants were tasked with encoding (and later recalling) only the first movement presented during the encoding phase. At the start of the task, participants were thoroughly instructed to ignore any passive movements that occurred after the initial movement. This allowed us to ask if motor interference plays a role in MWM (or more specifically, its putative effector-specific subcomponent), even when experienced interfering movements are not deliberately committed to working memory.

Because the interfering movement(s) added additional time between the encoding of the single relevant movement and the eventual recall phase, we also included two additional control conditions where participants experienced only a single movement (i.e., with no subsequent interfering irrelevant movements) but where the maintenance phase lasted the same amount of time as either the one-irrelevant-movement condition (3.7 s) or the three-irrelevant-movements condition (11.1 s). This control allowed us to ask if passive temporal memory decay could explain potential effects observed in the interference conditions and in the SS4 conditions of Exp. 1.

Like Exp. 1, we also included switch and no-switch conditions throughout. All hand switches had to be completed in less than 3 s, and no-switch trials involved a matched delay. The locations of encoded and irrelevant movements were drawn from the same distribution as Exp. 1 and similarly randomized across trials. Subjects performed three blocks of trials with 40 trials per block. Hand switching, the initial encoding hand (left or right), interference load, and delays were evenly counterbalanced. Subjects timed out on 4.4% of trials, and these trials were excluded from further analysis.

Experiment 3: visuospatial interference

Exp. 3 used the same switch/no-switch manipulation as Exp. 1 and Exp. 2; however, we interleaved a separate, concurrent visuospatial working memory task with the motor memory task, creating a dual-task situation (Fig. 3A). Each trial had five phases: (1) encode a single passive reaching movement (same encoding procedure as Exp. 2, no visual stimuli); (2) encode one or three visuospatial working memory stimuli (visual information only, details to follow); (3) maintain both motor and visuospatial working memory task information and either switch hands or wait, per instructions; (4) recall the encoded movement (same as Exp. 2); and finally, (5) recall the location of a single cued visuospatial working memory stimulus. Exp. 3 was broken into three experimental blocks, two blocks of 48 trials and one with 24 trials. Participants also had an eight-trial practice block to familiarize themselves with the task.

For the visuospatial working memory task, participants encoded the location of one or three colored circles (dots) on the display, with SS1 and set size 3 (SS3) trials randomly interleaved. Each dot was 1 cm in diameter and filled with one of six color-blind-friendly colors. The dots were arranged on an invisible circle with a diameter of 12 cm, and the center of the circle was positioned 42 cm away from the participant. In SS3 trials, each visuospatial working memory stimulus was at least ±45° from its nearest neighbor, and no positions were repeated across trials. Critically, the workspace for the visuospatial working memory task matched the workspace for the motor memory task: Motor memory trajectories also terminated on the invisible ring, and probed reach angles could not be less than ±45° from any of the simultaneously remembered visuospatial working memory stimuli on a given trial.

During the visuospatial working memory encoding phase (Phase 1), participants were instructed to memorize the color and location of the dot(s). After encoding a single movement (Phase 2), during maintenance (Phase 3), participants needed to hold both movement information and visuospatial information in working memory. The maintenance phase lasted 4 s, and in half of the trials, participants were instructed to switch hands. Note that instead of resting the unused hand on the table, they held a joystick (separate joysticks were fastened to each side of the robot table). When instructed, participants then reported the remembered movement in the same manner as Exp. 1 and Exp. 2 (Phase 4). Finally, in Phase 5, participants rotated the relevant joystick to report the remembered location of a single cued visuospatial working memory stimulus. That is, one of the visuospatial working memory stimuli from the encoding phase was displayed (or the sole stimulus on SS1 trials) at the center of the workspace. The first sensed movement of the joystick “teleported” the cued stimulus onto the invisible ring, and participants then rotated the joystick to navigate the stimulus around the ring to where they recalled that it had originally appeared. Once satisfied, participants clicked the joystick trigger to mark their choice. Participants were required to recall only one visuospatial working memory stimulus per trial. The visuospatial working memory recall occurred after the movement recall in order to avoid interference from joystick movements on movement recall.

Analysis

All analyses were performed in R (R Core Team, 2023). Euclidean error was computed as the distance between the reported movement end point and actually encoded end point. Response times were defined by the time elapsed between the recall cue and the first moment at which the participant moved out of the home position. Outlier trials (presumed “lapses”) were rare, defined as any recalled movements with ±30° angular error for SS1 trials in Exp. 1 (1.58%) and all trials in Exp. 2 (2.35%). A ±90° cutoff was used for SS4 trials in Exp. 1 (1.48%) and all trials in Exp. 3 (2.27%). These trials were removed prior to our main analyses. For the visuospatial working memory dual task (Exp. 3), memory precision in each condition was quantified by taking 1/SD, where SD is the circular standard deviation of the angular errors of people’s spatial memory reports.

Two-tailed paired t tests were used for two-way comparisons of interest (e.g., the effect of set size on error). For all t tests, our critical alpha was set a priori at 0.05. Our primary analyses of interest were focused on the hand-switch costs observed across different experimental conditions (e.g., recalling movements from different positions within the encoding sequence in Exp. 1). To that end, in Exp. 1 we performed a two-way repeated-measures ANOVA on the main effect of encoding position on the observed switch costs, and in Exp. 2 we performed two-tailed t tests comparing switch costs between the two levels of interference (i.e., one vs. three irrelevant movements) and the short and long delays of the two delay control conditions. For Exp. 3 we performed t tests to compare visuospatial working memory performance across set sizes and a two-way repeated-measures ANOVA to examine motor memory switch costs across set sizes. Where correlations are reported, Pearson correlation coefficients were used. Effect sizes for t tests are reported as Cohen’s d, and effect sizes for ANOVAs are reported as partial eta squared (η_p²).

Results

Experiment 1

Human subjects performed a MWM task, which asked them to encode reaching movements, maintain those movements in memory, and then recall them to the best of their abilities (Fig. 1). In Exp. 1, we asked subjects (N = 20 after exclusions) to encode one or four movements and then recall the single encoded movement (SS1) or one of the four encoded movements (SS4). Recall was performed with either the same hand they encoded the movements with (no-switch condition) or after switching hands (switch condition). We reasoned that in the switch condition, active, effector-specific information can no longer be directly used by the working memory system, allowing us to better isolate an “abstract” component of MWM that can be maintained in an effector-independent code.

Set size had a significant impact on the fidelity of MWM (Fig. 2A)—overall recall errors in the SS4 condition were significantly greater than in the SS1 condition, t(19) = 7.43, p < .001, d = 1.66. Although not unexpected, these results are the first, to our knowledge, to demonstrate classic cognitive load effects in a purely nonvisual MWM task. Moreover, though perhaps also unsurprisingly, switching hands prior to recall also had a deleterious effect on performance (Fig. 2A), with greater recall errors in the switch versus no-switch condition across both set sizes: SS1 overall switch effect, t(19) = 5.44, p < .001, d = 1.22; SS4 overall switch effect, t(19) = 3.53, p = .002, d = 0.79.

Fig. 2.

Results. (A) Experiment 1 main results. Top: Average errors in each condition, in centimeters. Unfilled circles represent set size = 1 trial (“SS1”). Lines refer to set size = 4 trials (“SS4”) and are arranged from left to right according to the position the recalled movement had been encoded during the sequence of four encoded movements (i.e., data at the far left refer to when the first or oldest encoded movement is recalled, and at the far right the last or most recent movement is recalled). Bottom: Hand-switch costs were computed as the difference between average switch and no-switch errors across each condition. A switch cost of 0 indicates that there was no benefit of recalling a movement with the same arm that encoded it. (B) Experiment 2 interference results. Top: Average errors when one versus three extra (irrelevant) movements occurred following encoding of a single movement. Bottom: Hand-switch costs for the interference data. (C) Experiment 2 delay results. Top: Average errors when a short versus long passive decay period directly followed encoding. Note that the short and long delay corresponded to the amount of time it took for one or three extra movements in the interference condition (i.e., 3.7 or 11.1 s, respectively). Bottom: Hand-switch costs for the delay data. Asterisks in the top row reflect comparisons between switch and no-switch conditions. All error bars = 1 SEM. *p < .05. **p < .01. ***p < .005.

We also observed numerical proactive interference effects—errors were, on average, smaller in the SS1 condition relative to when the most recently encoded movement was queried in the SS4 condition (this comparison controls for the amount of time between the end of encoding and recall), t(19) = 1.99, p = .06, d = 0.44, though this result was not statistically significant. We also note here that average Euclidean error and variable error (dispersion) were highly correlated (ρ = 0.99 across subjects), suggesting that systematic biases in participants’ memory errors did not affect our error analyses. Taken together, these results suggest that MWM is susceptible to load and mnemonic interference, similar to other types of working memory. And, in light of the significant switch costs, the results also suggest that some aspect of MWM might be stored in an effector-specific format.

How can we dissociate the memory dynamics of putative effector-specific and effector-independent components of MWM? We reasoned that in the SS4 condition, the position of an encoded movement within the encoding sequence would modulate the cost of switching hands via retrograde interference, with movements encoded early relative to the recall phase showing reduced switch costs relative to those encoded closer to recall. Switch costs were calculated by subtracting the mean participant no-switch error from their switch error. To that end, our primary analysis focused on the role that sequence position played in hand-switch costs in the SS4 condition—if hand-switch costs were uniform across the sequence position of the four movements encoded into MWM, this would suggest that the disruption of switching hands elicited a fixed cost on performance, or at least suggest that effector-specific memory is robust to interference (and/or temporal decay). However, partly inspired by concepts of iconic sensory memory versus proper working memory in vision (Sperling, 1960), we expected effector-specific MWM to be especially vulnerable to retrograde interference at the sensory level (e.g., actively maintained somatosensory traces for the encoding arm). Thus, recently encoded movements (e.g., the last movement encoded during the sequence) should benefit from a still-active effector-specific memory trace, whereas effector-specific memory of older movements (e.g., the first movement encoded) should be strongly disrupted by retrograde interference from the intervening movements. This prediction would be confirmed by seeing greater relative hand-switch costs for more recently encoded movements.

This prediction was supported by our analysis of the effect of sequence position on hand-switch costs (Fig. 2A)—switch costs were seen for the most recently encoded movement (fourth position), t(19) = 3.86, p = .001, d = 0.86, and, surprisingly, the second-most-recently encoded movement (third position), t(19) = 3.14, p = .005, d = 0.70. Notably, no significant hand-switch costs were observed for the movements encoded first and second in the sequence: first position, t(19) = 0.26, p = .80, d = 0.06; second position, t(19) = −1.23, p = .23, d = −0.28. Crucially, our key ANOVA analysis revealed a main effect of encoding position on hand-switch costs in the predicted direction, F(1, 19) = 11.41, p = .003, η_p² = 0.38.

These findings suggest that recently encoded movements appear to be more strongly represented in effector-specific MWM versus movements encoded further in the past. This supports the idea that effector-specific MWM reflects a maintained sensory trace tied to the encoding limb. However, our interpretation of the results of Exp. 1 raises a question: Was retrograde interference the deciding factor for the observed attenuation of effector-specific working memory, or was it simply the passage of time? That is, movements encoded early in the encoding phase of SS4 trials were also further in the past at the time of recall, suggesting that time, interference, or perhaps both factors contribute to the observed switch-cost pattern. Moreover, retrograde interference could be occurring because of the cognitive load of maintaining multiple active MWMs or because of sensory retrograde interference driven by additional kinesthetic inputs at the encoding limb. Indeed, the latter type of interference would be in line with our hypothesis that effector-specific MWM is primarily sensory in nature.

Experiment 2

In Exp. 2, we asked if putative effector-specific MWM was susceptible to motor interference and/or passive temporal decay. We operationalized interference in Exp. 2 by introducing irrelevant passive movements (i.e., movements not explicitly encoded into memory) that occurred after a to-be-remembered movement (see Method). That is, we asked if experiencing a movement at the encoding limb disrupted effector-specific MWM irrespective of cognitive load. We also included control conditions that tested the effect of passive temporal decay on MWM without interfering movements. We considered three potential outcomes and interpretations: If increasing the number of irrelevant interfering movements and increasing the length of temporal delays both did not modulate hand-switch costs, we could assume that the effector-specific retrograde interference effects seen in Exp. 1 were primarily related to the active maintenance of multiple movements in MWM and not to sensory interference or temporal decay. On the other hand, if reduced switch costs accompanied both increased interference and increased delays, we could assume that both factors (or temporal decay alone) contribute to retrograde interference effects. Finally, if we see that increasing the number of interfering movements reduces switch costs but see no such effect for delay, we could assume that the state of sensory information at the encoding limb is the key factor that determines the fidelity of effector-specific MWM.

We asked subjects (N = 24 after exclusions) to encode a single movement and recall that movement with either the same hand that they encoded the movement with (no-switch condition) or their other hand (switch condition). The recall phase followed the encoding phase after a variable amount of irrelevant interfering movements (one vs. three) or after a matched interval of time during which no irrelevant movements were experienced and the subject simply held still (Fig. 1C).

Replicating Exp. 1, hand switching produced a large and significant negative overall effect on recall: Figs. 2B and 2C, interference condition, t(23) = 6.59, p < .001, d = 1.34; Figs. 2E and 2F, delay condition, t(23) = 4.60, p < .001, d = 0.94. Turning to the primary effect of interest, we observed that increasing the number of irrelevant interfering movements from one to three was accompanied by a decrease in hand-switch costs, t(23) = 2.75, p = 0.01, d = 0.59. Increasing the number of irrelevant interfering movements did not magnify errors in the hand-switch condition, t(23) = 0.99, p = .33, d = 0.20, a condition that putatively isolates effector-independent MWM. In contrast, temporal delay on its own did not have a significant effect on hand switch costs, t(23) = −0.11, p = .91, d = −0.03. Together, these results point to sensory factors at the encoding limb, not passive decay, as the primary determinant in effector-specific MWM fidelity.

Importantly, although the effect of sensory interference on MWM switch costs was robust, it cannot explain the entirety of the interference effects observed in Exp. 1. That is, the results of Exp. 2 cannot explain the complete abolition of switch costs seen in the earliest-encoded movements in Exp. 1 (Fig. 2A). Thus, retrograde interference due to actively encoding movements into MWM must have also played some role in the results of Exp. 1. This suggests that effector-specific MWM shows a unique sensitivity to interference from competing sensory inputs but is also sensitive to cognitive load. Taken as a whole, our results point to two dissociable aspects of MWM: effector specific and effector independent.

Experiment 3

It is possible that our hypothesized effector-independent MWM code is homologous to a general spatial working memory system and does not necessarily reflect “motor” memory. On the other hand, this memory code may be distinct from generic spatial working memory. To test this, we used a dual-task design to ask if visuospatial working memory load interferes with MWM.

In this experiment participants (N = 14 after exclusions) had to simultaneously maintain both visuospatial working memory information and MWM information (Fig. 3A). We considered three possible outcomes: (a) If a large component of MWM measured in our task is simply a generic visuospatial working memory system that encodes the final hand position, we should see a significant performance reduction in MWM when the spatial working memory load is increased. (b) Alternatively, if only the effector-independent component of MWM overlaps with visuospatial working memory, an interference effect should be especially pronounced under hand-switch conditions. (c) Last, if MWM and spatial working memory are dissociable, we should see little to no disruption of MWM performance when the simultaneous spatial memory load is increased.

Fig. 3.

Experiment 3 task design and results. (A) Participants encoded a single movement, similar to Experiment 2. They were then shown one or three colored circles on an invisible ring and were instructed to memorize their location and color. During maintenance, participants had to hold both the movement and visual information in working memory. In half of the trials during maintenance, participants were prompted to switch which hand held the robot manipulandum. Participants then recalled the remembered movement. Finally, to recall the visual target, one of the colored circles would appear in the middle of the screen, and a joystick was then used to navigate to where they believed it was originally shown. Note that vision of the hands and arms was fully occluded throughout the task. (B) Recall precision for the visuospatial task was lower when participants were required to hold more items in visuospatial working memory. (C) Recall error in the motor memory task increased under hand switching but was not significantly affected by visuospatial working memory load. (D) Hand-switch cost for movement error showed no interaction with visuospatial working memory load. (E) Switch costs in the variance of movement direction recall was also not significantly affected by increased visuospatial working memory load. All error bars = 1 SEM. *p < .05. **p < .01. ***p < .005.

We first confirmed that increasing the visuospatial working memory load reduced the precision of visuospatial working memory itself (Fig. 3B): We observed a significant effect of load on visuospatial working memory precision, t(13) = 3.46, p = .004, d = 0.93, echoing classic visual working memory set size effects. To address our primary question of interest, we examined how an increased visuospatial load affects MWM. An ANOVA revealed a large effect of hand switching on MWM error (Fig. 3C), mirroring the results of Exp. 1 and Exp. 2, F(1, 13) = 36.05, p < .001. Crucially, we observed no significant effect of visuospatial working memory load on MWM performance, F(1, 13) = 0.40, p = 0.54, η_p² = 0.03, nor a significant interaction between visuospatial working memory load and hand switching, F(1, 13) = 0.00, p = .996.

The null effect of visuospatial working memory load on hand-switch costs (Fig. 3D) was further quantified using the Bayes factor. We computed a Bayes factor of BF₁₀ = 0.27, providing evidence in support of the null. We also analyzed the effect of visuospatial working memory load on hand-switch costs by looking at the (circular) standard deviation in movement angle recall (Fig. 3E). Again we observed no effect of visuospatial load on switch costs, with a subtle numerical trend in the direction opposite that predicted by interference between effector-independent MWM and visuospatial working memory, t(1, 13) = 0.72, p = .49, d = 0.19. Taken together, the results of Exp. 3 suggest that MWM, at least as measured in our experimental designs, may utilize resources independent of visuospatial working memory.

General Discussion

Working memory research has largely focused on sensory systems, like vision, with motor function primarily acting in conjunction with or as an output system for recoded nonmotor information (Curtis et al., 2004; van Ede et al., 2019). In this study we aimed to look at how movements themselves are encoded and maintained in working memory. To that end, we designed a nonvisual reaching paradigm in which participants passively encoded movement trajectories and then recalled those movements, with either the same or different arm under various load, delay, and interference conditions. We found evidence for dissociable effector-specific and effector-independent MWM codes, each with different sensitivities to load and interference. Furthermore, we found no evidence that MWM is subject to visuospatial interference. Our findings provide a descriptive account of multifaceted MWM functions and lay the groundwork for future behavioral, computational, and neural investigations of this understudied subsystem of human memory.

Many theories take a modular approach to the functions of working memory, positing different storage and/or processing resources based on the type of information maintained (e.g., spatial vs. verbal information; Baddeley, 2003). Exp. 1 and Exp. 2 point to a specific subsystem of MWM that maintains a kinesthetic memory based on sensations at the encoding limb, extending work on movement-specific working memory (Allen et al., 2023; Smyth et al., 1988; Smyth & Pendleton, 1989). Critically, our experiments distinguish between somatosensory memory and a different memory format that transfers across limbs. Is the latter memory code simply spatial working memory? The results of Exp. 3 argue against this, as we found that increasing visuospatial working memory load did not have a significant impact on movement recall. This observed lack of interference between visuospatial working memory and MWM is particularly surprising given evidence that established working memory modules like phonological and visuospatial working memory do, in some circumstances, share resources (Camos, 2017; Morey, 2018). This points to a heretofore underappreciated working memory module that may be used to maintain abstract information about recent motor commands.

Independent of working memory, the distinction between effector-specific and effector-independent codes has also been studied in the context of long-term motor memory, including studies of sequence learning (Bapi et al., 2000; Shea et al., 2011; Wiestler et al., 2014) and motor adaptation (Criscimagna-Hemminger et al., 2003; Joiner et al., 2013; Malfait & Ostry, 2004; Sainburg & Wang, 2002). In the sequence-learning domain, long-term memories for motor sequences have an effector-independent component that allows for transfer of sequence knowledge across hands in extrinsic (world-based) coordinates (Grafton et al., 2002). In motor adaptation transfer tasks, sensorimotor mappings are trained with one limb and then tested on the naive limb (e.g., a force field perturbation may be adapted to by the right arm and then tested on the left arm). Some of this work argues that transferable (effector-independent) long-term motor memories reflect an abstract vector in extrinsic space (Criscimagna-Hemminger et al., 2003). Considering that we implemented a strictly single-trial memory task with no protracted learning, it is not clear if or how previous work on motor learning, including recent discoveries of short-term memory effects in motor adaptation (Hadjiosif et al., 2023), is related to our results. That said, it may be the case that coding formats in MWM are similar to those observed in long-term motor memory, echoing work connecting working memory and long-term memory in nonmotor domains (Ranganath et al., 2005).

How is MWM implemented in the brain? Like any working memory system, it is likely that the prefrontal cortex is critical for people maintaining and manipulating motor information in working memory. Additionally, the aforementioned long-term sequence-learning and transfer research indicates that the premotor and primary motor cortex are associated with effector-independent representations (Bapi et al., 2000; Shea et al., 2011; Wiestler et al., 2014), and frontal and parietal areas have been implicated in abstract trajectory representations (Barhorst-Cates et al., 2022; Wong et al., 2019). Effector-independent MWM likely involves a distributed bilateral circuit that includes both frontal and parietal sensorimotor and association regions. Turning to effector-specific MWM, we speculate that its neural correlates would be more significantly lateralized and more closely linked to somatosensory inputs from the encoding limb. Future work using neurophysiological methods or transcranial magnetic stimulation could be useful for addressing these questions and for building on our psychophysical dissociation between MWM codes by looking for a similar neural dissociation.

Our study leaves open several questions for future investigation. First, all encoded movements were passively administered. Do our results generalize to unconstrained volitional movements? Although the experimental control given to us by the passive method was critical to our design—and more closely aligns with classic methods used for studying working memory—in future work, active movements should be tested. Another potential limitation of our task is the requirement of switching hands on the robot handle. This may have nonspecifically increased errors in hand-switch conditions. Although this potential source of error does not confound our key results, future studies, perhaps using separate robotic arms held by each hand, could reduce this possible source of noise. Additionally, we tested movement kinematics only in terms of straight-line spatial trajectories with end points; future studies could investigate working memory for other components of movement, such as force, velocity, or more complicated trajectories. Examining the role of handedness in MWM may also be useful in characterizing individual differences and shedding new light on theories of handedness. Finally, our sample was limited to young adult undergraduates; how our results may generalize (or change) in the context of, for instance, aging or neurological disease is unclear. For example, working memory performance is known to decline with aging—would the same result hold for MWM?

Finally, here we have used a definition of working memory that encompasses both short-term maintenance and manipulation of movements; it would be useful to directly examine more complex manipulations of information in MWM to better link it to more manipulation-centric conceptions of working memory. Overall, investigating MWM will help us better understand the interface between cognition and motor control.

Footnotes

Acknowledgements

We thank Taylor McClure and Samantha Goodcase for helping with data collection. We also thank the ACT Lab and Naser Al-Fawakhiri for helpful discussions.

Transparency

Action Editor: Krishnankutty Sathian

Editor: Patricia J. Bauer

Author Contributions

Hanna Hillman: Conceptualization; Data curation; Formal analysis; Investigation; Methodology; Visualization; Writing – original draft; Writing – review & editing.

Tabea Botthof: Investigation; Methodology.

Alexander D. Forrence: Investigation; Methodology; Software.

Samuel D. McDougle: Conceptualization; Formal analysis; Project administration; Supervision; Writing – review & editing.

ORCID iDs

Hanna Hillman

Alexander D. Forrence

References

Allen

R. J.

Waterman

A. H.

Yang

Graham

A. J.

(2023). Working memory in action: Remembering and following instructions. In Logie

Cowan

Gathercole

Engle

Wen

(Eds.), Memory in science for society (1st ed., pp. 241–C10P193). Oxford University Press. https://doi.org/10.1093/oso/9780192849069.003.0010

Atkinson

R. C.

Shiffrin

R. M.

(1971). The control of short-term memory. Scientific American, 225(2), 82–91.

Baddeley

(2003). Working memory: Looking back and looking forward. Nature Reviews Neuroscience, 4(10), Article 10. https://doi.org/10.1038/nrn1201

Bapi

R. S.

Doya

Harner

A. M.

(2000). Evidence for effector independent and dependent representations and their differential time course of acquisition during motor sequence learning. Experimental Brain Research, 132(2), 149–162. https://doi.org/10.1007/s002219900332

Barhorst-Cates

E. M.

Isaacs

M. W.

Buxbaum

L. J.

Wong

A. L.

(2022). Action imitation via trajectory-based or posture-based planning. Human Movement Science, 83, 102951. https://doi.org/10.1016/j.humov.2022.102951

Bays

P. M.

Wolpert

D. M.

(2006). Actions and consequences in bimanual interaction are represented in different coordinate systems. Journal of Neuroscience, 26(26), 7121–7126. https://doi.org/10.1523/JNEUROSCI.0943-06.2006

Bernardi

N. F.

Darainy

Ostry

D. J.

(2015). Somatosensory contribution to the initial stages of human motor learning. Journal of Neuroscience, 35(42), 14316–14326. https://doi.org/10.1523/JNEUROSCI.1344-15.2015

Brayanov

J. B.

Press

D. Z.

Smith

M. A.

(2012). Motor memory is encoded as a gain-field combination of intrinsic and extrinsic action representations. Journal of Neuroscience, 32(43), 14951–14965. https://doi.org/10.1523/JNEUROSCI.1928-12.2012

Camos

(2017). Domain-specific versus domain-general maintenance in working memory. In Ross

B. H.

(Ed.), The Psychology of learning and motivation (Vol. 67, pp. 135–171). Elsevier. https://doi.org/10.1016/bs.plm.2017.03.005

10.

Cohen

D. A.

Pascual-Leone

Press

D. Z.

Robertson

E. M.

(2005). Off-line learning of motor skill memory: A double dissociation of goal and movement. Proceedings of the National Academy of Sciences, USA, 102(50), 18237–18241. https://doi.org/10.1073/pnas.0506072102

11.

Criscimagna-Hemminger

S. E.

Donchin

Gazzaniga

M. S.

Shadmehr

(2003). Learned dynamics of reaching movements generalize from dominant to nondominant arm. Journal of Neurophysiology, 89(1), 168–176. https://doi.org/10.1152/jn.00622.2002

12.

Curtis

C. E.

Rao

V. Y.

D’Esposito

(2004). Maintenance of spatial and motor codes during oculomotor delayed response tasks. Journal of Neuroscience, 24(16), 3944–3952. https://doi.org/10.1523/JNEUROSCI.5640-03.2004

13.

Grafton

S. T.

Hazeltine

Ivry

R. B.

(2002). Motor sequence learning with the nondominant left hand. Experimental Brain Research, 146(3), 369–378. https://doi.org/10.1007/s00221-002-1181-y

14.

Hadjiosif

A. M.

Morehead

J. R.

Smith

M. A.

(2023). A double dissociation between savings and long-term memory in motor learning. PLOS Biology, 21(4), Article e3001799. https://doi.org/10.1371/journal.pbio.3001799

15.

Jaroslawska

A. J.

Gathercole

S. E.

Holmes

(2018). Following instructions in a dual-task paradigm: Evidence for a temporary motor store in working memory. Quarterly Journal of Experimental Psychology, 71(11), 2439–2449. https://doi.org/10.1177/1747021817743492

16.

Joiner

W. M.

Brayanov

J. B.

Smith

M. A.

(2013). The training schedule affects the stability, not the magnitude, of the interlimb transfer of learned dynamics. Journal of Neurophysiology, 110(4), 984–998. https://doi.org/10.1152/jn.01072.2012

17.

Jones

S. A. H.

Henriques

D. Y. P.

(2010). Memory for proprioceptive and multisensory targets is partially coded relative to gaze. Neuropsychologia, 48(13), 3782–3792. https://doi.org/10.1016/j.neuropsychologia.2010.10.001

18.

Keele

S. W.

Ells

J. G.

(1972). Memory characteristics of kinesthetic information. Journal of Motor Behavior, 4(3), 127–134. https://doi.org/10.1080/00222895.1972.10734928

19.

Krakauer

J. W.

Ghilardi

M.-F.

Ghez

(1999). Independent learning of internal models for kinematic and dynamic control of reaching. Nature Neuroscience, 2(11), Article 11. https://doi.org/10.1038/14826

20.

Allen

R. J.

Hitch

G. J.

Baddeley

A. D.

(2022). Translating words into actions in working memory: The role of spatial-motoric coding. Quarterly Journal of Experimental Psychology, 75(10), 1959–1975. https://doi.org/10.1177/17470218221079848

21.

Liu

Vannuscorps

Caramazza

Striem-Amit

(2020). Evidence for an effector-independent action system from people born without hands. Proceedings of the National Academy of Sciences, USA, 117(45), 28433–28441. https://doi.org/10.1073/pnas.2017789117

22.

Luck

S. J.

Vogel

E. K.

(2013). Visual working memory capacity: From psychophysics and neurobiology to individual differences. Trends in Cognitive Sciences, 17(8), 391–400. https://doi.org/10.1016/j.tics.2013.06.006

23.

Malfait

Ostry

D. J.

(2004). Is interlimb transfer of force-field adaptation a cognitive response to the sudden introduction of load? Journal of Neuroscience, 24(37), 8084–8089. https://doi.org/10.1523/JNEUROSCI.1742-04.2004

24.

McDougle

S. D.

Taylor

J. A.

(2019). Dissociable cognitive strategies for sensorimotor learning. Nature Communications, 10(1), Article 1. https://doi.org/10.1038/s41467-018-07941-0

25.

Morey

C. C.

(2018). The case against specialized visual-spatial short-term memory. Psychological Bulletin, 144(8), 849–883. https://doi.org/10.1037/bul0000155

26.

Oldfield

R. C.

(1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9(1), 97–113. https://doi.org/10.1016/0028-3932(71)90067-4

27.

Posner

M. I.

Konick

A. F.

(1966). Short-term retention of visual and kinesthetic information. Organizational Behavior and Human Performance, 1(1), 71–86. https://doi.org/10.1016/0030-5073(66)90006-7

28.

Ranganath

Cohen

M. X.

Brozinsky

C. J.

(2005). Working memory maintenance contributes to long-term memory formation: Neural and behavioral evidence. Journal of Cognitive Neuroscience, 17(7), 994–1010. https://doi.org/10.1162/0898929054475118

29.

Raw

R. K.

Wilkie

R. M.

Allen

R. J.

Warburton

Leonetti

Williams

J. H. G.

Mon-Williams

(2019). Skill acquisition as a function of age, hand and task difficulty: Interactions between cognition and action. PLOS ONE, 14(2), Article e0211706. https://doi.org/10.1371/journal.pone.0211706

30.

R Core Team. (2023). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/.

31.

Sainburg

R. L.

Wang

(2002). Interlimb transfer of visuomotor rotations: Independence of direction and final position information. Experimental Brain Research, 145(4), 437–447. https://doi.org/10.1007/s00221-002-1140-7

32.

Scheidt

R. A.

Ghez

(2007). Separate adaptive mechanisms for controlling trajectory and final position in reaching. Journal of Neurophysiology, 98(6), 3600–3613. https://doi.org/10.1152/jn.00121.2007

33.

Seemüller

Fiehler

Rösler

(2011). Unimodal and crossmodal working memory representations of visual and kinesthetic movement trajectories. Acta Psychologica, 136(1), 52–59. https://doi.org/10.1016/j.actpsy.2010.09.014

34.

Shea

C. H.

Kovacs

A. J.

Panzer

(2011). The coding and inter-manual transfer of movement sequences. Frontiers in Psychology, 2, Article 52. https://doi.org/10.3389/fpsyg.2011.00052

35.

Sidarta

van Vugt

F. T.

Ostry

D. J.

(2018). Somatosensory working memory in human reinforcement-based motor learning. Journal of Neurophysiology, 120(6), 3275–3286. https://doi.org/10.1152/jn.00442.2018

36.

Smyth

M. M.

Pearson

N. A.

Pendleton

L. R.

(1988). Movement and working memory: Patterns and positions in space. Quarterly Journal of Experimental Psychology Section A, 40(3), 497–514. https://doi.org/10.1080/02724988843000041

37.

Smyth

M. M.

Pendleton

L. R.

(1989). Working memory for movements. Quarterly Journal of Experimental Psychology Section A, 41(2), 235–250. https://doi.org/10.1080/14640748908402363

38.

Sperling

(1960). The information available in brief visual presentations. Psychological Monographs: General and Applied, 74, 1–29. https://doi.org/10.1037/h0093759

39.

Vaillancourt

D. E.

Russell

D. M.

(2002). Temporal capacity of short-term visuomotor memory in continuous force production. Experimental Brain Research, 145(3), 275–285. https://doi.org/10.1007/s00221-002-1081-1

40.

van Ede

Chekroud

S. R.

Stokes

M. G.

Nobre

A. C

. (2019). Concurrent visual and motor selection during visual working memory guided action. Nature Neuroscience, 22(3), 477–483. https://doi.org/10.1038/s41593-018-0335-6

41.

Wiestler

Waters-Metenier

Diedrichsen

(2014). Effector-independent motor sequence representations exist in extrinsic and intrinsic reference frames. Journal of Neuroscience, 34(14), 5054–5064. https://doi.org/10.1523/JNEUROSCI.5363-13.2014

42.

Wong

A. L.

Jax

S. A.

Smith

L. L.

Buxbaum

L. J.

Krakauer

J. W.

(2019). Movement imitation via an abstract trajectory representation in dorsal premotor cortex. Journal of Neuroscience, 39(17), 3320–3331. https://doi.org/10.1523/JNEUROSCI.2597-18.2019