Abstract
BACKGROUND:
One of the most interesting emerging medical devices is the medical avatar - a digital representation of the patient that can be used toward myriad ends, the full potential of which remains to be explored. Medical avatars have been instantiated as telemedical tools used to establish a representation of the patient in tele-space, upon which data about the patient’s health can be represented and goals and progress can be visually tracked. Manipulation of the medical avatar has also been explored as a means of increasing motivation and inducing neural plasticity.
OBJECTIVE:
The article reviews the literature on body representation, simulation, and action-observation and explores how these components of neurorehabilitation are engaged by an avatar-based self-representation.
METHODS:
Through a review of the literature on body representation, simulation, and action-observation and a review of how these components of neurorehabilitation can be engaged and manipulated with an avatar, the neuroplastic potential of the medical avatar is explored. Literature on the use of the medical avatar for neurorehabilitation is also reviewed.
RESULTS:
This review demonstrates that the medical avatar has vast potentialities in neurorehabilitation and that further research on its use and effect is needed.
Introduction
There is a growing body of literature in support of the use of avatar-based and virtual reality (VR)-based neurorehabilitation (Cho et al., 2012; Henderson et al., 2007; Holden et al., 1999, 2007; Holden & Dyar, 2002; Merians et al., 2002; Saposnik & Levin, 2011). Many of these studies suggest that the efficacy of VR therapies may be due to a cortical plasticity induced by an avatar-mediated engagement of the cortical body map and/or action-observation network (August et al., 2006; Llorens et al., 2017; Prochnow et al., 2013; Saposnik et al., 2010). And there is a growing body of neuroimaging-based studies that support the idea that these therapies induce neuroplastic changes (Adamovich et al., 2009; August et al., 2006; Jang et al., 2005; Merians et al., 2009; Saleh et al., 2011, 2013; Tunik et al., 2013, 2011; You, Jang, Kim, Hallett, et al., 2005; You, Jang, Kim, Kwon, et al., 2005). And while the idea that the neurorehabilitation of stroke related injuries relies on cortical plasticity is well developed and supported (Bütefisch et al., 2006; Cramer et al., 2011; Frost et al., 2003; Hara, 2015; Hermann & Chopp, 2012; Hodics et al., 2006; Murphy & Corbett, 2009; Nudo, 2007) well developed definitions of the mechanisms and networks behind the cortical body map (upon which this cortical plasticity is said to play out) have remained somewhat more illusive. This article will provide a preliminary background on the current understanding of cortical body representation, including a discussion of the ‘body map’ and the more recent ‘body matrix’, and will explore the literature on self-representation and self-identification, as conceptualized as embodiment, ownership, and agency, as well as action-observation based therapies. Understanding these concepts and therapeutic mechanisms is essential when addressing the neurological activation associated with the use of a medical avatar. However, just as the primary goal of rehabilitation is to improve performance, what is of ultimate interest here is which variations on avatar engagement and bodily representation will maximally enhance neuroplastic changes, and thereby, improve functional recovery for the patient. Therefore, the remainder of the article will address literature that is suggestive of ways that virtual manipulations of the medical avatar and its environment can drive activation and repair through the neuroplastic engagement of the networks associated with the body matrix, self-identification, and action-observation networks.
Avatar based rehab has been explored for use with many conditions, such as stroke rehabilitation (Adamovich et al., 2005; Connelly et al., 2010; Gaggioli et al., 2006; Jack et al., 2001; Lam et al., 2006; Saposnik & Levin, 2011; Saposnik et al., 2010); cerebral palsy (Bryanton et al., 2006; Fluet et al., 2010; Wille et al., 2009); Autism (Mitchell et al., 2007, 2007; Moore et al., 2005; Wang & Reid, 2011) and clinical psychology (Gaggioli et al., 2003; Lee et al., 2003; Leff et al., 2014; Rizzo et al., 2011). The medical avatar can take many forms. A representation of the patient’s hands, a partial medical avatar, has been used for various rehabilitation programs (Adamovich et al., 2005; Badia et al., 2013; Cameirão et al., 2010), while other programs use a full body representation (Donati et al., 2016). In some cases the medical avatar is seen from a third person perspective (3PP), where the body is fully represented in the environment, or a first person perspective, where just the hands are visible. The first person perspective (1PP) can also be such that the user’s body or avatar is entirely implied and the focus is on interactions with social others, as is the case with some autism studies, or the focus is directed out toward the environment, as is the case in some Alzheimer’s studies (Serino et al., 2015). Importantly, these different representations of the virtual self are a means by which the medical avatar can be manipulated, with potentially meaningful neurological impacts on neurorehabilitation. From a God’s eye view of the self below, to an over the shoulder “out-of-body” perspective, to a mirror image, or the implicit avatar, each of these different perspectives on the medical avatar could each differentially activate brain regions associated with neurorehabilitation. Therefore, it may be critically important, when considering the design of, say, a virtual stroke rehabilitation program, whether the patient sees only their virtual hands, or their entire virtual body, and if that virtual body is in mirror image to their movements or not, or if they are given multiple self-representations such as a 1st person close up of the hands and a 3rd person full body mirror image. This review intends to highlight some of the manipulations allowed by the medical avatar and how these manipulations might differentially activate the neurological networks associated with neurorehabilitation.
Background
Body representation
The question of the cortical representation of the self and the body has been explored through experimental manipulations of various defining aspects of self and embodiment, such as body schematics, afferent and efferent sensory signals, agency, ownership and perspective. Some evocative examples include the rubber hand illusion (RHI) (Ehrsson et al., 2007; Makin et al., 2008), the virtual hand illusion (VHI) (Ma & Hommel, 2015b), the body swap illusion, or full body transfer illusion (BSI) (Petkova & Ehrsson, 2008), the virtual body swap illusion (VBSI) (Gentile et al., 2015), as well as the induction of out of body experiences (OBE’s) (Aspell et al., 2012; Blanke, 2012; Blanke et al., 2005). Body representation can be broken down into the body schema and body image, the former being the multisensory sum of afferent and efferent information, and the latter being a permanent mapping of the body, also known as the body map (Kammers et al., 2009; Ma & Hommel, 2015a; Schwoebel & Coslett, 2005). These, the body map and body schema can be contrasted and manipulated across the dimensions of self-identification, agency, and ownership, as well as sense of location, relationship to environment, proprioception, and movement, and the simulation of these sensations (Holmes & Spence, 2006). A further contrast exists between top-down and bottom-up approaches. The top-down approach would argue that the illusions mentioned above (RHI, VHI, BSI) are driven by a change in a mental simulation or mapping of the body, in other words, changes to the body image. Research in support of this approach would include Ehrsson’s 2008 study that found that the illusion of rubber hand ownership in amputees was more easily and strongly induced in more recent amputees, suggesting that the longer a person experiences their body without a limb, the less plastic becomes their top down representation of their own body, making it less likely that they will accept a rubber hand, regardless of the illusory sensory information, as their own, as it now no longer lines up with a top-down representation of their own body (Ehrsson et al., 2008). Whereas, the bottom-up approach would argue that there is no static representation of the body, no body image, there is only the body schema, which is created through the multisensory integration of afferent and efferent stimuli. Therefore the illusion is driven by the sensory information being fed into the brain and the body schema is created thereby, rather than a top-down cortical representation driving the interpretation of that sensory information. A further example of the top-down/bottom-up debate can be found in the debate surrounding congenitally missing limbs (V. S. Ramachandran, 1999). Here a top-down approach would predict that people with congenitally missing limbs would not experience phantom limb (Simmel, 1961) whereas a bottom-up argument would predict they could (Melzack et al., 1997). If the top-down approach were correct it would follow that an object different enough from the mental representation of the body would inhibit the induction of the RHI/VHI. Tsakris champions this approach, producing research indicating there is a degree to which an object can differ from anatomical, spatial, and postural constraints, before it is rejected as a possible match to the internally held body image (Tsakiris and Haggard 2005; Tsakiris et al. 2010; Constantini and Haggard, 2007). However, in favor of a purely bottom-up approach evidence has accumulated to suggest that the RHI can be induced using a block of wood or a table top in place of the rubber hand (Armel and Ramachandran 2003), or even empty spaces (Guterstam, Gentile, and Ehrsson 2013), or with hands of a different skin tone (Farmer, Tajadura-Jiménez, and Tsakiris 2012) or skin luminance (Longo et al. 2009), suggesting that a contrasting top-down body image does not limit the generation of ownership. Indeed the VBSI was even achieved with the illusion of transfer to a body of a different gender (Slater et al. 2010) and Ma and Hommel are able to generate the illusion of body-ownership over a virtual balloon and a virtual square (Ma and Hommel 2015a). Furthermore, data from Press et al. provided neurophysiological evidence that ownership was achieved in both a rubber hand and a non-hand object (Press et al., 2007). These studies would suggest that there is no set, stable, deterministic body-image, but rather, there is a bottom-up integration of multisensory information supporting an ever-unfolding simulation of a body schema.
The body matrix: A multisensory integration
Previous reviews have brought together research across domains and advocate a meta approach that incorporates top-down and bottom-up theories of bodily representation, and that additionally incorporates peripersonal space. Constantini and Haggard posit a ‘body matrix’ that allows for a unique understanding of multiple body schemas and body images and their many dimensions. The body matrix allows that there are multiple body schema, with attributions of ownership, and agency, and self-identification applied to a self-perspective or object in peripersonal or simulated space (Constantini & Haggard, 2007). Moseley, Gallace and Spence propose “a network of multisensory and homeostatic brain areas” they call a ‘body-matrix’ and define as a “dynamic neural representation that not only extends beyond the body surface to integrate both somatotopic and peripersonal sensory data, but also integrates body-centered spatial sensory data.” (Moseley et al., 2012, p. 34). Recent research in support of this approach reported that the human somatosensory cortex extracts touch location from a tool using the same processes involved in body mapping, indicating that humans indeed extend the sense of embodiment out into peripersonal space and build a flexible and extended body matrix (Miller et al., 2019).
The functional nature, and clinical manipulation, of such an integrated body matrix hinges on the relative weighting of multisensory information. For instance, Kammers et al. reported on an experiment on the RHI with an action component. They found that proprioception is given more weight than vision, when making determinations for the body in action. In contrast, when the body image is used for perceptual judgement this is reversed (Kammers et al., 2009); thereby suggesting that there are multiple representations of the body, depending on the domain of interest, each with their own multisensory integration weight to contribute to the final balance. Another study by Kammers, using Transcranial Magnetic Stimulation (TMS), puts forward evidence for a dissociation between cortical representations of embodiment/localization, and action. The authors report their findings “concur with a multicomponent model of somatosensory body representations” wherein the inferior parietal lobule is engaged with perceptual body judgments but not actions (Kammers et al., 2009). In support of this multi-component interpretation is research by Schwoebel and Coslett, wherein of 70 stroke patients 51% were found to be impaired on a body representation measure, and a lesion analysis revealed that left temporal lobe lesions were associated with impaired performance describing body parts and lesions in the dorsolateral frontal and parietal regions with “on-line coding of body posture”. The authors propose three types of body representations, “a dynamic representation of relative body positions” derived from multisensory integration, a “topological map [of] body part boundaries and relations” and the third a dynamic image of the body and artifacts associated with it (Schwoebel & Coslett, 2005). To these representations ownership and agency are then attributed as determined by the integration of afferent multisensory information.
There is a rich history of literature exploring the relative weighting of this multisensory information in body representation, i.e vision vs. proprioception (Graziano, 1999), vision vs. somatosensation (Austen et al., 2004) (Austen et al. 2004). (For a review see (Holmes & Spence, 2006.) However the weighting happens, it seems to include in the equation the contribution of top-down attributions, such as ownership, and agency. For instance, a study by Pavani, Spence and Driver found that the magnitude of the visuotactile interactions could be modulated by ownership (Spence et al., 2000). A review of studies on multisensory integration and body schema, looking at both animal and human studies, supports the idea that there are various plastic body centered representations of space which rely on the multisensory integration from visual, tactile and proprioceptive afferent and efferent information (Maravita et al., 2003). A later review of clinical studies, led the reviewers to suggest that “ bodily self-consciousness” could be dissociated into “three key components: self-location, first-person perspective, and self-identification”. The authors continue to argue that this research into RHI, BSI, OBEs and the like demonstrate that there are three dissociable components with distinct neurological bases (Aspell et al., 2012, p. 12). Therefore, this body matrix relies on a collection of multisensory brain regions and may be dissociable into usefully manipulable subcomponents, each contributing their own relative weighting of bottom-up information, such as vision, somatosensation, or proprioception to the top-down contributions of ownership, identification, and body-centered representations of space. In support of this conceptualization, one study, by Grivaz, Blanke and Serino, explored these multisensory brain areas and their role in peripersonal space and ownership through a meta-analysis of functional imaging studies. The authors identified a bilateral peripersonal space network that included the superior parietal, temporo-parietal, and ventral premotor regions; a body ownership network that included the posterior parietal cortex, right ventral premotor cortex, and left anterior insula and regions associated with peripersonal space and body ownership that overlapped in analysis in only two clusters located in the left parietal cortex. Other than that, distinct activations were recorded for peripersonal space at the temporo-parietal junction and in the anterior insula for body ownership (Grivaz et al., 2017, p. 602). These results support the earlier assertion of a collection of dissociable networks of self-location, ownership, and peripersonal space. This work refines previous research by Ehrsson, which had associated the experience of ownership in the RHI with multisensory brain areas, particularly the premotor cortex (Ehrsson et al., 2004). A follow up study refined the design to control for the relative contribution of visual and somatosensory information wherein the RHI was induced entirely without visual information, in contrast to the 2004 study that relied on visual information. This study again reported activity in the ventral premotor region as well as intraparietal and cerebellar activity related to the subjective rating of the strength of the illusion (Ehrsson et al., 2005). Other researchers support this interpretation that the RHI is dependent on “the brain’s ability to detect statistical correlations in the multisensory inputs” but in keeping with a top-down interpretation insist that this correlation is modulated by a “pre-existing representation of one’s own body” (Pavani & Zampini, 2007). A further study using the full-body illusion induced at different body regions, applied a multi-voxel pattern analysis and found a pattern of activity in the premotor cortex that was independent of the region of the body used to induce the illusion. The authors therefore proposed that the premotor cortex facilitated a dynamic whole body multisensory percept, resulting from the synthesis of information provided by neuronal populations each of which contain visuo-tactile receptive fields associated with specific body regions (Gentile et al., 2015). This research makes it clear that when considering the application of a medical avatar for neurorehabilitation it is important to think about what brain regions it might activate and how manipulations of perspective, body representation, or even agency might affect regional cortical activation.
Self-identification: Embodiment, ownership, and agency
In order for a bodily representation, or simulation, to be of use, whether it be a matrix, or schema, or image, it must be subject to the attribution of ownership and the objective experience of agency. The relationship between body ownership, the sense that one owns a body, and agency, the sense that one is in control of a body has been conceptualized according to two models (Tsakiris et al., 2010). The first is the additive model that suggests that ownership is a component of agency, but that agency includes more than simply ownership. The second is the independence model that suggests that these are dissociable in both top-down and bottom-up processing, in that they are triggered by different afferents and recruit different brain regions (Ibid.). In a study designed to get at a question posed by Wittgenstein’s “What is left over if I subtract the fact that my arm goes up from the fact that I raise my arm?” the authors conducted an fMRI study comparing passive movement with voluntary action (Ibid.). Their results support the independence model with the authors reporting that activations in midline cortical regions were associated with body-ownership experiences driven by sensory information, that were absent in the agency condition and activation in the pre-SMA that was associated with the sense of agency but not body-ownership. Body-ownership they report is linked to the default network with agency linked to premotor and parietal regions (Ibid). These results are theoretically supportive of the perspective put forth by Synofzik, Vosgerau and Newen, wherein agency and ownership are heterogeneous both functionally and representatively (Synofzik et al., 2008). In terms of agency specifically, in a study similar to the one above, Farrer et al. used PET to compare the activation associated with the use of a virtual hand, allowing for various degrees of rotation, controlled by the subject, to that of a virtual hand controlled by another person. The posterior parietal activity co-varied with the degree of discordance between what the subject saw and what the subject did (Farrer et al., 2003). Another interesting approach looked at subjects with mirror-touch synaesthesia, wherein a person has a tactile sensation when watching someone else being touched. This condition has been proposed to be the result of a disordered sense of self-awareness. The authors of this study conclude that through the use of a study design that blurs the boundaries between self and other, they found that mirror-touch synesthesia affects both agency and ownership but differentially (Cioffi et al., 2016). Coming at the question from yet a different direction, another study found that the sense of action strongly contributed to self-recognition. Subjects were presented with images of their own and other people’s hands. The hands could execute movements that were congruent or incongruent with the subject’s own movements or not move at all. Subjects had the most difficulty making a self-attribution when movement was absent (van den Bos & Jeannerod, 2002). Following along these lines, Ma and Hommel conclude from their study on the role of agency in the ownership induced by the VHI, that ownership and the strength of the illusion were strongly affected by agency, with no effect for visual similarity, but, they point out, both agency and similarity created a bias toward ownership (Ma & Hommel, 2015b). The authors argue that while there may be an integration of bottom-up and top-down information in the processes ownership, their results suggest that this integration is strongly weighted toward feelings of agency and efferent feedback regarding objective agency. The authors point out that in the classic RHI, agency was not an option, and therefore visual and tactile information might have been given more weight than they actually contributed to sense of ownership. Importantly for the use of a medical avatar, with the congruent agency offered by the VHI, they argue that their results demonstrate that objective agency- the ability to control the movement of an external body or object- most strongly drives ownership. This theory is supported by work by Legrand and Ruby, in their review of neuroimaging literature asking the question “what is self-specific?”, they report a co-activation of regions for self and other movements, leading the authors to argue for “an alternative, more economical interpretation that brain areas activated for self and others cannot be determined to show self-preference” and support the proposition that these brain regions, rather than being dedicated to self-reference, are instead involved in “inferential processing, such as comparison, synthesis, and induction” suggesting that self-identification, agency and ownership are the result of a synthesis among receptive fields that leads to an inference of ownership that can be independent of self (Legrand & Ruby, 2009).
Motor observation/simulation, mirror neurons and the body matrix
Following from this idea and before concluding this section, a note should be made about the mirror neurons system, aka, the action-observation network, and the use of motor observation/simulation based therapies in the neurorehabilitation setting. A variety of observation/simulation-based therapies have demonstrated efficacy in the neurorehabilitation setting, from mirror therapy (Altschuler et al., 1999; Michielsen et al., 2010, 2011; Najiha et al., 2015; Ramachandran & Altschuler, 2009; Sathian et al., 2000; Stbeyaz et al., 2007; Yavuzer et al., 2008) to action observation (Celnik et al., 2006; Ertelt et al., 2007, 2012; Franceschini et al., 2012; Garrison et al., 2013; Santamato et al., 2010; Stefan et al., 2005) and motor imagery (Deutsch et al., 2012; Dickstein et al., 2004; Dickstein & Deutsch, 2007; Page, 2000; Page et al., 2001; Stevens & Stoykov, 2003; Yue & Cole, 1992; Zimmermann-Schlatter et al., 2008). The success of these therapies is regularly attributed to activation of mirror neurons (Rizzolatti et al., 2009) which are believed to populate the action observation network (Calvo-Merino et al., 2005; Garry et al., 2005; Maeda et al., 2002; Ramachandran & Altschuler, 2009), also referred to by others as a simulation network (Jeannerod, 1994, 1995: Legrand & Ruby, 2009; Mouthon et al., 2015; Stevens & Stoykov, 2003, 2004), or mirror mechanism (Rizzolatti & Sinigaglia, 2016). This simulation network has been functionally described as anticipating movement execution and simulating the possible consequences of future actions using the same processes involved in the production of actual movements, adhering to the same rules and constraints of physical movement, with this imagery using the same premotor, parietal, basal ganglian and cerebellar networks as actual movement (Stevens & Stoykov, 2004, p. 65). This simulation or action-observation network has been proposed to be engaged by VR-based therapies (Adamovich et al., 2009; Holden & Todorov, 2002). The proposed mirror mechanism contributes to the discussion of therapeutic design because it suggests that the specifics of how one engages with a therapeutic avatar might be of importance. For instance, the intent to imitate, or at least, the mental simulation of the action, seems to be an important component in the neuroplastic response to therapeutic intervention (Vourvopoulos & Bermúdez i Badia, 2016).
The medical avatar in neurorehabilitation
This literature underscores the importance of knowing which body schema within the matrix to engage, and which attributions of agency or ownership, and to what degree, should be promoted for maximum neuroplasticity. The medical avatar makes it possible to manipulate these in ways never before possible. If there are multiple cortical representations of the body and these multiple body representations are differentially subject to attributions of self-identification (such as ownership and agency), then the question becomes which representations and attributions matter for rehabilitation. For instance, a study by Longo et al. that varied skin luminance and hand shape, suggests that similarity with the illusory hand is related more to the “body imagistic” than the “body schematic”, that is, similarity plays a role in the determination of “what the body is, not where the body is” (Longo et al., 2009). The authors found that feelings of embodiment could engender a perceived similarity, and that similarity generated embodiment, suggesting that in the therapeutic setting embodiment could lead to feelings of similarity with an able bodied avatar, a top-down attribution of self-identification, whereas a differently abled avatar wouldn’t interfere with the development of ownership over the avatar as long as that avatar could drive the bottom-up development of a body schematic, i.e. it could provide feedback.
In the neurorehabilitation context there is an emerging body of neuroimaging literature of particular interest. Gentile, et al. used fMRI and the full body illusion (BSI) to explore how “congruent visuo-tactile signals from a single body part facilitate the emergence of the sense of ownership of the entire body.” The authors go on to say that based on multivoxel pattern analysis, they propose that “the dynamic formation of a whole-body percept may be mediated by neuronal populations in the ventral premotor cortex that contain visuo-tactile receptive fields encompassing multiple body segments” (Gentile et al., 2015, p. 328). Another fMRI study found that activity in the premotor cortex correlated with feelings of limb ownership. The authors conclude that this multisensory integration in the premotor cortex supports the attribution of ownership (Ehrsson et al., 2004). These results are in line with earlier primate research that concluded that individual neurons in the premotor cortex receive converging visual and proprioceptive information as to the location of the arm (Graziano, 1999). This role for the premotor cortex is further supported by the perspective on the neural mechanisms of embodiment put forth by Arzy et al., that states that embodiment and “corporeal awareness” are supported via bilateral premotor cortex activity (Arzy et al., 2006). Finally, in support of the broader network proposed by Gentile, recent neuroimaging work, using a VHI-like set up coupled with fMRI, studied the neurological activation associated when an otherwise faithfully tracking virtual hand developed a delay, thus, causing the participant to compensate for the delay (Limanowski et al., 2017). The imaging results lead the authors to support the idea of a distributed “cortical motor control hierarchy”. This consists of temporoparietal regions that make attributions of visuomotor congruence and possibly, thereby self-identification, the lateral occipitotemporal cortex for early visual error processing, and intraparietal regions they associated with goal errors and motor correction. Taken together, this research may shed light on which regions of cortical activation could maximize the plasticity of the body matrix, and the manipulation of which attributions, e.g ownership or agency might best drive this activation. In an article on cortical plasticity post stroke, Llorens et al. speculate that the ownership elicited by experiments such as the RHI, VHI, BSI, encourage the reconfiguration of the body matrix, and thus correct the distortions of body representation resulting from lesions (Llorens et al., 2017). One way to understand how to clinically manipulate which cortical regions are activated, and which attributions are made, in avatar-based neurorehabilitation, is to look at studies on avatar use and self-identification.
Avatar self-identification: Embodiment, ownership and agency
This line of interdisciplinary research, a mix of cognitive neuroscience and computer science, explores how body image and body schema can be altered through identification with an avatar. Many of these explorations of avatar engagement and self-identification have used a VBSI-based design paradigm (González-Franco et al., 2010; Lenggenhager et al., 2007; Pomés & Slater, 2013). A more recent study (Kasahara et al., 2017) demonstrated that spatiotemporal deformations of the virtual body changed the subject’s sense of embodiment. When the avatar was warped 25–100 ms toward the future, a predictive advance of movement, the subject reported that their bodies felt lighter, or that their avatar felt bouncy. Another subject reported “ I feel the existence of another who is trying to be faster than me” (Ibid). Suggesting that the bottom-up information that is received from an avatar body can influence the multisensory integration that leads to the qualia of embodiment, or the experience of having a body. As this study suggests, the experience of embodiment can be changed by an altered simulation of self. However, other studies have found that there are limits to the extent to which an avatar can be altered and still be eligible for illusions of embodiment or ownership (Chloe Farrer et al., 2003; Franck et al., 2001; Kilteni, Groten, et al., 2012; Kilteni, Normand, et al., 2012). In the following study the authors compared the threat response in a VHI with threat response reported in the classical RHI literature. They reported that the VHI could generate a threat response comparable to the RHI when the subjects had virtual hands, even when those hands had attenuated tracking, but not when they were embodied by a cursor (Yuan & Steed, 2010). The authors claim that the active use of a virtual body will automatically induce the VBSI, and found that indeed, the use of a virtual body could induce a galvanic skin response to threat comparable to that found in the RHI. Another study exploring the role of the top-down influence of the body image on motor output, explored the ways that having different avatar bodies changed the way subjects moved. The authors found that having different avatar bodies changed the drumming behavior of the subjects (Kilteni et al., 2013).
Avatar self-perspective
The avatar, as mentioned above can be seen from a 3rd person perspective to induce self-consciousness, and possibly engage cortical body mapping areas, or 1st person perspective, as in the autism studies, so as to induce other-awareness (Libby et al., 2005, 2009; Ruby & Decety, 2001; Vasquez & Buehler, 2007). The medical avatar can reflect the users current health status, or a positive future projection of the user, an ideal self, as might be envisioned in a weight-loss program (Fox & Bailenson, 2009; Kim & Sundar, 2012; Vasquez & Buehler, 2007). One study found a positive effect on exercise behaviors when the users self-identified with their avatar, suggesting that self-identification may also be a key aspect of the efficacy of avatar based therapies (Fox & Bailenson, 2009). But, also, the avatar can distinctly not look like the user, and this too could have therapeutic implications. An avatar could help prepare a patient for an upcoming amputation where they may be able to practice self-identifying with an altered body, or patients can train with an avatar to facilitate the operation of a prosthetic (Resnik et al., 2011) as well as alleviate phantom pain (Moura et al., 2012). Exploring these questions further, a VBSI study compared variations on perspective (1PP vs. 3PP where 3PP included a virtual mirror image as well as a self-avatar in the center of the field of view), tactile synchrony, and movement synchrony on the induction of the illusion. The authors report that variation of perspective was a stronger driver of the illusion than the other variations, and that first person perspective was a strong driver of ownership. They report that tactile synchrony and perspective were stronger drivers than movement. Interestingly, the avatars were non-gender matched to the participant. Based on the results the authors conclude that bottom up processing, that is, the integration of multi-sensory afferents combining somatosensory, visual, and motor signals, supersedes the top-down application of a body image.
Avatars and action observation
Virtual avatar based therapies have many advantages, beyond just the manipulation of perspective and ownership, from their ability to promote motivation with game-like design, to their ability to digitally capture movement for algorithmic analysis, and from their ability to provide ecological validity to their allowance for self-guided independent testing (Rizzo et al., 2004; Rizzo & Kim, 2005). The medical avatar in neurorehabilitation specifically though, provides its own unique set of clinically advantageous axes of manipulation. The avatar can reflect in real-time the movements of the user, or it can reflect a slightly improved performance so as to boost the user’s moral and motivation (Kim & Sundar, 2012). Conversely, it can reflect a slight deficit to encourage greater effort. Furthermore, the direction of motion and optic flow can be manipulated so as to induce a change in movement behavior, as is the case in one Parkinson’s study where virtual obstacles helped improve the patient’s gait (Mirelman et al., 2011). One of the advantages of the medical avatar is that it is a moving body that can be programmed to perform any action to precise specification, making it a valuable tool in the toolbox of action-observation or motor-observation based therapies. It has been well established that motor observation can drive neuroplastic changes and changes in performance. Therapies involving motor imagery (Dickstein & Deutsch, 2007; Hida et al., 2015; Jeannerod, 1995; Mocioiu et al., 2014; Page, 2000; Page et al., 2001; Simmons et al., 2008; Zimmermann-Schlatter et al., 2008), mirror therapy (Michielsen et al., 2011; Ramachandran & Altschuler, 2009; Rossiter et al., 2014; Sathian et al., 2000) and movement observation (Brass et al., 2000, 2001; Ertelt et al., 2012; Hotz-Boendermaker et al., 2011; Maeda et al., 2002; Mouthon et al., 2015; Szameitat et al., 2012) have been widely adopted in response to the abundance of supporting literature. Part of the importance of motor observation is the involvement of the visual system, because the visual system has the potential to override other modalities, including proprioception (Snijders et al., 2007), and, due to the relatively distant localizations of visual and motor modalities, they have distinct vascular supplies, therefore, the visual system can often remain intact when the motor system is damaged (Weiss et al., 2014). This visual information can be about one’s own movements or the observation of another person’s movements. Visual information about one’s own movements provides important feedback and this feedback in terms of “knowledge of performance” and “knowledge of results” is essential to motor learning (Molier et al., 2010; Subramanian et al., 2010). Virtual reality (VR) can provide unique and potentially valuable contributions to these forms of feedback knowledge. The virtual environment (VE) can offer not just visual, but auditory and tactile feedback as well, in real time, and the feedback afforded by VR is more precise, constant, and multimodal than what can be provided in traditional settings (Holden, 2005; Subramanian et al., 2010). Moreover, because the avatar and environment can provide positive feedback and experiential enrichment, it could encourage engagement and incite motivation (Deutsch et al., 2011). On the other hand, the difficulty of maneuvering a foreign body (in part - as in virtual arms, or in whole - as in an avatar) may actually improve the efficacy of virtual therapies, due, as one author suggested, to the increased motor planning and cognitive effort involved in a virtual manipulation, which may contribute to the improved performance in some virtual treatment conditions (Subramanian et al., 2013). It is clear that the observation of the virtual self, and the variable virtual feedback afforded by VE’s is a provocative area of research and growth for neuropsychological treatments. Visual information about the movements of others, virtual or otherwise, is also important as it allows “for an image to develop upon which to base movement” (Buccino et al., 2006, p. 32). Weiss et al. outline four ways that virtual environments can activate the mirror neuron system: 1. Through motion capture VR, 2. Through an avatar that mirrors movement, 3. Through a tutor or demonstrating avatar, 4. Through motor imagery facilitation (Weiss et al., 2014). Motion capture technology allows a real time video image of the user to be projected into a virtual environment. This type of system has been tested for movement accuracy (Weiss & Katz, 2004) and for body position information (Flynn et al., 2007). Avatars that mirror movements are becoming increasingly common thanks to commodity level depth cameras, such as Microsoft’s Kinect™ for Xbox™, which can, with reasonable accuracy and reliability, capture the user’s movements and body positioning and project it on a digital avatar. The demonstration avatar, or tutor avatar, has the advantage of being able to demonstrate ideal movements for the user to base their own movements upon (Eng et al., 2007; Holden, 2005). Motor imagery, a well-established movement observation based therapy that relies on the mental simulation of one’s own movements, is particularly useful when a patient is not able to achieve conscious movement of a limb at all. Motor imagery can provide a means of cortical recovery in such scenarios. Virtual reality can aid in computer assisted motor imagery therapies, wherein a subject’s brainwave activity during motor imagery can be converted into avatar movements, through a brain computer interface (BCI) allowing the user to see a reflection of their motor imagery where they would not otherwise be able to see a positive response to the same cortical activation in their own bodily movement (Bermudez i Badia et al., 2011, 2013; Gaggioli et al., 2006).
There is a strong body of evidence on the neurological basis of the action observation network engagement involved in the observation of others and oneself, and the processes of self/other identification involved therein. Increasingly authors are providing evidence for activation of the mirror neuron network in association with improvements seen with the use of virtual therapies (Jang et al., 2005; You, Jang, Kim, Hallett, et al., 2005; You, Jang, Kim, Kwon, et al., 2005). The body of literature on the specific neurological activation patterns associated with avatar engagement and observation has been expanding. A 2012 study by Ganesh and VanShie used functional magnetic resonance imaging (fMRI) to image 22 long time online gamers and 21 non-gamer controls. Participants rated the personality traits of their own avatar or avatars of known others. The authors found that the left inferior parietal lobe, an area associated with self-identification from a third-person perspective, and the anterior cingulate gyrus, associated with emotional self-involvement, were differentially activated with self-avatar conditions, suggesting a self-identification with the avatar (Ganesh et al., 2012). Part of the process of self-identification is the degree to which the VE elicits a sense of presence, the feeling of really being there in the VE (Heeter, 1992; Kiryu & So, 2007; Riva et al., 2004). The extent of the elicited presence has been hypothesized to modulate motivation as well as self-identification (Baylor, 2009). Another fMRI based protocol trained one group of subjects in a VE, with a virtual mirror based feedback rehabilitation program, and the other group in a real world mirror based feedback rehabilitation program, and found that in one case a stroke patient showed evidence that the VE condition activated the ipsilesional somatosensory cortex (Merians et al., 2009). Another exciting study looked at manipulating the virtual feedback and created conditions wherein the subject was provided with hypo- and hyper-metric feedback, both of which increased activation in the ipsilesional motor cortex. The authors suggest that these manipulations could be exploited in rehabilitation (Tunik et al., 2013). An earlier paper by some of the same authors compared neural activation in conditions where the subjects received visual feedback of the movement of their unaffected hand, the veridical condition, or a virtual mirrored image of the movement of the affected hand. The authors report that mirrored feedback resulted in activation of the ipsilesional sensorimotor cortex, even though the affected hand remained still during the task. The authors performed an additional control experiment and conjunction analysis that confirmed that the areas of motor cortex activated by mirrored feedback overlapped with the areas involved in moving the affected hand. The authors conclude that their data is suggestive of the feasibility of mirrored visual feedback and that the use of such a program could recruit select brain regions in stroke patients in order to promote neural reorganization and recovery (Tunik et al., 2011, p. 1). Interestingly, another study, using a before and after VR-based therapy paradigm, found a shift in activation from bilateral activation toward ipsilesional activation after virtual training, and this shift corresponded with motor function improvement(Jang et al., 2005). It has been demonstrated that ipsilesional cortex recovery contributes significantly to functional recovery post-stroke (Werhahn et al., 2003), and various studies have suggested that functional recovery depends on ipsilesional repair (Fridman et al., 2004; Frost et al., 2003; Ward et al., 2003, 2006: Werhahn et al., 2003). These VR studies suggest that VR-based therapies might make it possible to selectively induce ipsilesional reorganization in order to support functional recovery.
This evidence, taken together, suggests that the medical avatar has the ability to induce cortical activation and plasticity by tricking the mind about the circumstances of the body; by providing augmented feedback; by inducing motor simulation through self-identification with an avatar, producing a motor imagery/observation like neural signature. In conclusion, the full potential of the medical avatar remains unexplored but it seems clear that it is a powerful tool in the effort to motivate behavior and alter neurological activation patterns. However, very little is known about the role of avatar self-identification, perspective, sense of agency or presence, or other such axes of manipulation, such as positive prospective enhancement (ideal future self), or valence cueing, with the use of the medical avatar in neurorehabilitation. There is a great deal of work yet to be done to explore the full potentialities of this new technology, but it is clear that the medical avatar is a rich source of therapeutic manipulation and neuroplastic leverage and it will surely find itself center stage in the medical interventions and interactions of the future.
Conflict of interest
The author has nothing to disclose.
Footnotes
Acknowledgments
The author would like to acknowledge the support and guidance of her advisor Stephanie DeLuca and Nicholas Polys.
