Abstract
Communication about crime and the places it occurs has been an important area of study for criminology, sociology, public policy, and media scholars. Where incidents used to be communicated through word of mouth, physical evidence, and news outlets, recent advances in crime tracking, mapping, and mobile media have dramatically changed how individuals are informed about crime. Many organizations have adopted mobile text alerts, and recent advances in augmented reality (AR) technologies have made it possible to overlay visuals about crime on top of users’ physical surroundings. How people make sense of this visual, individualized, and location-specific crime information, however, is largely unknown and complicated by the fact that mobile technologies are challenging to study in situ, as people move through and experience urban place. Within the AR literature, while existing research has started to look at the ways that AR can affect people’s experience of place, the precise ways that people perceive and integrate AR displays into their understanding of place are still largely unexplored. This empirical study reports findings from a project utilizing AR as an urban probe, where we took participants (N = 57) around to places in a large metropolitan area in the United States and showed them visual AR crime information overlaid on the physical place where they were moving through. After seeing these urban probes, participants were asked what they noticed, remembered, and thought occurred in that place when shown AR crime information. The analysis draws on Lefebvre’s (1991) spatial triad to explain how people read places through the lens of AR, and also how they extrapolate, speculate, and make associations from AR information. Based on these findings, this study discusses the implications for mobile media scholars and their understanding of visual place-based communication, as well as for designers and policymakers considering the use of AR to communicate crime information.
Introduction
Crime occurs daily in urban areas, which has direct implications for the parties involved, the neighborhood, businesses operating in the area, and individual and collective perceptions of a particular place. Although crime has always existed in various forms, the ways that crime data about urban places get logged, presented, and communicated have changed dramatically. Before crime information was rigorously tracked and publicly available, researchers examined how crime information about places was passed along through word of mouth and collective memory (Furstenberg, 1971), media reporting (Glassner, 2010), and through the physical traces left by criminal activity such as broken windows (Wilson & Kelling, 1982). The rise of computing technologies and a push for accountability led many law enforcement departments to adopt real-time crime-tracking software like Compstat (Weisburd, Mastrofski, Greenspan, & Willis, 2004). Federal laws in the United States such as the Clery Act (1990) have also pushed for greater tracking and public disclosure of crime around college campuses (Janosik & Gregory, 2003). Crime tracking now allows people to receive crime information about places through crime ledgers and two-dimensional maps that plot criminal activity onto geographical locations (Buslik & Maltz, 1997; Harries, 1999).
Whether maps are an effective way to present crime information to communities, however, has been controversial. Proponents argue that crime maps could provide more transparency about law enforcement, inform the public about criminal activity, and ultimately empower members to get involved in their community (Harries, 1999; Wartell & McEwen, 2001). Opponents counter that mapping could decontextualize crime and lead to greater stigmatization of geographical areas (Chainey & Tompson, 2012; Wallace, 2009). Certain types of visual crime maps have also been found to result in different levels of fear associated with place, such as graduated symbol maps compared to density maps (Groff, Kearley, Fogg, & Wartell, 2005).
Mobile media has further complicated crime communication, as it enables individual exposure to specific crime events and gives people access to localized and location-specific maps. Many organizations utilize mobile networks to send text alerts to notify people about crime in a particular area, and ethnographic studies have shown that people rely on their phones to move around in nocturnal urban environments to avoid crime (Satchell & Foth, 2010). New mobile crime aggregation and mapping services such as Everyblock and Spotcrime made crime maps available to users anywhere they choose to access that information (Paulsen & LeBeau, 2012). Mobile researchers have also begun to examine how crowdsourced crime applications such as ComfortZones or Fearsquare could create location-based networks that make people feel more secure (Blom et al., 2010; Garbett, Wardman, Kirman, Linehan, & Lawson, 2014).
More recently, there have been new visual augmented reality (AR) applications, which allow place-based overlays of crime information. Defined as technology that overlays real-time, interactive, digital artifacts on the real world (Azuma, 1997), the ability to visually overlay AR crime information onto physical place adds new visual, spatial, and embodied dimensions to mobile crime communication. Whether delivered through a mobile handheld or head-worn device (Liao, 2018), AR can visually overlay two-dimensional images, video, audio, and three-dimensional representations of crime, including maps, text, crime details, or some combination therein. Spatially, AR experiences are activated by location and can overlay physical objects and places. Lastly, users need to be physically present in a particular space/place to call up relevant AR content, creating an embodied experience of mobile media.
Given these capabilities, AR merges various forms of crime communication directly with physical place. This complicates many of the previous findings regarding crime communication, which were working with some of the limitations of certain communication media and how people experienced them. These mass media studies took a more macro-level perspective to study crime communication, rather than an individual and in situ perspective (Innes, 2004). Using the context of crime, this study attempts to address this gap in understanding situational and spatial communication processes by reporting on empirical data from users engaging with AR as an urban probe. Defined as a “series of lightweight provocative urban proto-tasks to inspire direct discussion from people about their current and emerging public urban landscape,” urban probes are especially useful for asking questions about mobile media, understanding how people interact with and perceive urban artifacts, and gathering real-time field observations of how people engage with daily urban place (Paulos & Jenkins, 2005, p. 343). This study modeled its urban probe on an AR application called Spotcrime, which pulled crime reports and displayed them through AR on top of users’ physical location. By taking people to various places in the city and showing them AR crime information, this study uses urban probes as an intervention to understand how people perceive, process, and experience AR in relation to space/place. The implications of these findings for mobile and spatial theory are then discussed.
Relationship between crime, place, and signals
The relationship between crime and place has implications both individually and societally for residents, businesses, property owners, developers, and schools. As more data get collected about crime and place, the debate over how to communicate this information to communities has been controversial because there are various theories about how signals of crime might get interpreted (Innes, 2004; Rich, 2001). The “broken windows” theory argues that signals of crime that emanate from a place may increase fear about that place and precipitate more crime (Wilson & Kelling, 1982). This theory assumes that evidence of prior disorder in a place can inspire more crime, which creates a downward spiral toward more crime. While the “broken windows” theory assumes physical presence in a location, it does not necessarily predict whether an external signal of crime in that place (e.g., from a mobile application) would result in similar types of interpretation about place.
Alternatively, another perspective within criminology argues that certain types of maps and indicators of crime raise levels of fear of crime generally (Groff et al., 2005; Rich, 2001), with place being one factor but not the decisive or even memorable factor. While crime has historically been more observable in urban areas than in suburban areas, Furstenberg (1971) observed that the crime rates in certain places and fear of crime do not always correlate. Recent studies have found disparities in how incidents of crime are reported compared to how people construct mental maps of high-crime areas (Matei, Ball-Rokeach, & Qiu, 2001). These studies, however, do not consider embodied experiences, focusing more on large-scale maps and general fear of crime (Innes, 2004).
The different assumptions and conclusions of these perspectives have led to calls for a more situated and communication-based approach to understanding crime, known as the “signal crimes” perspective (Innes & Fielding, 2002). The signal crimes perspective argues that these dominant theories of crime and place both overgeneralize fear of crime and misunderstand its causal role in attributions of place by overlooking “how such understandings [of crime] are imbricated in the wider symbolic construction of social space” (Innes, 2004, p. 336). Scholars have argued that these macro-level studies about crime fail to engage with the lived and embodied experience of how people construct space/place in dynamic ways (Lupton & Tulloch, 1999). Methodologically, they argue that “whereas quantitative analyses have tended to implicitly impute a level of fear as a stable component of individual identity, more qualitatively oriented studies have suggested fear is situated” (Innes, 2004, p. 337). Hence, more qualitative, phenomenological, and spatial approaches to understanding people’s interpretations of crime signals can help hone in on whether people use crime information to interpret the place, or whether they will use the place information to interpret the crime. How they take the AR stimulus to imagine and assess the crime to attribute meaning to place is also unknown.
AR and interaction order
Just as the directionality of interpreting crime information is complicated, there is a similar lack of consensus in the mobile AR literature with regard to predictions about how people process emerging visual media in relation to physical space. The interaction order by which people process on-screen mobile information and physical/social information was one of the first questions that mobile media scholars raised (Licoppe, 2013; Ling, 2008). Mobile media not only added to the experience of place, but it also juxtaposed new information which could potentially change the presentation of place and alter the order in which cues about the place were perceived (Farman, 2013). Many of the first studies on mobile locative applications examined how people constructed hybrid understandings of place, or used mobile information to establish territoriality and familiarity with physical surroundings (de Souza e Silva, 2013; Humphreys & Liao, 2013).
Despite recent momentum in the study of AR and space/place, it has employed mainly large-scale field surveys (Colley et al., 2017) or experimental methods (Dey, Billinghurst, Lindeman, & Swan, 2018; Oleksy & Wnuk, 2017). While these studies make the case that there are certain relationships between AR and space, less is known about the interaction order by which participants foreground real and augmented interactions. Most of the existing AR research on psychological processing has focused on user attention in AR (Dey et al., 2018), using eye-tracking studies to assess salience and visual attention (Veas, Mendez, Feiner, & Schmalstieg, 2011), subtle gaze direction (Bailey, McNamara, Sudarsanam, & Grimm, 2009), or cueing and visual search (Lu, Duh, & Feiner, 2012). While these studies inform us about what people look at and the potential for AR to interfere with spatial experience, the focus is primarily on design outcomes for AR interfaces (Vaittinen & McGookin, 2018), rather than on assessing the interrelationships between AR attention and physical space/place.
This study offers an important methodological and theoretical bridge between mobile theory and AR. Methodologically, it builds on AR user studies, where the vast majority of recent research on user perception has been conducted in a lab (N = 60) compared to the field (N = 3; Dey et al., 2018). The disparity led those authors to conclude that “indoor visual perception is well studied whereas more work is needed to investigate outdoor visual perception” (Dey et al., 2018, pp. 17–19). One difficulty with understanding AR in everyday use is that places and their cues are constantly changing. Similarly, perception of place is subjective, which means that people approach place differently, based on individual differences and characteristics. By examining how people explain their real-time experience of AR urban probes, this study specifically builds on work that tries to understand precisely how AR can “fundamentally mediate the everyday practices of urban life, subtly shaping senses of place as particular interpretations of events and locations are foregrounded or side-lined” (Graham, Zook, & Boulton, 2013, p. 1).
Second, in the majority of AR studies, most of the dependent measures were quantitative ratings of AR, error/accuracy measures for tasks, and completion time (Dey et al., 2018). Many studies have focused on the effects of AR, whether it is increasing engagement (Aitamurto, Boin, Chen, Cherif, & Shridhar, 2018), learning in educational settings (Dunleavy, Dede, & Mitchell, 2009), or situational awareness (Vaittinen & McGookin, 2018), as opposed to the process of interpretation in space and in relation to space. While these outcomes and task-based measures are undoubtedly important, they tend to privilege the quantifiable measures of attention as opposed to the interpretive and phenomenological experience of processing and meaning-making around AR. Given that mobile media scholars have used this phenomenological approach to find important and subtle uses of mobile media, whether it is people’s inattention to their physical surroundings (Licoppe, 2013) or mobile media intruding on physical space (Ling, 2008), it is important that we take this approach to understand how people perceive both visual AR cues and place-based cues while out in urban space. By focusing less explicitly on a task or on eye tracking and more on people’s reported explanation of what they notice and think about when viewing AR in a physical location, this study helps us understand the interaction order of AR experiences and how people use AR to make sense of space/place.
AR and spatial theory
Beyond attention, the visual, spatial, and embodied nature of AR draws on theories of space/place. Sociologists and cultural theorists have long theorized about urban space/place, with the distinction that space is the possibility, while place is socially and culturally produced (Dourish, 2006; Farman, 2013; Tuan, 1977). Our understandings of place are based on the physical architecture of the place and also through the continual reproduction and reenactment of place through people’s spatial practices and labels for those places (Lefebvre, 1991). People’s perceptions of place are not uniform; although we may inhabit and move through the same spaces, the understanding and relationship to place is collectively negotiated and also situated and relative to each person (Lefebvre, 1991).
While these theories articulate a complex and ongoing process of creating space/place, AR represents a potential visual intervention into our experience of place. Because AR media can visually present various types of spatial information, we draw on Lefebvre’s (1991) spatial triad for describing and understanding AR spatial representations in relation to physical spatial representations. Lefebvre’s (1991) spatial triad describes three prongs that combine to produce space. The first is representations of space, which are the physical artifacts that conceptualize space (e.g., maps, blueprints, etc.). The second is representational space, or people’s descriptions and urban realities that shape how people experience, inhabit, understand, and refer to that space (e.g., discourses, imaginations, and visions of that space). Lastly is spatial practice, which is the way people perceive space as they observe themselves and others within it.
Space can be analyzed and understood as the interrelationship between these prongs, such that maps and blueprints can intervene and bring to life the buildings that introduce new spatial realities practices, and perceptions of space. Alternatively, spatial practice can change representational space, or people’s lived experience and imaginations of that space. Lefebvre’s (1991) spatial triad parses out different ways of portraying space and explains how place is constructed as a combination of those forms (see Figure 1).
Lefebvre’s spatial triad
Using AR crime as a context allows us to study aversion and negative experiences when understanding place (Manzo, 2005), a relationship that is often underexplored: “any exploration of place as a phenomenon of direct experience . . . must be concerned with the entire range of experiences through which we all know and make places” (Relph, 1976, p. 6). Through the lens of spatial theory, AR also offers an important intervention into our experience of place, because it can introduce an additional layer of information onto physical space, from each prong of the triad (e.g., placing a map onto the scene adds a representation of space). Dourish (2006) observed that “our interest must be directed towards the ways in which information technologies create new ‘virtual spaces’ that transcend and overlay the ‘real’ spaces of the everyday world” (p. 304). Other scholars have utilized Lefebvre’s (1991) spatial triad to understand how mobile media complicate our conceptualization and experiential embodiment of space (Farman, 2013). While these works have argued that our experience of space arises from the interplay between our mind, the media, and surrounding structures (Farman, 2013), which interpretations get foregrounded or side-lined and how people make sense of place using AR is still an open question. Just because AR information gets presented alongside physical place does not necessarily mean they will be processed simultaneously and with equal weight given to both. It is also unclear how different AR presentations of prongs on the spatial triad will interact with physical information and work together to produce particular meanings. Drawing on these distinct features of AR to study in situ crime communication, this study aims to answer the following research questions:
RQ1: When experiencing AR crime displays, how do people interpret the AR information in relation to the information they receive from the physical space?
RQ2: When experiencing AR crime displays, how do people assign meaning to the different prongs of Lefebvre’s spatial triad (representations of space, representational space, and spatial practice) and assess the relationship between AR and physical spatial understandings?
Methods
Spotcrime as an urban probe
To understand how people respond to AR crime information in situ, we utilized a directed urban probe that exposed participants to an AR stimulus. Urban probes are multifaceted and well suited to exploring situated perceptions of space, as “a lightweight, provocative, intervention methodology designed to rapidly deconstruct urban situations, reveal new opportunities for technology in urban spaces” (Paulos & Jenkins, 2005, p. 341). They are particularly useful in understanding real-life encounters, as urban probes “provide methods that aid researchers in gathering fragmentary glimpses into the rich texture of people’s daily urban street life” (Paulos & Jenkins, 2005, p. 343).
We designed our AR urban probe based on an application called Spotcrime, which overlaid cartoon depictions of crimes and a text display of the crime. Spotcrime is a mobile application that launched an AR version in 2010 through the AR browser called Layar. We created three distinct locations for crime signals based on the most frequently occurring crimes: theft, burglary, and aggravated assault). We chose these crimes based on commonality, but also because they vary in terms of severity, victim, and violence—theft means taking property without the use of force, burglary means illegal trespass in order to steal property, whereas assault has to be against a person.
Through the mobile application HP Reveal, we designed three crime signal interventions that mirrored what people would see through Spotcrime. The interventions included a spatial description about the type of crime, location, and time (see Figure 2). We designed these probes to ensure that participants received consistent information within their experience. Using the visual recognition feature, we programmed HP Reveal to recognize certain visual trigger locations on campus (e.g., food trucks) and overlay the visual and textual crime content. These specific locations were selected due to proximity so that participants could visit three distinct locations, and because these food trucks provided unique trigger images that enabled the app to activate more easily.
AR crime overlays.
Procedure
Participants were recruited from undergraduate courses at a university in the east coast of the United States. Participants were offered a $10 gift card as a participation incentive, which was approved by the university’s Institutional Review Board. Participants were provided with a smartphone with an AR mobile application called HP Reveal installed. A research assistant showed the participants how to use HP Reveal and directed them outside to three separate spots on campus, each approximately 5 minutes apart. At each location, after seeing the AR crime information, participants were asked to fill out open-ended questions about what they thought happened in that place, how they felt about being in that place, and whether that information changed their perception of the place. While urban probes are able to reveal a wide range of observational and self-reported data, these questions were selected to explore the subtheme of place. One of the key advantages of urban probes as a method is that they are particularly well suited to address questions about “what cues . . . we use to interpret place and how will urban computing re-inform and alter our perception of various places” (Paulos & Jenkins, 2005, p. 341). This study specifically reports on the qualitative, open-ended written responses from participants. The handwritten responses to the open-ended questions were typed into a document. Each participant was assigned a number to anonymize the data.
There were 57 participants that completed the study and saw the AR stimulus in all three locations. Participants were given a demographic questionnaire to fill out. The age of the participants ranged from 18 to 36 (M = 20.7, SD = 3.3), of which 73.7% were female and 26.3% were male. The ethnic composition of the sample was: 49.1% Caucasian, 19.3% Asian or Pacific Islander, 14% African American, 3.5% Hispanic or Latino, and 14% that indicated “Other” or “Prefer not to answer.”
The research team familiarized themselves with the data by reading them and then open-coding them to generate an iterative set of thematic codes about the crime and the place where it took place (Charmaz, 2006). Codes included what facts were noted about the augmentation itself, what facts were noted about the crime (e.g., time of day, specific objects involved in the crime, whether they identified the victims or the perpetrators of the crime), and what facts were noted about the place (e.g., whether they noted locations, whether they imagined scenarios about how the crime occurred in that place, and whether they talked about feeling less safe in that place). After thematically coding certain posts, the research team grouped responses together into broader themes that indicated particular interpretations of crime and place (e.g., using physical place cues to read the AR information, extrapolation about the crime from the AR cues). We took these grouped responses and analyzed them by writing detailed memos identifying similarities and differences between them, and analyzing the nuances and complexity of particular themes (Charmaz, 2006). Participant identification numbers are used to report all data.
Findings
Using physical place cues to read AR information
With AR visually depicting a crime that had been committed at the spot where the participant was standing, one common response was to start analyzing the physical features of that place to explain the AR event. One type of response focused on a known location that may have been the target of a crime: “A person was robbed probably after getting money from the PNC [bank]” (P21). In addition to targets, people analyzed the physical architecture of the place and whether the layout could have contributed to the crime: “This area is between larger buildings with a good amount of shade” (P4). Participants observed that the number of people surrounding the location and possible getaway routes could have contributed to the occurrence of crime in that place: “I could see why there may be theft, because it’s near an alley and crowded with food trucks” (P13). It was also common for participants to look for objects in the physical place that would have been stolen/targeted, whether it be a wallet, a phone, or a bike: “I see lots of bikes around” (P2).
Within these responses, we can see how several participants directed their attention to specific features of the place to explain the AR information. Additionally, the physical cues they identified ranged in their temporality, ranging from fixed locations (e.g., banks, ATMs) to semifixed spots (e.g., food trucks) to transient cues (e.g., people, bikes). These were all utilized to potentially reconstruct the scene and explain/contextualize the AR crime information.
Using AR information to extrapolate and personify crime
While the AR intervention about crime was limited to a visual cartoon and a text description, many of the immediate responses contained imagined and extrapolated information well beyond what was shown in the AR stimulus. One theme we found in these responses was the personification of both victim and perpetrator of the crime, even though description of them was not provided. Oftentimes, the victim was presumed to be a student, while the perpetrator was thought to be unaffiliated with the school: “The location is close to the outside of campus and it’s a more discrete area. I’d guess that someone came on campus from the surrounding neighborhood and assaulted someone” (P29). The circumstances under which the crime took place were also the subject of speculation, with responses assuming that the victim was alone: “Someone was walking alone late at night and got robbed” (P16). Others added that the victim was potentially careless: “I would guess someone was walking by themselves with headphones in not paying attention—someone came up and stole something” (P39).
Another extrapolation participants made was to imagine the crime to be more violent, even though theft does not involve physical force: “I would guess that a woman was walking by herself and was assaulted and robbed” (P55). In a few responses, a weapon was included in the description of the perpetrators: “Student was walking to class/dorm and get robbed at gunpoint, maybe by more than one person” (P16). A number of participants took the general information about the crime and increased the level of its severity in their descriptions (e.g., gunpoint, sexual assault) beyond what was listed in the AR intervention.
These extrapolations revealed certain assumptions that people had about crime. First, since participants were students, they imagined themselves as the targets, and they imagined the perpetrators as outside members of the community. Second, they imagined circumstances that caused the victim to be the target of the crime, whether it was the time of day, number of people, or their lack of attention. Lastly, the heighted intensity of the crime was another assumption that people made, along with subsequent details about the victims (e.g., female) and the crime and perpetrators (e.g., robbed at gunpoint by more than one person).
Using extrapolations from AR to make associations with place
While the places where people saw the intervention were well known to the participants, some took the general AR information about crime to change their associations with the place. For those who described associations of fear and lack of safety, the personal relevance of the place was often noted: “I am extremely nervous because something like this happened and because I take this path home” (P5). Others made a broader spatial association to campus in general: “It changes my perception a lot because I thought I was safer on campus . . . because it’s closer to the heart of campus so it makes me feel worse” (P36). Then there were those who took the specific instances of AR crime to describe a feeling of increased frequency of crime in that place: “I feel as if a lot of crimes happen on campus” (P44), as well as an emotional response about the place: “Horrified. Very unsettled . . . Absolutely, I didn’t think in broad daylight on this street crime would occur” (P50). Participants would take this stimulus to make broader assumptions about crime trends in places and to change their associations with a place from safe to unsafe. Although one might imagine their prior experience/familiarity with the place as a factor for mitigating fear, many expressed that this information violated the expectation they had of the place, which made it worse.
Many respondents also explained ways that they could protect themselves and how this knowledge would affect their future actions regarding that space. There were respondents who would be less willing to return to that location over concerns for their safety: “It does affect my willingness to return here. I don’t feel safe here anymore” (P13). Others were not ready to rule out returning, but still expressed trepidation: “Yes, it makes me a little hesitant to return to the buildings in this area” (P17). Even those who accepted that they would come back, imposed conditions or explained specific steps they would take to protect themselves: “I will be more mindful of my stuff and what I bring. What time I walk here. Will not park my car here” (P41). Many believed that they could protect themselves by being more cautious: “I will return but with more caution as many people like to repeat what they’ve done or others will do the same” (P51). A sentiment about walking in groups was common, as was a condition about time of day: “I feel safe about this spot during the daytime, but if it is during night, I would be in a group of people” (P18). One intention to be more cautious was related to the memory of the AR intervention every time they would pass through: “I will still return, but I’ll just remember what occurred here” (P7).
These responses indicate that participants saw the AR information and assumed the position of the victim of the crime and viewed themselves as potential future targets. The intended actions were also attempts to minimize their fear about the place where they would move through in the future. Whether it is a mindset of awareness, precaution, remembrance, or a more tangible action of not parking cars in a particular location or avoiding the place altogether, there were a wide range of future actions people wanted to engage in to prevent crime from befalling them.
Discussion and implications
At the macro level, there is an ongoing debate about crime mapping and communication as to whether spatial crime information increases general levels of fear, specific levels of fear, and the degree to which those are attributed to place (Buslik & Maltz, 1997; Groff et al., 2005; Harries, 1999; Rich, 2001). Within those debates, there have been calls for research that is more situated within specific places (Innes, 2004), particularly as mobile media is continually changing the aggregation, availability, and personalization of crime maps and information (Blom et al., 2010; Garbett et al., 2014; Paulsen & LeBeau, 2012). AR displays of crime are a unique form of crime signal, in that they provide visual, spatial, and embodied signals of crime in space/place. This study begins to unravel how those signals are interpreted, which contributes to our understanding of both crime literature and mobile spatial theory.
Compared to previous studies that have primarily focused on the effects of AR in lab settings, this study, which analyzed a range of in situ qualitative responses to AR crime information, helps further our understanding of how AR media can affect people’s interpretation and experience of urban space/place. Scholars have noted that understanding the role that technology plays in constructing place requires assessing “the . . . parallels between the flow of information through media and the flow of information in physical settings” (Meyrowitz, 1986, p. 38). The question is not just how they present physical information in parallel across space/place, but specifically how the AR media intersects with physical place, and how people make sense of those relationships between AR information and physical information. AR as a visual mobile medium complicates Lefebvre’s (1991) spatial triad, by being a simultaneous visual overlay of a representation of space, an image that alters and changes the perceived space, and labels that describe representational space (see Figure 3). In this particular case, by compressing crime events that took place in the past, it takes certain descriptions of crime (representational space), displays them over physical space (spatial practice), and flattens them onto a singular crime map (representation of space).
The role of AR in Levebvre’s spatial triad.
Interaction order of AR crime information and physical place
While studies of locative technologies recognized their potential for subtly foregrounding certain forms of historical and geographical information alongside place (de Souza e Silva, 2013; Farman, 2013), and understood that meaning is derived from both the augmentation and the features of the place itself (Graham et al., 2013; Hofmann & Mosemghvdlishvili, 2014; Liao & Humphreys, 2015), the specific interaction order for processing and reconciling them was still largely unknown. Where some have argued that mobile media intrudes upon and detracts from people’s physical interactions (Ling, 2008), others have argued that the embodied nature of mobile media renders these binaries obsolete: “our cultural understanding of what ‘co-present’ means has been so vastly overhauled to the point that even the categories themselves become problematic” (Farman, 2013, p. 99). This study is one of the first that empirically examines the relationship between these mobile and physical cues when presented visually and simultaneously through AR.
One of the key findings is that even when the AR information and physical information are presented alongside one another within the visual scene, they are not privileged equally. While existing studies have primarily focused on understanding how AR can direct or divert user focus (Bailey et al., 2009; Dey et al., 2018; Veas et al., 2011), the question of how meaning is made is more complicated than simply mapping attention and tracking what people are looking at. This study found that people used the physical information to read and contextualize the AR information, which is an important revelation for the directionality and scope of how people make sense of AR and physical information. We did not find support for the idea that people would use an AR crime signal in situ to explain and make sense of certain features of that physical place, like broken windows or graffiti (Wilson & Kelling, 1982). Participants did not report statements that used the crime signal to explain physical blight. Instead, we found that participants noticed cues from the physical place and used those to explain the crime (e.g., presence of ATM, alleyways). In mapping this onto Lefebvre’s (1991) triad, this study found that the physical spatial practice information was used to read the AR information, as opposed to the other way around (see Figure 4).
Using physical spatial practice to interpret AR spatial practice.
The crime context also lends itself to a specific type of speculation about the event. While other AR applications are attempts to replicate specific historical events that occurred in place such as the Tiananmen Square protests (Liao & Humphreys, 2015) and World War II sites (Oleksy & Wnuk, 2016), the crime application is less of a remembrance of a specific documented event in history and more of an annotation of an unknown recent past event. This study found that when being presented with general information about a place, people utilized a variety of AR cues to extrapolate about the prior event, whether it was the circumstances of the crime or the severity of it. The scenarios that people would come up with for the crimes in those locations varied depending on whether they were focused on the victim/perpetrator, circumstances, or severity. Some were rooted in something they saw in the physical environment, while others were formed entirely in their imagination. Bringing up the issue of crime in AR spatial practice motivated the extrapolation of the event and how people thought about the AR representational space (e.g., crime imagination, description, visions; see Figure 5).
Using AR spatial practice to extrapolate about AR representational space.
This finding also helps explain one of the reasons why presenting crime information to people might increase fear levels (Buslik & Maltz, 1997; Harries, 1999), particularly because AR depictions can cause people to personalize the incident of crime. While research has shown that people may react emotionally and critically to mobile crime maps in the aggregate (Blom et al., 2010; Garbett et al., 2014), the way that AR individualizes crime in space can create a different kind of place-specific association because it represents “signal crimes and disorders [which] are the types of problems that people ‘tune’ into, in order to manufacture for themselves some sense of the risks that are distributed throughout social space” (Innes, 2004, p. 352). Despite the limited detail of the AR information, it essentially serves as a hyper-specific mediated signal which reframes the place and, in combination with physical cues, gives people license to radically reimagine and reenvision a specific event that places them at the center of that imagination.
Using extrapolations from AR to make associations with place
Understanding how people interpret these spatial realities simultaneously, and the directionality of their spatial understandings, is a critical first step toward understanding why and how AR can lead to certain spatial outcomes (Oleksy & Wnuk, 2017; Vaittinen & McGookin, 2018). People’s responses to an AR intervention revealed a sequence of spatial processing. The first step was to use cues from the physical space to process the AR information, and the second was to extrapolate the AR depiction to certain events and associate that with their understanding/imagination of the AR representational space. It was a combination of real and imagined readings of the crime within the AR space that directed people to make certain categorical and negative emotional associations with physical place. Participants reported several emotional responses (e.g., unsettled, unsafe) along with future behavioral intentions (e.g., not returning alone, not parking cars) that indicated a change in prior perception of place. Even with short-term exposure to a cartoon visual representation and basic text information about crime, the interrelationship between how people interpreted the AR information and physical place can fundamentally reshape the perception and production of those places (see Figure 6).
Using AR representational space to make attributions to physical representational space.
This process of associating AR crime information with space is important, because it helps build our understanding of how AR, as a locative technology, can alter or complicate people’s construction of place. Construction of place is a complex phenomenon, which is situational, contextual, and subjective (Manzo, 2005). Even when dealing with physical space exclusively, spatial scholars have argued that the construction of meaning is the result of a combination of factors, whether it is representations of space, representational space, or continual reproduction and reenactment of place through people’s spatial practices (Lefebvre, 1991). AR further complicates these frameworks because it allows people to overlay and visualize prior spatial events while physically present in a place, which alters the experience-in-place at that moment and also calls into question their past experiences in the place.
While existing studies have examined the effects of AR on perceptions of space and place (Liao & Humphreys, 2015; Oleksy & Wnuk, 2016), this study identifies a process and directionality by which those perceptions get shaped. It is first the physical space that is used to interpret and extrapolate the AR spatial information, the sum of which then gets associated with Lefebvre’s prong for constructing physical space (i.e., representational space). While existing mobile theory has argued that there is a mutual shaping between physical space and mobile experience (Farman, 2013), this study builds on that understanding by illuminating specific and complex processes that people experience in space and by offering empirical support for a particular starting point and directional relationship that people engaged in to construct meaning (physical spatial practice => AR spatial practice => AR representational space => physical representational space). This has implications for our understanding of the underlying mechanisms that can contribute to AR effects. It also builds on our understanding of spatial theories and how mobile media can complicate these interrelationships. This specific interpretive mechanism is likely accentuated by the crime context, given the essential role that perception of safety plays in people’s relationship with place (Innes, 2004; Manzo, 2005). As a corollary to geographical studies of AR, which show how it can direct people to move to certain places on a large scale (Colley et al., 2017), this study builds on existing research that examines the possibility for AR to cause inhibition in and avoidance of place (Salen Tekinbaş, 2017).
Comprehending how people understand and make sense of AR information in relation to place is important for designers and policymakers. While much of the initial fear regarding AR was the extent to which games like Pokémon Go would distract or diminish people’s perception of physical place (Ayers et al., 2016), these studies were mostly looking at AR information that just happened to be overlaid onto a place, as opposed to content that was directly relevant to it. When AR is designed with information explicitly about a place, however, the visuals are given meaning and interpreted alongside the physical space itself, rather than overriding physical space. In terms of policymakers interested in alerting people about crime, the inclination of people to fill in details about AR events, coupled with imagining themselves as the victim, has implications for communication about crime and the emotional responses to crime. Specifically, for those considering alternative forms of mobile alerts or visual messaging, converting public information that exists in a tabular or map form into AR may engender more subjective responses and communicate a message about space/place different than intended.
Conclusion
AR is an emerging technology that is still being developed, negotiated, and understood. As the technology becomes more advanced and enters common use, questions about how people perceive AR media in relation to space and place will be even more pressing (Liao, 2019). Future research could build on this with studies that follow people over extended periods of time, have multiple repeated AR interactions, and explore interrelationships between different types of AR content sequentially (e.g., switching from one type of AR content to another). Because AR technologies are able to overlay many different forms of visual information, future work could also investigate different visual displays of AR and various permutations depicting representations of space, representational space, and spatial practice. Understanding these forms can further build our spatial models for understanding the relationship between AR and physical space. Lastly, other media contexts besides crime visualizations may engender different processes and responses. This study is a first step in utilizing urban probes and Levebvre’s (1991) spatial triad to understand how people read place using an AR application, but there are many current and future applications that could benefit from this approach. Our understanding of AR and how it is perceived in different places will continue to evolve along with emerging AR media, changing usage patterns, and users’ understanding of the technology, all of which will raise new research questions and challenges surrounding AR.
Footnotes
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
