Abstract
BACKGROUND:
Deep breathing exercises are known to help decrease stress. Wearable and ambient computing can help initiate and support deep breathing exercises. Most studies have focused on a single sensory modality for providing feedback on the quality of breathing and other physiological data.
OBJECTIVE:
Our research compares different feedback modalities on an individual’s experience and ability to perform breath-based techniques at work.
METHODS:
We designed three different interactive prototypes that used light, vibration and sound feedback modalities. We tested each prototype with 19 participants whilst they were performing typical work tasks in a naturalistic setting, followed by semi-structured interviews.
RESULTS:
We found that sound was the most successful feedback for the majority of participants, followed by vibration and ambient light. We developed an analytic tool, the Extended Cycle of Awareness, to facilitate understanding of the patterns of awareness and the flow of experience generated by participant interaction with prototype systems that provide feedback on the quality of breathing. Participants followed one of three different types of patterns: (1) ignoring the feedback; (2) not understanding the feedback and being overwhelmed by it; (3) successfully using the feedback to initiate deep breathing and reflect on the change in the quality of breathing.
CONCLUSIONS:
We offer a set of design recommendations for crafting interactive systems to support deep breathing at work, including personalization, designing for the cyclical process of attention and awareness, and designing for reflective practice.
Introduction
Office work is a common source of stress. Coping strategies to deal with the stress of office work vary from exercise to social support [1]. Mindfulness techniques have recently entered the mainstream business world, with many workplaces offering emp-loyees wellness programs to improve mental health, well-being and productivity [2]. One of the basic mindfulness techniques is to focus on one’s breathing, and to perform deep breathing. Deep breathing refers to using the full capacity of the lungs to inhale air into both the chest and diaphragm [3]. Practicing deep breathing exercises increases oxygen consumption, which assists the parasympathetic nervous system by decreasing the heart rate and blood pressure. The practice is known to revitalize and increase alertness [4], which has obvious benefits for productivity [5]. However, the practice of deep breathing by office workers is not commonplace, raising opportunities for technology to assist in the adoption of such beneficial mindfulness techniques. Of particular interest is the use of wearable and ambient computing devices to monitor stress levels and encourage deep breathing [6–11]. Whilst most studies have focused on a single modality for providing feedback on the quality of breathing and other physiological data, there is a lack of research comparing different modalities and how these affect an individual’s experience and ability to perform breathing techniques.
In this paper, we present a comparative study of three different modalities: light, vibration, and sound that provide feedback when the participant is not breathing from their diaphragm during work-related tasks. Our research aim is to investigate the impact of different feedback modalities on alerting people to the quality of their breathing. We do this thro-ugh developing and then testing three prototypes, each providing feedback to participants through one modality: light, vibration, or sound. To investigate the impact that the prototypes had on the participants, we looked at two different research questions: Which feedback modality best assisted with practicing deep breathing? When using the prototypes during a work-rel-ated task, where was the participant’s attention and awareness?
We begin the paper with background on the relationship between deep breathing, body awareness, and emotions, followed by the use of biofeedback for enhancing body awareness through ambient and wearable technologies. We then describe the methodology for our design and user evaluation study and present the results of the formal user evaluation. We developed an analytic tool to facilitate understanding of the patterns of awareness and the cyclic flow of experience generated by participant interaction with prototypes that provide feedback on the quality of breathing during work-related tasks. This tool is an adaptation of the Cycle of Awareness framework [12] that was built around a scenario where the wearer of the technology engages in a dialogue between their body and feedback received from the technology. Such a dialogue between a human body and machine is beneficial for enhanced reflection and bodily self-awareness. We discuss the key findings that emerged from the study. We conclude with a set of design recommendations for crafting interactive systems to support deep breathing at work, and point to directions for future research.
Background
Deep breathing
Breathing is an involuntary and often unconscious act that our bodies perform to keep us alive. However, this type of passive breathing often only engages a portion of the lungs in respiration. The opposite of this type of breathing is a conscious breath that uses most of the lung capacity. The two different ways to breathe are often called shallow breathing and deep breathing [4]. Shallow breathing involves taking in a partial breath in the chest. Deep breathing involves “expanding the diaphragm with the abdomen rising with each inhalation” [4].
A review of the relationship between respiration rate and emotion suggests that whilst deep breathing is associated with relaxation and pleasant emotions, shallow breathing is associated with anxiety, tension, and unpleasant emotions [3, 15]. Deep breathing exercises found in therapy practices, mindfulness, and yoga have been used to improve negative psychophysiological conditions such as stress [3, 16–18]. Deep breathing regulates the autonomic nervous system, and the extra oxygen produced during deep breathing is used to increase alertness and attention [3, 10]. Paying attention to the breath can increase bodily self-reflection and awareness of the connection between the quality of breathing and emotional state [4, 5]. Deep breathing has the potential to reduce stress and has proven to help increase productivity [20]. Each person has a stress threshold that increases performance; however, if stress increases further beyond the individual’s maximum, then their performance will decrease [21].
Deep breathing exercises not only increase our alertness and capacity to sustain attention, but also expand our awareness of body signals to take breaks when we are stressed or distracted [14]. Technology has the potential to be used as a training tool to help us be more aware of bodily signals. In the next section, we review key examples of interactive technology with this goal in mind.
Ambient and wearable computing approaches
In seeking to develop the prototype to answer our research questions, we explored many different feedback modalities. However, for the study presented in this paper, we decided to limit our scope to focus on light, vibration, and sound feedback modalities. These three types of modalities represent a range of spatial relationships to the body and choice of focal placement (Fig. 1).
Feedback modalities of light, vibration, and sound mapped to dimensions of spatial proximity to the body (proximal/distal) and focal placement (central/ambient).
Light feedback modalities are often used to passively present information to the participant as distal, ambient displays [5, 23]. Ambient displays are not intended to be the primary focus of attention; for example, Fortmann et al. [24] developed an ambient light system to encourage people to take breaks while working. It was a simple system where the light turned on to prompt participants to take a break. Other examples of light feedback modalities had the participant only focus on the light when they participated in various body awareness exercises. The Breathing Light designed by Höök et al. [23] provided a fading light that changed as the participant listened to a pre-recorded Feldenkrais Awareness Through Movement lesson. The MoodLight system developed by Snyder et al. [6] is intended to be used for communicating the emotional state of a person. It takes measurements of a person’s arousal levels and displays the data as color and intensity in the lighting system. In the study, the researchers used MoodLight as a tool for practicing mindfulness exercises, and as a resource for dialogue between the participant and an observer. Within the ambient light display studies, Roo et al. [22] and Thieme et al. [8, 9] explored scenarios where the ambient light was not at the forefront of the participant’s attention, and the participants were focused on other tasks. However, the researchers did not discuss how the light feedback modality changed the experience one had with their body data after the initial novel use of the prototype wore off.
Vibration feedback modalities have the most proximal relationship to the body, given they are typically part of wearable devices. They actively alert participants and are difficult to ignore. They are often used in rehabilitation, where the location of providing vibrotactile feedback on the body can be computationally controlled and made contextually relevant to the physiotherapy exercises. For instance, Ananthanarayan and colleagues [26, 27] developed a vibration feedback prototype to help participants train for knee rehabilitation and exercise. Other researchers used it for virtual reality rehabilitation and game development. For instance, Lv et al. [29] developed a virtual reality game for hand rehabilitation, and Chen et al. [30] developed a vibration feedback video game controller. Vibration wearables have also been developed for people who are visually impaired or need assistance due to a disability. Grussenmeyer et al. [31] developed vibration feedback from tablets and smartwatches for people who were blind, while Carcedo et al. [32] developed vibration feedback for people who were colorblind. Charan et al. [32] tested a wearable posture correction sensor in the workplace. They found a significant reduction in back and neck stress in participants using the sensor. Vibration feedback modalities are also one of the most popular feedback modalities on commercial fitness and mental health trackers such as the Fitbit [33] and the Spire [34]. Vibrations are easily recognizable when providing feedback on measured body data [32], nevertheless, similar to the light feedback modality, there is limited research on how vibrations influence the experience of understanding the changes of one’s body state during body-based mindfulness practices such as deep breathing.
Finally, sound can be considered to have a proximal relationship to the body, although perhaps not as close as vibration. Chen et al. [30], and Vidyarthi et al. [9] developed interactive applications related to deep breathing that use sound feedback modalities and a variety of sensors to measure body data. Virtual Meditation Walk [35] is a virtual reality experience that teaches participants suffering from chronic pain to practice Mindfulness-Based Stress Reduction. The galvanic skin response (GSR) data measuring the participant’s arousal levels is then used to change the soundscape and visuals of the weather while the participant is walking through the virtual world. It is intended for immersing in a virtual environment where a metaphor of mindfulness is communicated through the behavior of the virtual world. Sonic Cradle [9] is an exploratory mindfulness experience where participants sit in a dark room and control the soundscape with their breathing. Ambient Walk [36] is inspired by mindfulness walking exercises. Sounds are generated by the walking speed and breathing data of the participant. In all three examples, the sound is used to help participants relax and explore their body data in a creative way. However, there is limited research on how sound cues can assist participants to learn about their altered body states whilst performing tasks such as work. These types of immersive systems would not be suitable for use in a work situation where the aim is to integrate mindfulness into the practice of work. To our knowledge, there is only a single study that researched the experience participants had with different feedback modalities. Klamet et al.’s WeaRelaxAble [37] found that merging feedback modalities gave the participant the best experience. However, the prototype they designed was quite large and complicated to use as it required attaching many different pieces of wearable technology to the body. Furthermore, the device was only tested under laboratory conditions and not in real work situations.
The study was composed of two parts: design and evaluation. In study one (design), we explored potential solutions for each feedback modality through an iterative design process. Each prototype was developed through technical experimentation and informal user testing to iterate the prototypes so that they were suitable for a formal user evaluation study. In study two (evaluation), we tested the prototypes with 19 participants and collected their feedback. In the following sections, we describe the protocol and methods we applied for the informal and formal user studies, prior to describing the final interaction design for each feedback modality.
Study one: Informal user testing
Participants
For phase one, we tested the feedback modalities on four graduate students (three females and one male), age range 20–30. Participants were recruited through word of mouth. The study was developed to test the protocol and prototypes. Therefore, the study included a much smaller sample size than we used in our formal user evaluation.
Protocol
We tested the three prototypes with the four par-ticipants to narrow down some parameters and specifications. Each participant had an informal conversation with the researchers where they suggested changes to the prototype, including the breathing patterns, visual colors, vibrations and sounds. We iterated the design of the prototype four different times with the participants. The results of these informal user evaluations were used to make design decisions to refine the development of the three prototypes.
Study two: Formal user evaluation
Participants
The light, vibration, and sound prototypes were tested on 19 graduate students (eight female, 11 male, age range 20–35), while they were sitting at their desks in their respective workspaces for a naturalistic setting. Graduate students were chosen for convenience. Convenience sampling was appropriate for this study because graduate students can potentially experience periods of work-related stress and thus could benefit from learning deep breathing. For qualitative research, Creswell [38] recommends 5 to 25 participants in a study, while Morse [39] suggests at least 6 participants. Therefore, we collected results from 19 participants and were able to choose a sample size that would be large enough to obtain enough data to answer our research questions.
Protocol
Participants tested the prototypes by taking part in four different sessions over a two-week period. The first session was intended to accustom participants to wearing the breathing sensors. The researcher started the session by instructing the participant on how to breathe from their diaphragm by getting them to place their hand on their stomach and take a deep breath. Once the participant was comfortable with deep breathing, the software program in the prototype averaged five deep breaths of the participant as a baseline using the raw data collected from the two breathing sensors. In each of the remaining three sessions, participants used one of the three prototypes while carrying out an everyday work-related activity (Table 1). Once the device was set up, researchers would leave the space and return one hour later. Each participant was asked to work on their own tasks in their labs on typical everyday activities such as reading papers, programming, designing, writing, and marking undergraduate student work. Each participant experienced each prototype, one per session, with the order of prototypes counterbalanced. None of the participants were familiar with deep breathing exercises before the study.
Schedule for participants where T (Test Breathing Sensors), L (Light Feedback Modality), V (Vibration Feedback Modality), and S (Sound Feedback Modaltiy) were tested on different days
Schedule for participants where T (Test Breathing Sensors), L (Light Feedback Modality), V (Vibration Feedback Modality), and S (Sound Feedback Modaltiy) were tested on different days
After participants used the prototypes for an hour, participants were interviewed about their experience with the prototype. Questions that were asked included: Describe their experience with the prototype; Self-identify if their stress levels changed when using the prototype; Describe any feelings or sensations they had with the prototype and highlight anything that stood out during the experience; and Identify where their focus of attention and awareness was during the hour. Which prototype they preferred (after they tested all three prototypes);
Ethical approval
The study received ethical approval by the Simon Fraser University Human Ethics Committee. In the next section, we describe the design decisions and final interaction design for each feedback modality, as informed by the results of the informal user study.
The design of the feedback modalities
Each prototype is designed to help support a shift of awareness in the quality of breathing towards deep breathing using both the chest and diaphragm. The feedback of the prototype is intended to alert a user when their breathing pattern shifts from optimal deep breathing to shallow breathing. The breathing data is collected from a Thought Technology ProComp 2 system and two respiration sensors [40], which will be referred to in the paper as the breathing sensors.
The two breathing sensors are placed over the user’s clothes around their diaphragm and chest (at the level of the thorax) –see Fig. 2. The data from the respiration sensors are first sent through a smoothing algorithm developed by Thought Technology in the API and then into the visual programming tool, Max 7 Cycling 74 [41] via the middleware M + M: Movement + Meaning CANARIE Networked Enabled Platform (NEP). The middleware allows the data to be sent to a Max patch that controls the dynamic behavior of the prototype feedback.
Thought Technology ProComp 2 system with two respiration sensors placed on the chest/torso of the wearer.
The placement and feedback type varied between the prototypes, as summarized in Table 2. We applied a similar design strategy to each prototype, where the system only activates and provides dynamically changing feedback when the alert mode is entered. When it is not in alert mode, then the output of the prototype is either static (light), still (vibration), or silent (sound). This approach is illustrated in Fig. 3, with a diagram of the user interaction flow for each feedback modality.
Summary of the design choices for the prototypes
Summary of the design choices for the prototypes
Feedback modality user interaction flow. The four stages of interaction are illustrated for each prototype’s feedback modality.
Figure 4 shows the setup for each prototype –where the prototype is placed on the body and in the environment, in the context of a person performing work-related tasks at a computer. The design of the feedback for each prototype is now explained in the following sections.
Final setup of light, vibration, and sound modality prototypes in the natural work environment.
The placement of the light feedback modality began by testing RGB LED strips that were connected to an Arduino Uno microcontroller. We tried a few different colors: red, green, blue, and yellow. Three out of the four participants liked the color blue the best. Initially, we applied a strategy of programming the rhythm of the fading light to emulate the user’s breathing patterns. However, our participants found this strategy did not help them breathe from their diaphragm. We then changed to a more effective strategy of the fading light, acting as a guide to lead them towards deep breathing with a fading pattern of 4 seconds in and 4 seconds out. The parameters of the light fading pattern - fade duration and alert mode threshold - are explained in section 4.2.
For the user interaction flow, we implemented the light modality fading pattern diagram. The diagram illustrates the output of the prototype, in this case, light, in relation to the breathing data. By reading the two figures together (Fig. 3 and Fig. 5 (A)), we can see changes in breathing and how the light feedback varies across the four stages of interaction. We did not test any other ways we could use light for feedback modalities due to the fact that all four participants spoke positively about the fading light.
(A) Light fading pattern diagram; (B) vibration fading pattern diagram; (C) sound fading pattern diagram; and (D) breathing diagram in relation to the stages of user interaction flow.
The vibration feedback modality prototype was developed using three Lilypad vibration motors. With the vibration feedback, we tested the prototype in four different locations: the lower arm, the upper arm, the stomach, and the chest.
The four participants all agreed the chest and the stomach were very uncomfortable. Three out of the four participants liked the lower arm because they felt it was the most comfortable. Therefore, we decided to place the vibration motor on the forearm. When we began testing the prototype with the four participants, we applied a strategy of programming the vibration feedback to emulate the user’s breathing patterns. However, the four participants found this experience uncomfortable, so we changed to a similar guiding strategy as employed in the light prototype. When a user is not breathing optimally from their diaphragm, the three vibration motors placed on the lower arm subtly vibrate one at a time, creating a sensation of being stroked up and down the arm. This fading pattern is illustrated in Fig. 5 (B) for the four stages of the user interaction flow. Controlled by a Lilypad Arduino board, the prototype was wirelessly connected to the computer using an XBee. Due to the need to run Max7 and the middleware software, we decided to use the researcher’s computer to run all the prototypes during user testing. The computer was placed on the participant’s desk in a place where it was not noticeable. The computer screen was able to be closed while the testing occurred. The prototype was designed to wrap around the forearm under the clothes and stay within the body’s intimate space [40].
Sound feedback modality
The sound feedback modality prototype was used with professional studio over-the-ear headphones (AKG-MK-II) that were plugged directly into the laptop. We tested different earbuds and over-the-ear headphones. Three out of the four participants liked the headphones the best, hence our decision to use the over-the-ear headphones.
Similar to the other two prototypes, we implemented the sound modality fading pattern diagram (Fig. 5 (C)). When a user was not breathing optimally from their diaphragm, the system would enter alert mode (stage 2) and trigger one of three natural sound recordings. Unlike the light and vibration feedback, we did not use a guiding strategy to alter the patterning of the output. Instead, the character of the natural soundscape provided some variation in the output. We tested four different soundscapes with the participants, and no conclusion was drawn. Therefore, we decided to give the participants a choice in different soundscapes to use. We began the tests with the soundscape playing, and then the participants would hear silence when they were not breathing optimal deep breaths. However, all four participants preferred to work in silence, and the sound only be triggered when they were not breathing optimally from their diaphragm –we thus changed the system to work in this way (Fig. 5 (C)).
Parameters of fade duration and alert mode threshold
Two important parameters controlling the dynamic behavior of the feedback signal are the fade duration and the alert mode threshold. To determine the rate of the fade duration, each of the four participants were asked to take a few deep breaths. Through trial and error, we determined an optimal duration of 4 seconds for the fade-out and fade-in section of the signal. This is in fact a comfortable rate used for deep breathing exercises in therapy stress reduction exercises seen in Mindfulness-Based Stress Reduction [14, 21], Acceptance and Commitment Therapy [17], and Cognitive Behavior Therapy [19].
We then tested what a comfortable number of breaths were to initiate alert mode. The output of the prototype changes to provide feedback on the quality of breathing. However, we needed to determine the threshold when alert mode would be activated. Through exploration with the participants, the calculation of a personalized breathing threshold was defined. It was concluded that if the participant took five consecutive shallow breaths that were above the threshold, then the prototype would transition to the ‘alert’ mode. To turn off alert mode, participants would have to take five consecutive deep breaths that were below the programmed threshold. This threshold is calculated during the calibration period of the user interaction with the prototype.
Results of user evaluation study
The results of the formal user evaluation are now presented. Two researchers independently coded the interview transcripts using thematic analysis. A consensus was then reached between the researchers on a common set of categories for coding. The categories included Participant’s Awareness of the Prototype, Participant’s Focus or Distraction to the Work-Related Task, Participant’s Reflection on Breath, Participant’s Thoughts, Emotions and Body Awareness, Participant’s Stress Level, and Participant’s Preferred Feedback Modality. We then organized the findings into two sections: (1) feedback modality to support deep breathing, and (2) cyclic patterns of awareness.
Feedback modality to support deep breathing
One of our primary research questions was to find out which feedback modality best supported deep breathing during work-related tasks. Our results indicate that the sound feedback modality was most successful, with vibration and light less so, in supporting participants to perform deep breathing. We describe these preliminary results now and then provide further interpretation in the following section.
Sound feedback modality
Seventeen of the nineteen participants found that the sound modality prototype helped them become more aware of their breathing, and apart from one of these participants, to that end, felt they were more productive than when they were not using the prototype. Participants 14, 15 and 17 felt that with an awareness of the sounds, they were more focused on the task-at-hand. Participant 17 recalled that they were not distracted by the device at all. They stated the device “was really easy to turn off” by initiating deep breathing, and it was easy to continue working on their project.
Only three participants reported that they were distracted by the sound feedback, which resulted in difficulties focusing on their task. Participant 7 stated, “the only thing I noticed is when I heard the sound it slowed down everything. I think not really the sound. Just the sound triggers the breaths and the breaths made me slow down during the work. I feel a little bit weird because today I’m doing some casual writing, so I feel good. But I feel if it were really intense work that can really slow down the kind of work”. For participant 18 the distraction of the sound feedback led to feelings of stress: “felt calm when [the sound] turned off but anxious when [it] would turn back on”.
Sixteen out of the nineteen participants enjoyed using the sound modality prototype. Those who responded negatively to the sound prototype criticized the physical experience of the headphones, not the sound itself. Participant 3, 12 and 13 did not enjoy using the sound modality prototype because they felt that the earphones were uncomfortable. Nevertheless, most of the participants enjoyed using the prototype. Participant 1 felt that this prototype was the “most intuitive and salient” and participant 7 found it “less aggressive and less distracting” than the other two prototypes.
In the initial discovery phase, many of the participants explained that they were excited when the music turned on. However, as the hour progressed, they found the silence comforting. Participant 17 explained that initially “when the sound would turn on, I would get excited. I didn’t really get the hang of it at first, so I was listening to the sounds (wasn’t really trying) but then I got the hang of it and I preferred the silence. [My] first impression changed”. Participant 19 explained that at the beginning of the experience, he thought the prototype used negative reinforcement, but by the end, he “liked the silence”.
Vibration feedback modality
With the vibration modality prototype, ten of the participants found it difficult to move their awareness away from the vibration feedback, while others felt the opposite. Of those who found it difficult to move their awareness away from the prototype to achieve optimal deep breathing patterns, participant 5 felt that “something was always alarming [them]”, and participant 13 felt that the device made them feel “constricted and could not move as much”. Participants who felt the awareness of the prototype was overwhelming were not able to initiate deep diaphragm breathing. While commenting on the vibration prototype, participant 14 indicated “[the feedback] was there all the time. I could feel it from beginning to end. It was terrible”.
The other nine participants felt it was easy for them to initiate deep breathing when they began to become aware of the prototype. Participant 8 found the prototype “comforting” and explained that “it felt like a massage”. Participants 6 and 10 reported that the device felt comfortable and enjoyable only after an initial period of discovery and exploration, during which they accustomed themselves to the device. Participant 6 commented that the prototype “felt like a friend”. Participant 12 enjoyed the “very smooth rolling vibrations” of the prototype.
During the interviews, participants were asked if they participated in any somatic activities that were not deep breathing exercises such as mindfulness. Activities the participants reported included performance art and dance. It was interesting to note that the participants’ enjoyment of the vibration prototype seemed related to their engagement in other somatic activities.
Light feedback modality
Sixteen of the nineteen participants reported that their awareness throughout the session was often focused on a distraction or their work task. Participant 18 explained that: “At the beginning, the device really caught my attention... [I] went from fifty percent of my focus on the ball at the beginning of the session to maybe five percent at the end”. Participants felt it was difficult to pay attention to the light modality prototype when they were focused on their work. Participant 13 said, “I did not notice [the light feedback prototype] as much. I think I was able to ignore it more easily than I was able to ignore the others... I wasn’t focusing particularly on breathing; I wasn’t concentrating on that too much”.
User preferences and self-reported stress
During the interview, we asked participants to report their preferred feedback modality. One participant preferred the light feedback, six participants preferred the vibration feedback, and twelve participants preferred the sound feedback. The user’s preference did not match their ability to perform deep breathing.
We were also interested to know if any participants perceived changes in their stress levels during the session. To analyze the verbal feedback, we only included participants that specifically stated that their stress “decreased”, “increased”, or stayed “neutral”. If the participant did not comment, then we counted them in the “did not comment on stress” category. We have tabulated the self-reported levels of perceived stress in relation to each of the three feedback modalities of the prototypes in Table 3.
Participants’ self-reported links between perceived stress and the type of feedback employed in the prototype
Participants’ self-reported links between perceived stress and the type of feedback employed in the prototype
Ten out of the nineteen participants using the light prototype reported that their stress levels decreased. The remaining participants reported no change in stress levels or did not comment on the prototype’s effect on their stress. The vibration prototype was the least effective at encouraging participants to breathe with their diaphragm, therefore the least successful for self-perceived stress reduction. Most participants did not perceive any change in their stress levels or had no comment with respect to stress. Three participants reported that they were aggravated by the vibration prototype, which resulted in an increase in stress. It was interesting to note that the seven participants who reported a decrease in stress resulting from the vibration prototype also indicated that they participate in other somatic practices such as yoga and dance (these body practices heighten tactile and kinaesthetic perception). Eleven of the participants reported a decrease in stress when using the sound prototype. Those who experienced an increase in stress indicated that the over-the-ear headphones felt foreign and did not fit their ears properly. Four of the participants reported that their stress level did not change when using the sound prototype, and the remaining had no comment to offer regarding changes in stress levels.
We wanted to delve more deeply into understanding how the participants engaged with the feedback modalities and whether there were any patterns of behavior that could provide insight into how best to design these types of feedback. To do so, we applied and extended the Cycle of Awareness framework developed by Núñez-Pacheco and Loke [12]. The original Cycle of Awareness was designed as a tool to support designers in creating wearable technologies to assist people in making sense of computer representations of physiological data such as pulse and breath in correspondence with bodily sensations and feelings. It decomposed the interaction with the technology into a sequence of stages, where the user’s awareness shifted between the device/feedback, their body, and the external context. In our case, the bodily interaction with the device is embedded in the context of working on a task; thus we added the user’s task-at-hand into the cycles of awareness - see Table 4.
The extended cycle of awareness
The extended cycle of awareness
Through analyzing the data with the Extended Cycle of Awareness tool, we discovered that the participants tended to respond to the feedback in three distinct ways. The three main types of response patterns include: (1) completely ignoring the feedback; (2) being overwhelmed by the feedback and not understanding the feedback; and (3) successfully using the feedback to initiate deep breathing and to reflect on the reasons for the change in the quality of breathing. We also noticed a link to the self-identified stress level, depending on what response the participants made.
To aid visualization of the response patterns, we developed a diagram to depict the pathways that a participant’s awareness cycles through in Fig. 6. The participant’s awareness shifted between the primary foci of the task, prototype/feedback, breathing, or other sources of distraction. The complete set of diagrams is shown in Fig. 6. The diagram is composed of a matrix of cells, with the focus of awareness (T: task, A: prototype, B: breathing, D: distraction) across the rows and the stages of interaction down the columns. For each stage of interaction, the level of awareness (not aware, aware, highly aware) can be indicated for at least one of the foci of awareness. We define ‘highly aware’ as a heightened focus of awareness, compared to a typical level of everyday awareness.
Extended cycle of awareness: Patterns of awareness for distinct routes. Pattern 1 is illustrated in A1 and A2, pattern 2 is illustrated in C1 and C2, and pattern 3 is illustrated in B1 to B4.
Next, we discuss each of the three distinct patterns.
The light feedback modality was the only type that participants ignored. Most of the participants reported that they were not aware of the light modality prototype when it would begin to fade. Therefore, participants were not aware of the feedback when it began to go off and either focused on the task or were distracted by some other source not related to the feedback. As a result, they did not become aware of their breathing. This is visualized in Fig. 6, A1 and A2. Our design choices for the ambient light display turned out to be a poor choice, as the ambient character did not draw enough attention to alert the participant to when they were performing shallow breathing.
Pattern 2. Overwhelmed by the feedback and not understanding the feedback
The vibration feedback modality was the only type that participants became overwhelmed by the feedback of the prototype. For those participants that ended the interaction in stage three, they ended up staying highly aware of the vibration feedback on their arm, rather than changing their focus to initiate deep breathing. Participants struggled to focus on their breathing and therefore became frustrated with the device as it did not seem to be working for them. They could not use the feedback constructively to help them shift the type of breathing they performed. Participant 11 observed, “initially, I was trying to make it stop by breathing but I couldn’t concentrate on my work. I was doing work and it kept vibrating... I [couldn’t] adjust my breathing pattern”. The distracting nature of the vibration feedback also made it difficult for them to return focus to the task-at-hand. This is visualized in Fig. 6, C1 and C2. Our design choices for the vibration prototype were inappropriate for some of the participants. It is unclear whether these participants reacted negatively to our specific design or would prefer not to use vibration at all for feedback on their breathing.
Pattern 3. Successfully using the feedback to initiate deep breathing and to reflect on the reasons for the change in the quality of breathing
Several participants who used the vibration and sound prototypes were able to use the feedback to initiate deep breathing successfully. They were able to transition their awareness between the task-at-hand, the device feedback, and importantly, their own breathing to initiate and maintain deep breathing, and finally return to the task-at-hand. This ideal cycle of awareness is visualized in Fig. 6, B1 to B4.
In the interviews, participants with this response pattern made causal links between the changes in the feedback and the quality of their breathing through reflecting on the experience. Seventeen of the participants who were aware of the prototype enough to initiate deep breathing felt a heightened relationship between their body and its reaction to certain tasks. Participant 1 recalled while using the sound modality prototype, “I guess probably the main thing would’ve been just my noticing the relationship between just certain things I would be doing with the rest of my body and the sound because that’s all part of the same package. It’s all part of this heightened state”. Overall, the participants paid more attention to their body and how their environment and tasks affected their emotional state. This new-found awareness decreased their perceived level of stress. The participants reflected that this awareness was due to deep diaphragm breathing.
We found that two of the participants had a tendency to judge themselves, for example, creating an imaginary competition with others was an active barrier. We often give our opinion on an experience –such as if we are good or bad at something. Participant 5 recalled while using the sound feedback modality, “I was more focused on breathing. However, I felt like I was failing when I didn’t pay attention to my breathing”. By comparing themselves negatively to an idealized self, these participants undermined the purpose of deep breathing during the test.
Participants that were able to let go and, instead of judging themselves, were able to reflect on it, identified a feeling of decreased stress during stress-inducing tasks. Seven of the participants observed that tasks that usually caused them anxiety (e.g., sending emails, finances, and programming) did not increase their stress levels during the sessions. Participant 19 observed while using the vibration feedback modality, “during that hour, I don’t think there was a lot of anxiety. The fact that I was writing an email to my boss, I’m sure that’s high on the anxiety level but I do not feel anxious writing to my boss”. Twelve participants who usually experienced increased stress due to the task-at-hand found it was easy to let go and create a calm demeanor during the session. Participant 16 observed while he used the sound modality prototype, “I was stressing about the dates my friend is going to come and visit me and how I mixed them up... I kind of flipped a little bit. Little bit of a panic. I noticed that the stress level went way down super-fast after that. I recovered quickly”.
In conclusion, participants who reflected on their experience and understood why their breathing states changed, felt they were more focused on their tasks and reported that they did not feel as stressed when doing tasks that usually caused them stress. In the following section, we translate these findings into a set of design recommendations.
Design recommendations
The design of technologies to support the practice of mindful behaviors, in this case, deep breathing to promote calm, focus, and reduced stress, should take into account the following set of considerations.
Personalization: Participant’s preference of feedback modality
Each feedback modality that was tested in the study has different design concerns and issues. It is important to state that the issues we found and suggestions for improvement could be easily implemented in future studies. With the light feedback, the biggest issue was that the ambient display was easily ignored. Some suggestions our participants gave for the light feedback modality include the ability to change color, changing the feedback modality into a strip around the computer screen to increase visibility, or even having the feedback directly on the computer screen and subtly changing the color. Alternatively, the light feedback modality could be more useful in other applications where breathing is a primary focus of the task-at-hand. For example, the Breathing Light [24] supports a relaxing exercise by focusing on the light to slow down breathing.
Some of the participants who were not familiar with a body-based practice (such as yoga or dance) felt the vibrations were very overwhelming. The decision to design the vibration feedback as a continuous signal was informed by our user evaluation goal of being able to compare across types of feedback modality. However, in everyday work environments, it could be beneficial to have the vibration modality feedback alert the participant in short vibration bursts instead of being always on until the participant initiates deep diaphragm breathing. This solution can already be seen in many commercial devices such as the Spire [34]. The short bursts of feedback could be more appropriate for everyday wearables, rather than the use of continuous vibrations that led in some cases to distracting and overwhelming experiences.
Only three participants had an issue with the sound feedback and that had more to do with the earphones than the sounds themselves. If the prototype was used in everyday life, rather than the semi-controlled conditions of the study, then participants would have been able to choose their own headphones or earphones. Some participants usually listened to their own music while they work and suggested a variation of changing the sound to shift between their preferred music (instead of silence) and the natural ambient soundscape.
The majority of participants reported the sound prototype as their preference and as the one they found leading to decreased stress. However, participants who did have a body-based practice preferred the vibration prototype over the other two. Therefore, it is important to note that depending on the experience and the user’s background and familiarity with body-based practices, designers should allow for personalization of different feedback options. These options will assist designers in making the most appropriate design choices so that the user develops a comfortable and non-distracting relationship with the technology to increase their health and well-being.
Designing for cycles of awareness
In our study we found that the way we designed the light and vibration feedback led to some participants being either unaware of the (light) feedback or hyperaware of the (vibration) feedback. The sweet spot seems to be a level of awareness in between these two extremes, where the user can successfully incorporate mindful breathing into the flow of work by becoming aware of the device’s feedback and then using the feedback to help modulate their breathing. A proposed design recommendation drawn from our findings is that the feedback modality should alert a user enough so that they are aware of the change of body state but is not so overwhelming that they cannot focus on their body in order to change the behavior. This design solution might seem self-explanatory; however, it can be difficult to find the optimal awareness of both the feedback and the ability to initiate deep diaphragmatic breathing.
The rhythm, intensity, and duration of the feedback are important parameters that require nuanced finetuning in order to support bodily interaction towards a successful experience (in our case, performing mindful deep breathing whilst working). In our prototypes we attempted to do this by slowly fading the light and creating a dynamic vibration pattern with 4-second intervals. However, we found that in the case of the light, the dialogue was not strong enough, and with the vibration, some of the participants felt that they could not ignore it enough to reply to it actively. With the most successful prototype, the sound, participants found they were able to respond to the dialogue easily and initiate deep diaphragmatic breathing. Therefore, the guidance of the technology should be subtle enough so that it does not keep focus on the feedback, but strong enough so that the user does not ignore it.
Höök et al. [25] refer to a correspondence relationship between the user and the system, where the user engages in an active dialogue with the system in order to make sense of how it is guiding their activity and how the biofeedback resonates with the user’s perception of their own body. This concept bears similarity to the Cycle of Awareness framework [11, 12], which emphasizes meaningful recognition and ownership of a device’s biofeedback by the wearer. Through our own study, we extended the framework of the Cycle of Awareness into the Extended Cycle of Awareness, which depicts an active dialogue not only between the participant, their bodily awareness and system feedback, but with the participant’s work-related task and distractions.
The Extended Cycle of Awareness can be used by designers as a design or evaluation tool. In the ideation phase of design, it can be used to map the stages of awareness and interaction, by considering the focus and flow of the user in a specific context. When evaluating a design, or analyzing data as we did, it can be helpful for drawing out and clarifying in a visual format, the patterns of awareness and the cyclic flows that may be supported or impeded by particular design choices. The visual charts can assist in understanding the experience that users may have with shifting awareness between the task-at-hand, the device/feedback, and their bodily interaction.
The role of reflection in designing technology to support deep breathing
One key finding that was not in the initial res-earch questions was the concept of reflection. Self-reflection, or the ability to gain important insights about oneself, was a key aspect that emerged out of participants’ success with performing deep breathing in response to the feedback. Whereas spontaneous reflection occurs without prompting, guided reflection can enhance self-awareness and behavior change beyond the immediate user-feedback intera-ction.
Reflection is often described as a process whereby an individual would review certain personal experiences or issues of interest with the aim of generating a level of meaning, insight, or self-knowledge [44]. In recent years, there has been increasing interest in designing for reflection in the field of human-computer interaction (HCI), mainly due to the pop-ularity of personal informatics technologies (e.g., physical activity trackers) that generate a consider-able amount of personal data [43]. Personal informatics is a class of HCI systems that collects personal information from participants and then presents that data back to them [45]. A class of design solutions and HCI systems that focus on eliciting personal reflection about self are described as participant-driven [44], characterized through a focus on supporting participant agency in reflecting, learning, and taking actions that impact their behavior, particularly in relation to health outcomes. Ahmadpour and Cochrane [46] describe three qualities for design solutions, particularly personal informatics systems, that, when reinforced, can create individual reflections about health and well-being. Those qualities are (i) recognizing reflection as an ongoing and evolving process (rather than a state) that requires continuous feedback to support reflection rather than offering ‘a ref-lection’, (ii) creating personalized opportunities for contemplation, for instance through feedback on pro-gress or efforts, and (iii) informing participant’s behavior through meaningful feedback. Several res-earchers [43–45] stated that designing effective visualization of data in a meaningful format, relevant to the type of reflection intended and different to the format it was collected (e.g., quantified), is an important strategy for supporting reflection.
Returning to the findings from our study, partici-pants that were able to initiate reflection (for example, during the session one participant noticed that the sound prototype turned on when they were distracted) during their experience reported a connection between their breathing state and self-identified stress levels. With a reduced stress level, participants were able to be more prepared for stressful situations and have a better understanding of what types of activities made them both distracted and stressed. Guided reflection also had an unintended impact in the study through the set of structured interviews conducted after each of the four test sessions. We speculate that the guided reflection offered by the interview may have assisted some participants in becoming aware of the interconnection between various events, the quality of their breathing and emotional states, with an accumulated effect as they progressed through the four test sessions. Therefore, structured interviews could be embedded into mobile applications similar to structured therapy sessions found in Mindfulness-Based Stress Reduction [16], Acceptance and Commitment Therapy [17], and Cognitive Behavior Therapy [19]. The reflective component of the system can encourage self-reflection and evaluation of past experiences with the aim of attaining insights into why one’s body data is changing state. This can potentially result in building more resilience in users when stressful situations arise during everyday life.
Limitations
There are several limitations that we faced during the study. Firstly, we used a homogeneous sample, which affected the generalizability of the results. Future studies could benefit from a larger sample size and recruiting participants from different demographics. Secondly, we only ran the study in a single type of work environment, namely a university research environment. Further studies could be conducted in different settings to make sure that the results are not only applicable to the work environment. Thirdly, our method for inquiring into participant stress levels was informal in nature. Established questionnaires from psychology that collect quantitative data pre- and post-study could result in additional understanding of how participant self-reported stress levels change after using the prototype. Another potential way to test stress levels would be to incorporate other sensors, for example Anurag et al. [47] used skin conductivity as a physiological measure of stress that correlates to the participant’s self-report.
Conclusion
In this paper we presented the results of testing three different feedback modality prototypes responsive to breathing data: light, vibration, and sound, with 19 participants. The prototypes were designed to alert participants when they were breathing shallowly from their chest and would stay on until they changed their breathing pattern to deep breathing from their diaphragm. We found that the sound modality prototype helped the majority of participants initiate deep breathing better than the vibration and light. With the vibration modality prototype, only participants that had a somatic practice were able to initiate deep breathing, and most of the participants found the vibration feedback overwhelming and distracting. Most participants ignored the ambient light modality prototype. We also found that participants who initiated deep breathing and were also able to reflect on the quality of breathing, self-identified with a decrease in stress and an increase in productivity. The Extended Cycle of Awareness framework was helpful as an analytic tool for understanding the patterns of awareness and the cyclic flow of experience generated by user interaction with our designed prototypes. It helped to highlight how the different feedback modalities played a role in supporting the user to become aware of the quality of their breathing and whether or not they could successfully transition their attention between the task-at-hand, the feedback, and their breathing. Through this study, we learned that reflection is a key factor in designing mindful technology. This finding was outside our original research questions, but is of significance for future work. We put forward a set of design recommendations for crafting interactive prototypes to support the practice of deep breathing, including the incorporation of structured reflection for enhancing self-awareness and well-being.
Footnotes
Acknowledgments
We would like to thank all the participants, reviewers, Marilyn Cochrane, Emily Cramer, and Moving Stories for their support of this project. This research was supported by the Social Sciences and Humanities Research Council of Canada.
Conflict of interest
None to report.