Abstract
Introduction
Workplace design continues to be the focus of research studies to determine impacts on worker productivity, satisfaction, and well-being. Increased urbanization and careers requiring knowledge work [1] have concentrated workers in buildings with multiple office designs, where both individual focus and collaborative behaviors are necessary to produce high quality knowledge products. Some office designs have solely focused on aesthetic appeal. Some office designers have reacted to business models to reduce costs to corporations and maximize space capacity by placing more employees into fixed spaces. Yet, some designers have applied design, engineering, and psychology principles to enhance productivity with the intent to facilitate behavioral, cognitive, and physical activities associated with knowledge work.
The issues with office designs are numerous, and the research, while strongly supporting the need to examine office designs on a more detailed and empirical level, is not entirely conclusive on what office designs, as a whole, work well for which types of tasks. Because of their predominance, open plan offices (OPOs) are frequently studied, and these offices are used for a multitude of jobs that tend to center on knowledge work or activities requiring employees to use knowledge in problem solving and decision-making. The common design theme is the use of open floor plans with employees placed in close proximity to each other. The OPO often consists of either cubicle designs using dividers rather than floor-to-ceiling walls that serve to establish a space for individual or multiple workers; or the open bay design which simply places workers in an open space where workstations and other workers are visible and audible.
In North America alone, about 40 million employees conduct their work in OPOs [2] and in the United States, 68% of employees are in OPOs [3]. Based on their summary of the literature, Seddigh, Bernston, Danielson, and Westerlund [4] reported that the proportion of office workers in Sweden exceeded the number of skilled labor, “blue collar” workers and this phenomenon applies to most countries in northern Europe. Research from the Corenet Global [5] report indicates that from 2010 to 2012, the square footage allocated to workers decreased from 225 ft2 to 196 ft2 (68.58 m2–53.64 m2). It is somewhat surprising to see the proliferation of OPOs across the globe, even as the negative attitudes toward them seem to increase (at least based on reports in the popular media). In the popular media, discussions are prevalent that support a general dislike of OPOs, primarily on the part of workers who must use them [6, 7]. While the discussions have not been highly research-based, the pattern is evident. Workers have a genuine dislike of OPOs, finding them to be distracting, intrusive, and reflective of foregone days when management constantly “checked” on workers to ensure they were working [8]. According to Kaufman [8], some of the supposedly most innovative corporations in the world (i.e., Apple, Google) rely on OPOs. These corporations may be misled by a “false sense of improved productivity” which seems evident based on reports of employees. Kaufman, an advertising agent, wrote in the Washington Post:
A year ago, my boss announced our large New York ad agency would be moving to an open office. After nine years as a senior writer, I was forced to trade in my private office for a seat at a long, shared table. It felt like my boss had ripped off my clothes and left me standing in my skivvies.
There are numerous and relevant OPO research questions to be explored by researchers to determine the most optimal designs that allow cost savings while enabling efficient and effective task performance, but due to the complexity of OPOs, designing internally and externally valid research activities is not always easy. OPOs consist of people, technologies, physical and organizational environments. They are essentially sociotechnical systems and should be examined with systems-related rules, such as the exclusion of reductionism (the need to use macro-variables rather than micro-variables associated with reductionist science) and equifunctionality, where humans, technologies, and organizations function interdependently and humans are actors who act based on the coupling of all parts of a system. While there may be instances where certain functions are done by humans and others by the technologies present in the workplace, the allocation of function does not decouple the elements of a sociotechnical system such as an OPO. For the purposes of this paper, sociotechnical systems are defined as systems that rely on interrelationships between humans, technologies, and environments to complete shared tasks within a defined time and space (albeit sometimes done using technologies that support telepresence). This definition is based on some of the work by Ropohl [9] and Emery and Trist [10] who focused on the inter-relatedness between people and the technologies they use within contexts of operation. Thus, the purpose of the study presented is to examine human performance from an ecological perspective considering behavior within an OPO while using office technology and in a context that replicates the near environments of OPOs. While the study is not naturalistic by design, an attempt was made to avoid reducing the research setting to the point where a systems focus is lost.
Background
Open plan offices and knowledge work
OPOs are sociotechnical systems; and introduce complexity due to the interdependencies between workers, office technologies, organizational and task features, and the ambient environments [11–13]. Figure 1 depicts the concept of OPOs as sociotechnical systems. The two-headed arrows illustrate the coupling between humans performing as individuals and in dyads or teams, office technologies, and the immediate environmental/ambient contexts. Although not conceptualized here, a systems view also considers OPOs to be embedded within organizational climates and cultures, as well as prevailing work cultures in a particular society. Humans in individual settings or in groups are represented at the personnel level. Workers have various cognitive and behavioral capabilities that will mediate or moderate their performance, including how they interface with the external and with the technologies required to perform their work. These human-human, human-technology and human-environment interfaces may be influenced by workers’ capabilities in focused and divided attention, individual differences in how information is processed (e.g., differences in working memory) and how workers choose to collaborate or work individually.
The technological component consists of the technologies used by humans, some of which are as basic as workstation tools such as seats and desks. But, printers, computers, mobile devices, head sets, and desk phones are technologies that introduce a number of ambient stimuli into the environment. Ringer noises, printer noises, and server noises, which may range from very low to close to 65dBA, can be particularly annoying to workers in OPOs. As conceptualized in Fig. 1, work in OPOs is conducted in an interdependent space, where the coupling of humans, technologies, and environments must be appropriate for the nature of the work to ensure productivity and worker satisfaction. Unfortunately, the context in which office work is conducted is often non-optimal for performance and productivity, because the inter-dependencies are not explored or understood. The study to be discussed here does not, in fact, address all interdependencies; as such a simulation would require significant resources and time. However, efforts were made to explore a subset of interdependencies. Thus tradeoffs were made between realism and control.
Often, the user-focused intent of office design is to support knowledge workers who rely on multiple modes of operation to perform their jobs, but decisions are often made without fully understanding the demands of office work and other factors that will affect worker performance, such as technologies to be used and types of cognitive requirements. Individual differences in capabilities and preferences are also not often identified. Knowledge work spans numerous occupations, educational levels, and organizations and is broadly defined as “discretionary behavior focused on the use of knowledge [1, p. 10].” Through an extensive review of definitions of knowledge work, Kelloway and Barling [1] found four common components of knowledge work, and include jobs that: Creating new knowledge and innovations. Applying existing knowledge to find new solutions to problems. Packaging knowledge to transfer to others. Acquiring existing knowledge to conduct research or training.
Many knowledge workers’ tasks are comprised of all or some of these components, resulting in the need to manipulate information and data to conduct work-related tasks. This social-cognitive demand is significant, and workers must rely on individual efforts and collaborations to conduct the work in a manner that produces high-quality knowledge products.
The capacity to concentrate more office workers in one space is associated with direct cost savings for corporations, and this belief has driven the proliferation of OPOs. Cost savings of 20% have been indicated for OPS when compared to traditional offices (private offices) [14]. The data substantiating the cost savings is not clear. OPOs are also believed to encourage team building and collaboration. However, workers complain that with increased collaboration, the need for privacy and quiet work is no longer supported [15–17]. The activity of “work”, especially knowledge work, requires work spaces that support various modes of operation [18]. Based on worker surveys in high-performance work environments, a 2008 study, Gensler, a consulting agency, identified four modes in which office workers engage: 1. Focus mode; 2. Collaboration mode; 3. Learn mode; and 4. Socialize. These four modes were considered necessary to conduct work tasks and engage in workplace culture, and should be used as a design foundation for knowledge work.
Summary of open plan office issues
While some studies have found major and moderate links, others have found none. In the extant literature, however, there is generally consistent support for negative impacts of OPOs on worker satisfaction, some aspects of performance, and perceived stress. Oommen, Knowles, and Zhao [19] reviewed over 50 research studies on OPOs and found some important patterns across all studies. First, workers in OPOs complain of a lack of personal privacy. Workers also complained of noise, limited personal space, and interruptions of their work due to auditory and visual distractions. Table 1 provides a summary from the review.
The Cost-effective Open Plan Environments [20] project used a survey method to identify the barriers and facilitators of work in OPOs. Participants were 779 office workers in large cities located in Canada and the United States. The primary goal was to determine the factor structure of the data using exploratory factor analysis with oblique rotation (oblimin method). Three dominant factors emerged and accounted for 50.71% of the common variance: 1. Satisfaction with privacy/acoustics; 2. Satisfaction with ventilation/temperature; and 3. Satisfaction with lighting. Upon examination of the privacy/acoustics factor, ten items loaded on this factor in a range of 0.51 to 0.79. These factors are provided in descending order from highest factor loading to lowest (p. 8): Amount of noise from other people’s conversations while you are at your workstation. Level of privacy for conversations in your office. Degree of enclosure of your work area by walls, screens or furniture. Level of visual privacy within your office. Frequency of distractions from other people. Distance between you and other people you work with. Amount of background non-speech related background noise you hear at your workstation. Size of your personal workspace to accommodate your work, materials, and visitors. Your ability to alter physical conditions in your work area. Aesthetic appearance of your office.
A pattern persists across items that seem to support the sociotechnical and interdependent nature of OPOs. For instance, socio-cognitive factors play a role when spatial layouts allow workers to hear other workers conversations as the respective workers try to engage with workstation technologies. Similarly, non-speech related stimuli such as printers and phones seem to be linked to workers’ perceptions of their workstations. Other factors such as control, size, visual privacy, and distractions are also evident in the items that account for one of the factors.
Veitch, Newsham, Boyce, and Jones [21] used a linked mechanisms map to test the effects of lighting in open plan offices on attitudes, cognitive, and behavioral factors. Participants worked in a simulated open plan office. At strong link was found between satisfaction with lighting and attitudes about the attractiveness of the office space. This was the largest effect size identified through mediated regression analysis. Additional effects were found for the link between the variables associated with preference for lighting and mood and the variables associated with health and well-being. This study has highlighted the complex nature of analysis and study of open plan offices as a sociotechnical system, with linked mechanisms between physical environment, technology (office objects), behavior, and cognition.
Open plan office designs, per se, are not inherently negative for workers. The contributors to worker dissatisfaction relate to the distracters that accompany open plan office designs. For instance, data from over 3000 survey respondents were analyzed relevant to open plan offices [22]. Participants used an open-ended response and ratings to report annoyances in open plan offices with and without windows. The primary annoyances were: Conversations of proximal workers Overhearing others talking on phones Noise produced by office equipment such as printers and shredders Echoes
Several studies have found significant negative correlations between job satisfaction and ringing phones, nearby conversations between workers, and workers talking on phones, as well as annoyance with noises while working [22–24].
Not only are these events associated with annoyance, but the number of workers within an OPO space seems to be related to the number of sick days a worker experiences. In a study of 9,252 workers ranging in age from 18–59, researchers have found that, compared to cubicle offices with one worker, cubicle offices using shared space (two to six workers in the same cubicle space) reported more sick days [25]. These results were based on adjustments for age, gender, socioeconomic status, body mass index, alcohol consumption, smoking habits, and physical activity.
Some studies have focused specifically on cognition or cognitive acts predominant in knowledge work. Haapakangas, Kankkunen, Hongisto, and Virjonen [26] examined the effects of speech masking on subjective ratings acoustic satisfaction, mental workload (using the NASA TLX) and performance. To mask the intrusion of office noise, these researchers used spring water sounds and vocal music in separate conditions. Compared to spring water sounds, vocal music negatively impacted mental workload, acoustic satisfaction, and performance. The nature of speech, even when accompanied by music (as in a song) seems to interfere with cognitive processing in office-related tasks.
Other research has also supported a similar pattern. Smith-Jackson and Klein [12] studied the impacts of speech distracters on task performance and subjective mental workload (using the NASA TLX [37]). This study developed hypotheses that were framed on the basis of two specific theories – the Irrelevant Speech Effect [27] and the Changing-State Hypothesis [28]. The irrelevant speech effect posits that speech is a distracter because of the load it imposes on the operator’s working memory. Speech is a distracter when operator’s try to conduct work that requires processing verbal resources. In essence, surrounding speech competes for the same working memory resources a worker is using to process verbal information. The overlap might occur because of activities such as reading, processing what is read in order to make decisions or produce knowledge outputs, watching and understanding vocalizations from speech-based tasks such as listening to others on a conference call or watching individuals who are speaking in videos. The irrelevant speech theory also posits that, the more meaningful the non-task related speech, the more distracting it will be because meaningful speech draws attention. Another theory is the Changing-State Hypothesis [28], which posits that the source of distraction from non-task related speech is due to the degree to which the non-task related speech simulates natural patterns of speech stored in the workers long-term memory. If non-task related speech follows a pattern identical to a conversational schema as perceived by the operator, then the non-task related speech will be distracting in the context of performing a verbal task. If the speech is vocal music, for example, it will not be as distracting as hearing a conversation. The area of contention between these two theories also parallels the former and existing disagreements regarding attentional filtering for specific tasks. In the Irrelevant Speech Effect, disruptions would be predicted to occur in late filtering within the attentional mechanism, while in the Changing State Hypothesis, the processing for relevance/irrelevance would occur in the early selection stage, where meaning is not yet processed, but other non-verbal components are processed (i.e., tone,inflection).
The Smith-Jackson and Kleiner [12] study used a proofreading task with a sample of 54 participants. The three speech interference types used were continuous speech (using 2 individuals speaking), discontinuous speech (one-sided speech developed by editing one person from the conversation), and the quiet condition (representing non-speech interference). The speech recordings produced sound levels of 65 dBA. Ambient sound levels during the quiet condition were about 45 dBA; sources includedcomputers and heating and ventilation systems.
The speech was not related to the task; two males were recorded while discussing a movie they had both seen. The discussion was somewhat controlled, in that both males were instructed to speak as naturally as possible and to share the time so no one person dominated the conversation. Additionally, the conversation was somewhat neutral or absent of significant distracters such as inappropriate language or laughter. A proofreading task analysis program [29] was used to generate an editing task for 54 participants to edit misspelled words, homonyms, and spoilers (illogical words that do not fit in the context of a sentence). The study used a within-subjects design, counterbalancing all three conditions, and included individual differences measures related to focused attention. The findings, in sum, showed disruptions in performance related to verbal interference, task performance, and mental workload compared to quiet, but mostly in the context of individual differences. The presence of full meaning (continuous speech) disrupted task performance and increased mental workload more than the other two conditions. Participants with stronger focused attention capabilities (based on subjective responses to an absorption scale) showed less performance disruption compared to those who had more difficulty with focused attention. While verbal interference seems to disrupt performance and increase mental workload, the final conclusion is that there are many factors that moderate the relationship between speech in OPOs and verbal task performance.
Several other studies have found performance decrements caused by noise and/or visual distractions. Similar to the Smith-Jackson and Klein [12] study, the effects were in the same or different directions as those hypothesized [30–33], which again demonstrates the complexity of studying OPOs. Fig. 2 provides a conceptual model of the proposed links between the environmental attributes of OPOs, mental workload, and performance.
In spite of the plethora of research, there have yet to be any studies that fully replicate open plan offices with office workers conducting natural tasks. Such an idea was posited by Gale and Christie [34], who stated the need for ecologically valid studies to be conducted to understand the complex nature of office work, understand the evolution of office work, technologies, and workers, and make predictions necessary to support future office designs. Gale and Christie described a concept referred to as the
Objectives
This study was organized using a conceptual hierarchy [13] that was based on Rasmussen’s [36] approach to organize thinking, problem solving, and innovation using multilevel identities that relate to complex functions and purposes (Fig. 3). A conceptual hierarchy for OPOs is necessary to organize the language, variables, and design thinking associated with key sociotechnical attributes. The conceptual hierarchy was developed using four levels of abstraction and considered the OPO as a sociotechnical system: 1) Overall purpose of the OPO research setting; 2) Functions of the sub-system components; 3) functions of the physical technologies in the operating space; and 4) Specific features and spatial relations of artifacts within the operating context. The conceptual hierarchy was used to organize the study to provide some level of naturalism while also controlling selected variables. Three hypotheses were tested to determine the impact of speech interference and specific individual differences on performance and mental workload. Office layout was also tested by comparing dependent measures using two types of office layouts – cubicle and open bay.
Method
The study was designed to allow some control over the presentation of stimuli and minimize control on certain factors. In this way, the research environment was a quasi-naturalistic environment in the form of a scaled world; providing control yet naturalism to the extent possible. For instance, participants could adjust the workstation monitors and chairs to comfortable heights, and the voice recordings consisted of natural discussions.
A mixed methods and mixed factor design was used. In the mixed methods component, data formats were quantitative and qualitative. Both subjective (self-report) and objective (performance) methods were used. Subjective methods included interviewing participants, measuring mental workload using self-report, and identifying individual differences using self-report questionnaires. Objective methods consisted of using a simulated (scaled world) environment to gather data and use of scoring sheets to measure task performance. Researchers used scoring sheets to maintain consistency of scoring of all tasks.
A 2 (task type) X 3 (interference type) X 2 (office layout type) mixed factor design was used. Task type and interference type were within subjects; office layout type was a between subjects factor. Half of the participants were assigned to the cubicle condition and half were assigned to the open bay condition. The independent variables were task type, interference type, and office layout type. Task type had two levels – web navigation and document editing. There were three levels for interference type – continuous speech, discontinuous speech, and quiet. The order of assignment to conditions was counterbalanced using a Latin Square.
The individual differences measures were also used as independent variables based on groups established by using a mean split. Additional dependent variables were task accuracy on the document editing task and the web navigation task and subjective mental workload.
Three hypotheses were tested:
Hypothesis 1: Task performance will be negatively impacted by speech interference conditions compared to quiet/no speech.
Hypothesis 2: Mental workload will be higher in speech interference conditions compared to quiet/no speech.
Hypothesis 3: Individual differences will influence performance and mental workload.
Participants
Participants were selected using advertisements on radio stations and fliers posted in populated areas. Participants were screened, and twenty were selected for the study. Because of missing data, three participants were dropped from the data set. The criteria for participation included: 18 years of age and over; typing speed of at least 30 wpm; at least one year of experience in an office work environment; and normal or corrected vision (self-reported). Participants were screened for typing speed using Spyrix KeyloggerTM and a typing task. The mean age of participants was 36.28 years (SD = 14.82); 71% were female. Sixty-five percent (65%) of the participants used computers five or more times per day; the remaining used computers one to four times per day. Years of experience in office environments varied widely from one to 45 years, with a median of eight years.
OPO environment
The CAFÉ OF EVE set-up was implemented in a 19 ft. (5.79 m) X 29 ft. (8.84 m) space, which allowed the placement of two types of OPO layouts – open bay, which uses only a visual divider between the participants’ workstation and adjacent workstations and cubicle design, which uses enclosures around the entire workstation. The open bay layout divider measured 3 ft.2 (0.91 m2) while the cubicle dividers measured 4 ft. (1.22 m) wide by 5 ft. high (1.52 m). The workstations in each design conformed to the design heights recommended in ISO 9241, and the seats were adjustable. Figure 4 illustrates each design (participants’ seated in adjustable chairs are not shown). An identical collection of objects was placed in each workstation and included such items as paper racks, post-it notes, pens, staplers, paper, and folders. An HP 4000 Series office printer and Gateway model 900 series server tower was used to provide ambient background noise and both were placed equidistant from participants regardless of the office layout used. The server ran constantly, while the printer noise was activated at set intervals during task performance. Combined, both devices produced ambient dBA levels ranging from 45–52. Overhead lighting consisted of fluorescent bulbs and ranged from 500–558 Lux. Temperature ranged from 72–75°F (22.22–23.89°C), measured at the beginning of each session.
Apparatus and questionnaires
Demographics: A demographic questionnaire was used to elicit information regarding age, gender, years of experience in office environments, educational level, hours of computer usage, and ethnicity.
Key Logging and Screen Capture: Snagit® by Techsmith was used to capture screen shots as participants completed tasks. Spyrix KeyloggerTM was used to capture keystrokes.
Mental Workload: The NASA Task Load Index (NASA TLX) [37] was used to elicit self-reports of mental workload.
Individual Differences: The Cognitive Failures Questionnaire [CFQ; 38] was used to acquire self-reports of lapses in memory, perception, or attention. This scale uses a 0 (never) to 4 (always) Likert-type scaling format. Sample items are as follows: Do you read something and find you have it been thinking about it and must read it again? Do you find you can’t quite remember something although it’s ‘on the tip of your tongue’? Do you fail to hear people speaking to you when you are doing something else?
Broadbent et al.’s questionnaire is a unidimensional questionnaire with a Coefficient alpha reliability of r alpha = 0.79. The scale has been validated for social desirability (using lie scales/items) and has high construct validity when measured against psychometric scales for memory, slips and lapses.
The Perceived Stress Scale [PSS; 39] was used to measure global stress; targeting stress experienced in the past six months. This Likert-type scale used a 0 (never) to 4 (very often) psychometric scale format. Coefficient alpha reliability measures ranged from r
alpha
= 0.84 to 0.86 using three different samples of participants. Construct validity was established using correlations with Life Events scales. Sample items include: In the last month, how often have you been upset because of something that happened unexpectedly? In the last month, how often have you been able to control irritations in your life? In the last month, how often have you felt that you were on top of things?
The Expanded Tellegen Absorption Scale [12, 40] measures the ability to focus attention, independent of focusing attention on somatic states. In a previous study [12], a 2-factor structure was identified that included task absorption and imaginative absorption. Scale anchors were (1) uncharacteristic to (5) characteristic. Sample items are as follows: I listen so hard when having a conversation that I am not distracted by the world going on around me. One of my better qualities is the ability to put 100 percent of my attention to a task at hand. When concentrating on a task, it is difficult for other people to get my attention.
Tellegen and Atkinsson [40] isolated one trait factor, absorption, which seems to reflect a focused attentional ability. In a previous study, an Expanded Tellegen Absorption Scale [ETAS; 12] was developed using additional items to measure focused attention. Using exploratory factor analysis, this scale demonstrated a reliability of r alpha = 0.77 to 0.78. Construct validity was assessed using the Cognitive Failures Questionnaire and demonstrated strong correlations with specific items on the ETAS.
Tasks for this study included an editing task and a web navigation task. The editing task included three equivalent documents selected from an economics text book (sample in Appendix A). Three types of errors were added to the documents–homonyms, misspelled words, and spoilers. These documents were rotated across participants and across conditions. Documents were equivalent based on length and readability (Flesch-Kinkaid at 10th grade level). The web navigation task required participants to answer travel-related questions to plan a trip from origination to destination (Appendix B).
Interference: Speech interference conditions were developed using recordings of conversations between two people. Recordings were not highly controlled in terms of gender or pacing of speech. Speakers were only instructed to discuss social events and project-related work, and were also asked to avoid highly emotional or controversial topics. Topics were approved before recording and several recordings were made. The researchers selected recordings that were least provocative, yet most natural.
Recordings were played using an Altec Lancing Multimedia Computer Speaker System. Decibel readings were taken at the participants head to ensure the sound level was consistent for all participants. Continuous speech using a recording of two individuals speaking at 60 dBA to 65 dBA (conversational speech levels in near proximity). Discontinuous speech at the same dBA using a recording of only one side of the conversation from the same recording used for continuous speech. This one-sided version was used to mimic a phone conversation where workers here only the individual in proximity to them. Quiet condition (control), where the only sounds were ambient office sounds ranging from 40 to 45 dBA. Most of the ambient sound in the room was attributed to a Gateway Model 900 series server.
Additional Distracters: An HP Laser printer 4000 series was used as an additional distracter. During the 5-minute task performance, the printer was activated by researchers at the 1.5 minute mark. One page was printed. In the final 1.5 minutes of the task, a cell phone ring was emitted and then stopped.
During task performance, one confederate was seated diagonally and behind visual dividers for both conditions. During task performance, the confederate typed on a document laptop keyboard using the same document each time, and at roughly the same speed. Thus, each participant experienced typing noises from a cubicle (located diagonally from where they sat) or on the other side of the open bay divider (and diagonally). The confederates were blocked visually from each participant.
Procedure
This research was reviewed and approved by the Institutional Review Board at North Carolina Agricultural and Technical State University. Upon entering the session, informed consent was reviewed and acquired from participants and eligibility was verified. Participants then completed the demographic questionnaire and the individual differences questionnaires. Individual differences questionnaires were counterbalanced for order of presentation. Once completed, the performance session began.
Participants were provided with general instructions of the tasks to be performed and were instructed to complete all tasks as quickly and accurately as possible. This instruction was used to avoid forcing a speed/accuracy tradeoff. Before beginning, the paired comparisons NASA TLX workload cards were given, allowing participants to prioritize the dimensions based on their perceptions of the greatest contributors to workload. After the workload sorting cards were completed, participants were given further instructions on the purpose of the study.
Participants were given a familiarization task that included 2.5 minutes of an editing task and 2.5 minutes of a web navigation task. After the familiarization task and once all questions were answered, the experimental session began. Each task lasted five minutes. Participants completed six tasks based on the 2 (task condition) X 3 (interference conditions) counterbalanced assignment. After each task was completed, NASA TLX ratings were acquired on the six dimensions. In this paper, only the quantitative data is discussed. After completing the session, participants were compensated for their time ($25 gift card). Experimental sessions lasted about 1.25hours.
Data analysis
Internal consistency checks were conducted on the three individual differences questionnaires, Perceived Stress Scale (PSS), Expanded Tellegen Absorption Scale (ETAS), and the Cognitive Failures Questionnaire (CFQ). The criterion for acceptance was set at 0.70, following the convention of Nunnally [41]. The PSS yielded low reliability initially. Ultimately, four items were retained resulting in r alpha of 0.86. The ETAS consisted of two dimensions. The first dimension, imaginative absorption, measures a tendency of individuals to be lost in their own thoughts. This subscale yielded r alpha of 0.70, so all 7 items were retained. The second dimensions measures task absorption, or a tendency to become immersed in ones’ tasks. Two items were found to have low item-to-total correlations, and were subsequently dropped. The final r alpha was 0.72. The CFQ, a unidimensional scale, yielded r alpha value of 0.72. All items were retained.
The two-level office layout factor yielded two groups. The cubicle layout sub-sample was n = 7, while the open bay layout sub-sample was n = 10. Variance equivalence was tested using the Folded F statistic on all dependent variables to determine whether the homogeneity assumption was met. Variances were equivalent for both groups.
It was expected that participants’ scores would converge in general toward a performance pattern demonstrated by the strength of the correlation. No such convergence was found, with the exception of performance in the quiet and continuous conditions. Editing and web navigation task performance were significantly correlated in the quiet condition at r s (14) = 0.54, p < 0.05. Likewise, editing and web navigation task performance in the continuous speech condition were significantly correlated, r s (14) = 0.53, p < 0.05.
Collinearity was checked using Spearman rho correlations. None of the variables were highly correlated (>0.80), with the exception of ratings across all speech interference conditions as predictors for the temporal and physical dimensions of mental workload. These two dimensions were thus removed from the analyses. Since the individual differences measures were also used as predictors, collinearity was also tested. None were highly correlated so all individual differences predictors remained in the analyses.
A Shapiro-Wilk test was conducted on the dependent variables of task accuracy and mental workload. Results indicated a mix of normal and non-normal distributions. Thus, a General Linear Model (GLM) was used to conduct multivariate and univariate tests. Pillai’s trace was used for the F-test of multivariate effects across the dependent measures, due to the different group sizes within groups. Multivariate post-hoc tests were conducted using the Method of Contrasts to conduct means comparisons. The least squares means method was used for post-hoc means comparisons for all univariate tests.
A multivariate General Linear Model (GLM) was used to test the within and between subjects effects, using layout as the predictor. Mental workload values and task accuracy were entered as dependent measures in two separate models to test H1 and H2. Additionally, separate models were used to test the individual differences hypothesis (H3). The test of H2 was conducted using a GLM. Workload values were calculated using the product of the paired comparison weights and the ratings on each dimension for each task. A mean workload rating across task types was also calculated and this value was entered into a MANOVA model using the GLM procedure. Hypothesis three (H3) was tested by splitting the individual differences variables into two groups (high scorers and low scorers) and groups were adjusted for balance (8 in the low groups and 9 in the high groups).
Results
Effects of interference types on performance
A main effect of speech interference condition was found at the criterion p-value, F (5,10) = 3.22, p = 0.05. Based on the post-hoc tests, web navigation task accuracy in discontinuous and continuous speech conditions was significantly lower than in editing during the continuous speech condition. Across all conditions, editing under continuous speech conditions produced the highest task accuracy. Figure 5 illustrates the performance differences across conditions. No other performance differences were identified.
Effects of interference types on mental workload
The Pillai’s Trace multivariate F-value was significant, F (2,15) = 4.36, p < 0.05 for effects of interference types on mental workload. Post-hoc tests revealed that mean workload ratings on the web navigation task in the discontinuous condition were significantly higher compared to the continuous speech conditions (Fig. 6). No significant differences were found when this same test was conducted on the editing task.
When mental workload measures were broken out by dimension, a significant effect of condition was found on participants’ perceptions of how well they thought they met the task goals, F (2,14) = 3.83, p < 0.05. Higher ratings indicated participants perceived their own task performance to be poor or inadequate. Post hoc tests revealed the ratings of performance in the continuous speech condition were significantly poorer (higher mental workload) than in the discontinuous speech condition. Figure 7 illustrates the results.
Effects of individual differences on performance and workload
For multivariate effects, a significant interaction was found between task performance accuracy and task absorption scores, F (5,7) = 4.34, p < 0.05 across the conditions. Univariate tests revealed a significant difference between high and low task absorbers in the quiet condition for the web navigation task, F (1,15) = 4.47, p < 0.05. Those who reported high task absorption or a tendency to focus strongly on primary tasks while filtering distracters had lower accuracy scores compared to those who reported a lower propensity to focus on primary tasks while filtering distracters (or low distractibility) (Fig. 8). One pattern across the individual differences data can be seen in the higher variability under interference conditions among low task absorbers compared to high task absorbers (Figs. 8 and 9). The reactive differences in low task absorbers when faced with a different interference condition was more variable. No other effects were identified. A similar pattern emerged from the means of accuracy on the editing task; however, the differences were not significant (Fig. 9).
Effects of experience on performance and mental workload
The individual differences hypothesized to influence performance and mental workload were trait-related individual difference reflecting dispositions. However, skill-based individual differences may also influence how workers perform in OPOs. Two secondary factors were selected for exploration as a means to check for confounds – age in years and years of experience in office environments. Spearman rho correlations were high for age and years of experience, r s (15) = 0.85, p < 0.0001. So, experience was entered into separate GLM models as a two-level predictor variable to determine the effects on performance and mental workload. No significant effects were found.
Discussion and conclusions
Hypothesis one focused on differences in performance between the quiet (non-speech) and speech interference conditions. When comparing across all tasks, web navigation task accuracy was significantly lower in the discontinuous and continuous speech conditions compared to the editing task in the continuous speech condition. The editing task in the continuous speech condition actually showed the highest accuracy scores, albeit non-significant with the exception of the two aforementioned conditions. The editing task, although more verbally intensive, did not seem to be as sensitive to differences in speech interference types. Since the knowledge work necessary to conduct the navigation task was more difficult and varied (reading, making decisions, navigating, and selecting), this may account for relatively higher accuracy for the editing tasks overall compared to the navigation tasks. Overall, the quiet condition showed now advantages in performance accuracy.
Hypothesis two focused on mental workload. Mental workload measures were also found to show more patterns for the web navigation task than the document editing task. Overall mental workload ratings for the navigation task were higher in the discontinuous condition compared to the continuous interference condition. One additional finding was that ratings on the extent to which participants perceived their performance in terms of how well they performed the task was significantly worse for the continuous speech condition compared to the discontinuous or quiet conditions.
By far, it is apparent that at least one individual difference does influence task performance. Participants were divided into high and low task absorption groups using scores on the task absorption dimension of the ETAS. At least for the web navigation task, low task absorbers (less focused) performed significantly better than high task absorbers in the quiet condition. The same pattern occurred across the other two conditions, but significance was only found for the quiet condition for the web navigation task. A similar pattern was observed for the editing task but the differences were not significant. None of the other individual differences demonstrated an effect on performance or workload and no effects across interference types. The findings, in sum, showed disruptions in performance related to verbal interference, task performance, and mental workload compared to quiet, but mostly in the context of individual differences. The presence of full meaning (continuous speech) disrupted task performance and increased mental workload more than the other two conditions. Participants with stronger focused attention capabilities (based on subjective responses to an absorption scale) showed less performance disruption compared to those who had more difficulty with focused attention. While verbal interference seems to disrupt performance and increase mental workload, one conclusion is that there may be many factors that moderate the relationship between speech in OPOs and verbal task performance.
The results of this study were mixed. Consequently, this study only adds to the efforts to address the impacts of OPO designs. The complexity of studying the OPO phenomenon from an empirical perspective and within a scaled world replica continues to be a challenging endeavor. It is clear, however, that experience in office environments, does not impact performance or mental workload under distracting conditions. This is important, especially as some anecdotal discussions around OPOs suggest that workers will ultimately get used to functioning in OPO settings.
Similar to Smith-Jackson and Klein (2007), some performance disruption occurred in the speech conditions when compared to each other. But, unlike the previous study, performance in the quiet condition was no better than performance in speech interference conditions. This raises the question of whether the ambient noise in the quiet condition is actually just as distracting as the higher sound levels in the speech conditions. In essence, the comparison is between two speech conditions at 65 dBA, and one condition at 45 dBA, albeit with no speech. Perhaps this difference is negligible and equally distracting. In other words, the true impact may simply relate to noise of any kind, which challenges the explanatory power of Salame and Baddely’s Irrelevant Speech Effect [27]. It may not matter how much speech, but what matters is the level of noise. Moreover, Jones’ Changing State Hypothesis [28] would have predicted more interference in the continuous speech condition, yet editing task accuracy in the continuous speech condition showed the highest performance accuracy and was significantly higher compared to two other conditions.
The previous 2009 study also showed that high task absorbers performed better in all conditions compared to low task absorbers, yet the opposite trend occurred in this study. Those reporting lower capabilities in focusing attention performed better than those reporting strong abilities to focus attention. Maher and von Hippel [42] found that individuals who were poor in inhibitory ability were less satisfied with open plan offices. The extent to which inhibitory ability and task absorption co-vary to account for individual differences in open plan office contexts should be explored in future studies. Additional explanations may relate to self-perceptions of focused attention capabilities, which may be problematic, but the psychometric scales used in this study had strong reliability and convergent validity. One interpretation of the seemingly contradictory findings in this study may lie in the advantage of being distractible. While multitasking from a cognitive perspective is not likely to happen (the cognitive system simply switches rapidly between tasks), perhaps the tendency to be distracted provides an advantage to low task absorbers because they switch between tasks more easily. Or, it is possible that highly distractible individuals are not as disrupted or stressed by interference as much as those who are less distractible. The phenomenon of task set inertia could be relevant also. If switching is more difficult among high task absorbers, then constant disruptions might break task set inertia making it more difficult to switch back to the primary task [43]. The pattern of relative stability across conditions demonstrated among high task absorbers. This pattern is of interest and should be pursued in future studies.
An important observation emerged from the results – the difference between how participants perceived their performance and how they actually performed. Overall, performance trended higher in the continuous speech conditions. But, when examining mental workload ratings associated with perceptions of performance, participants perceived their performance to be worse in the continuous speech conditions. Thus, performance and mental workload, at least on this dimension, have dissociated. Participants’ perceptions of performance did not correlate with their actual performance. The short task times may not have been sufficient to allow participants to gauge the impact of speech on their working memory.
The study discussed here is only one part of a larger study; thus the sample size was small. The question remains: What matters in OPO design and to whom does it matter? The results are mixed and indicate advantages and disadvantages associated with individual differences. Compared to the previous study, this study has found additional support for the impact of individual differences on performance under OPO-simulated conditions. While the pattern is not fully understood, it is clear that individual differences should be examined as moderators or mediators in explaining the relationship between OPO-simulated interference and outcomes such as performance and mental workload.
In terms of recommendations for design, it is important to understand the types of knowledge work and cognitive processing within work systems. Further studies are needed to establish a fully operational CAFÉ of EVE to examine performance across longer periods of time to identify the dynamic patterns across a typical workday. This study provided a foundation on which to test and develop design frameworks for OPOs.
Conflict of interest
The authors have no conflict of interest to report.
Footnotes
Acknowledgments
This research was funded by the Office Ergonomics Research Committee and the research team is grateful for the resources provided to support the CAFÉ of EVE development and study implementation. We are also thankful to Demetrius Martin who assisted with the design layouts and Troy Hayden for his participation on the team.