A Systematic Literature Review on Healthcare Facility Evaluation Methods

Understanding POE and Its Outgrowths and Implications

Some authors, such as Altizer et al. (2019) and Davoodi et al. (2021), cited Preiser et al. (1988) to define POE as an evaluation process, carried out systematically and rigorously, in buildings constructed and occupied for some time—preferably, occupied for at least 6 months (Cranz et al., 2021). Ornstein et al. (2009) pointed out that this process is a set of appropriate, comprehensive, and objective methods and techniques applied to the building in use, which include local and remote observation, such as walkthroughs, interviews with stakeholders, focus groups, and so on (Altizer et al., 2019; Barnes, 2002; Cranz et al., 2021; Ornstein et al., 2009).

Specifically, the main focus of the POE is to evaluate the social performance of a building, because it promotes a systematic assessment of an environment in use to verify the perception and the satisfaction level of the people who use it (Cranz et al., 2021; Phiri & Chen, 2014a; Shin et al., 2017). This includes, but is not limited to, functional requirements for thermal and light comfort, humidity and ventilation, air quality, usability, flexibility, maintainability, and so on (Ornstein et al., 2009; Yang & Guangsi, 2020).

Broadly speaking, POE introduces two purposes. The first is to present a rigorous and continuous evaluation of a social and behavioral perspective of a building’s performance. Therefore, explicit human needs with current building performances can be compared (Brambilla & Capolongo, 2019; Connellan et al., 2013; Cranz et al., 2021; Preiser et al., 1988; Woon et al., 2014). The second purpose is to verify in this process if the principles and guidelines that the design intends to follow were achieved and if they were adequate. Thus, POE can evaluate the rationality of design strategies from a behavioral and social psychology perspective (Altizer et al., 2019; Yang & Guangsi, 2020). By doing so, POE not only provides feedback for improvement in the analyzed building but also promotes continuous improvement in future designs, by allowing learning from previous experiences and applying the learning when designing new projects (Altizer et al., 2019; Davoodi et al., 2017; Preiser et al., 1988; Vischer, 2009).

During its development, which originated in the late 1960s, POE was used for various purposes, adopted not only by architecture and construction firms but different sector organizations interested in assessing designs (Altizer et al., 2019; Yang & Guangsi, 2020). Consequently, it has already been applied to many different purposes and typologies, mainly health facilities. The studies addressed different types of buildings and they discussed strategies to make buildings more sustainable (Brambilla & Capolongo, 2019; Woon et al., 2014), the burnout problem (Lupo et al., 2021), maintenance programming (Ornstein et al., 2009), and continuous improvement (Marmot et al., 2005; Vischer, 2009).

Joseph et al. (2014) linked alternative terms used to designate POE, such as facility performance evaluation, facility assessment, building audits, environmental design audits, practitioner-focused facility evaluation, as well as facility evaluation (Watkins & Keller, 2008), landscape performance series (Yang & Guangsi, 2020), and building performance evaluation (BPE; Gupta & Gregg, 2016; Preiser et al., 2018; Steinke et al., 2010; Yang & Guangsi, 2020).

In general, new nomenclatures answer questions about previous POE understanding. Some alternatives are proposed to overcome time and study aim limitations. Watkins and Keller (2008) preferred to use facility evaluation because they understood that POE implies evaluation after occupants have moved into a building, while new demands suggest systematic research before and after building occupancy.

Depending on when the building is evaluated, different observation characteristics should be considered, and specific methods should be adopted to obtain the evidence. PrOE is conceptualized to evaluate the results in the design before introducing space (Shin et al., 2017). However, as with POE, the PrOE application has limitations in user-oriented assessment, as it is based on existing similar buildings where the client differs from the real user (Shen et al., 2013; Shin et al., 2017; Whelton & Ballard, 2003).

Paraskevopoulou and Kamperi (2018) understood that preoccupancy research is a premeasure, while postoccupancy research is a postmeasure of the individual and unique object. During the 1980s, there was an increased demand to measure and extrapolate findings to compare and generalize assessment results. In Brazil, POE research maintained the same terminology, adding new attributions so that research could be identified as a POE study (Ornstein et al., 2009). On other research fronts, original POE approaches evolved into more performance-oriented research with the BPE (Brambilla & Capolongo, 2019; Li et al., 2018; Meir et al., 2009). One critique of the POE methods was (and, in part, still is) that the focus has been on user experience and perception rather than on predetermined performance criteria evidence (Elf et al., 2019; Vischer, 2009), which makes effective assimilation difficult in other research, design, and building procurement processes (Hadjri & Crozier, 2009; Joseph et al., 2014). In the earlier 2010s, Joseph et al. (2014) and Taylor (2011) mentioned that POE was an underused way (12.5%) to collect evidence about design strategies for a project.

Based on the findings, we understand that part of the criticisms leveled at the POE related to its contribution limits stems from the late application time. That is, the direct contribution to the evaluated design occurs too late when the building has already been constructed and occupied. Thus, learning is useful for applications in future designs, which reduces the interest of those responsible for the current enterprise to invest in POE. We also observed that the criticism of the difficulty of measuring and transposing the experience to other processes motivated the beginning of developing the PDE and impacted the discussion about the conditions to recognize the results of the POE as data sources for the EBD. As already noted, “PrOE” meets the demand for quick assessments, before occupation. However, as studies on PrOE were developed, they noted the demand to overcome the limitations of the conventional design evaluation, anticipating the evaluation before starting the construction, that is, during the design development (Shin et al., 2017). Part of the PrOE literature addresses simulation models for user activity evaluation and circulation, security, access control, burglary prevention, air quality, energy saving, space usage, and so on (Coleman et al., 2018; Göçer et al., 2016; Shin et al., 2017). These simulations aim to predict and evaluate the building performance or organization before freeing up resources and time to execute the project, reducing the risks of design problems usually identified only after occupancy (Shen et al., 2012), and address studies that deal with an assessment before and during the beginning of the project development, throughout the project briefing and programming.

According to Ornstein et al. (2009), PDE corresponds to an analysis of the design programming and the early architectural design, which is applied by experts in design performance evaluation. Unlike the EBD and POE bibliographies, the reports retrieved from PDE do not link a clear definition nor present enough information for a comparative discussion. Apparently, it is an open debate in the field of research. This perception is reinforced by the PrOE reports that actually deal with PDE initiatives. Emphatic recommendations prevail for changes to occur in the project life cycle to include means of assessment in the predesign phase (Ornstein, & Andrade, 2012).

The difficulty of evaluating design occurs even in contexts where the integration of the EBD to the BLC is more developed, as in the initiatives of contractual regulation by the United Kingdom National Health Service (NHS Estates & Department of Health, 2004). The literature highlights the persistence of the research/practice gap (Moslehian et al, 2021), and the design problems occurrence linked to private finance initiative in public–private partnerships (Hignett, & Lu, 2009; Symons, 2021). These occurrences are contractual issues but are also related to methodological difficulties in design evaluation.

The simulations tend to operate with greater imprecision or error possibilities when compared to POE or even PrOE results because there is a gap between the predicted performance and the real performance (Coleman et al., 2018). Conversely, even considering such limitations, PDE focuses on the just-in-time support of the design decision-making considering performance projections, while POE and PrOE occur late and require a great deal of time and cost to obtain limited results in the evaluated environment improvement because they start after completing the construction (Shin et al., 2017). Applying POE and PrOE results in design assumes that data based on similar existing buildings can be used as a basis for designing a new project. However, it must be considered that the original client may differ from the current user (Shen et al., 2013; Shin et al., 2017), which, added to the difficulty of mapping who the ‘real clients’ are, reduces the validity of some findings, especially in client-oriented assessments (Shin et al., 2017).

Another relevant aspect is that an instrument can be classified into PDE but can also be used for other evaluation types, such as POE or PrOE. This occurs because, depending on the time of evaluation, a design evaluation can be a retrospective diagnosis or predictive assessment (Shin et al., 2017). That is why some instruments developed for retrospective design diagnosis in PrOE can be used for predictive evaluation in PDE (Pereira et al., 2023). This depends on confirmation in studies designed to confirm the reliability of the simulation.

Discussing comprehensive programming, Watkins and Keller (2008) provide three topics that PDEs can address to resolve the theoretical differences between design practitioners and researchers: First, the explanation for “why a design solution does or does not work” applies to the actual design; second, comprehensive programming can balance normative with positive theory in project teams where designers and researchers work together; and, third, if data from multiple building projects or case studies are combined into one database, the researcher can make inferences beyond the parameters of one project. The third topic is particularly important to integrate PDE with EBD and increase the consistency of simulations with the ratification of multiple cases.

EBD—Project-Oriented Research and Evaluation Perspective

The term and the notion of using evidence-based research to support design decisions have been transposed from evidence-based medicine (Phiri & Chen, 2014a; Vischer, 2009). This practice in the medical field aims to reduce uncertainty and the decision-making burden that medical professionals experience (Vischer, 2009).

The concern to support and share responsibility for decisions appears in the definition proposed by Hamilton (Stichler & Hamilton, 2008, p. 3), whose quote is reproduced by several authors: “Evidence-based design is a process for the conscientious, explicit, and judicious use of current best evidence from research and practice in making critical decisions, together with an informed client, about the design of each individual and unique project” (Davoodi et al., 2017; Lupo et al., 2021; Paraskevopoulou & Kamperi, 2018).

Furthermore, some EBD definitions emphasize that this “current best” corresponds to robust evidence or data from already built environments and credible collected user experiences, arising from an analysis based on recognized methods and rigorous studies, to influence both the design process and design results concerning the effects of the physical environment on well-being and their critical interpretation (Davoodi et al., 2021; Elf et al., 2019; Phiri & Chen, 2014a; Stichler & Hamilton, 2008; Vischer, 2009).

The EBD approach requires understanding what evidence is in this case. Synthetically, evidence is a cause-and-effect relationship between design and outcomes affecting users (Watkins & Keller, 2008). Different from the traditional practice based on anecdotal evidence, normative, and best practice alone, the EBD process focuses on the robust collection of evidence and validation through a rigorous process (Joseph et al., 2014; Phiri & Chen, 2014b). Patience (Vischer, 2009) considered that evidence can be strong when it is based on independently verified data, and that weaker evidence is also admissible, that is, when this evidence is based on weaker data and the informed opinion of authorities on the subject (experts), demonstrated based on available data (Vischer, 2009). The second case of evidence (weaker) responds to a conditional relationship in some EBD definitions, in which they emphasize that it is a process based on the best available research evidence (Watkins & Keller, 2008) to achieve the best outcomes possible (Altizer et al., 2019).

While based on research results, EBD is less of a product and more of a process, and most importantly, although it involves a research process to investigate the best available evidence, EBD is still a design process (Davoodi et al., 2017; Joseph & Hamilton, 2008); that is, a social and creative process. Thus, it is not recommended to assume that the best available evidence is a fixed and static guideline to support design decisions (Davoodi et al., 2017; Stichler & Hamilton, 2008).

EBD can be understood as a part of the design process or a parallel process. Objectively, EBD involves using research to make design decisions and then evaluating the impacts of those decisions (Joseph et al., 2014). As a method, EBD cannot replace current design methods, but rather, it is understood as a complement to these methods (Davoodi et al., 2017). In this condition, design professionals analyze the evidence, interpret it in the unique context of the project, and propose hypothesis-based design interventions to obtain the expected results. They then measure the impact of the proposed project and report the findings to contribute to the evidence base (Joseph & Hamilton, 2008).

An informed client offers support to EBD strategies and contribution to making decisions based on the best information available, obtained from research and project evaluations. Usually, research on EBD involves improving results, efficiency, and effectiveness services; it increases patient, family, and staff environment perception and satisfaction; implementation of the best current practices and provides flexibility to adapt to the future (Phiri & Chen, 2014a). Information sharing helps meaningful and effective collaboration between the designer, client, and users (Davoodi et al., 2017) and puts patients, staff, and organizational outcomes at the center of all design decisions throughout the design process (Joseph et al., 2014).

The social and strategic dimension contributes to EBD considered the most widespread theory or approach available to describe and promote quality improvement in the design process, especially in health architecture, and it is recognized as a promising basis for future developments in architecture in general (Anåker et al., 2017; Brambilla et al., 2019). Unlike PBD, the EBD does not rely heavily on numbers and engineering quantities (Davoodi et al., 2017; Shi, 2010), as it combines qualitative and quantitative assessments, increasing flexibility to define methods (Davoodi et al., 2017; Lavy et al., 2015; Pati, 2011). This feature is suitable for practice-based environmental design research, which presents the dual purpose of creating knowledge and solving a specific design problem (Geboy & Beth Keller, 2018; Watkins & Keller, 2008).

A Framework Analysis

The framework proposals identified in the analyzed reports relating to EBD, POE, and PDE can be categorized into types: (1) conceptual framework processes in general (Brambilla et al., 2019; Joseph & Hamilton, 2008), (2) frameworks that related a POE or PDE to EBD (Davoodi et al., 2017; Steinke et al., 2010), and (3) frameworks based on a specific discipline or application (Lygum et al., 2018; Refshauge et al., 2015; Sidenius et al., 2017). This categorization is similar to that proposed by Davoudi et al. (2017).

A widespread framework of EBD relates the logical sequence of EBD steps in a cyclic process. This structure is cited in reports such as Joseph et al. (2014), Davoodi et al. (2017), and Davoodi et al (2020). Davoodi et al. (2020) present a theoretical model of EBD-SIM (EBD simulate) implementation and integrate it into this EBD framework. The application of the framework in this study shows evidence of its validity for structuring design-oriented research while revealing an inconsistency: The framework directly relates the EBD stages to the BLC, which encompasses the predesign, design, construction, occupancy, and facility management stages. However, the workflow indicated in the study characterizes it as a POE. That is, the presented EBD cycle is not equivalent to the BLC stages, but only one, referring to the occupancy. Other studies also successfully performed the complete EBD cycle and occurred at a specific stage of the BLC (Joseph et al., 2014; Shin et al., 2017). Some cases involve a multiphase approach, covering programming and design, construction, POE, and others (Battisto & Franqui, 2013). This approach can be understood as an advantage, but this is not a condition for conceiving an EBD. Thus, based on the findings, we argue that it is not convenient to directly relate the EBD steps and BLC stages. Moreover, as a result, we question if it is feasible to carry out a complete EBD cycle at different BLC stages.

Davoodi et al (2020) developed their study during the POE to correlate a building performance simulation based on current visual comfort metrics and real human perception obtained with instantaneous and annual visual comfort perception questionnaires.

The authors pointed out the importance of performing POEs to collect real data, thus improving the adjustment of the metrics adopted in the simulation they developed in EBD-SIM. However, after achieving a high correlation between the simulation and perceived visual comfort, the simulation can be used for designs in which the result obtained from the simulation shows such realistic indicators that a decision will meet the desired expectations. In this case, “finding relevant sources of evidence” corresponds to the input of information from the simulation that can occur for different purposes. For example, Shin et al. (2017) proposed a program that indicated which burglary-vulnerable spots there were in the target space by predicting burglars’ behavior. It is a project assessment that involves behavior and does not depend on construction and occupancy to be put into practice. Other types of simulation aim to verify the quality of user and resource flow, equipment use, airflow, lighting, internal temperature, and so on.

Coleman et al. (2018) addressed the necessary development of performance evaluation systems that consider ways to predict and measure building performance. The authors suggest a four-quadrant scheme delimited by a horizontal axis that characterizes the performance evaluation types and a vertical axis that indicates the focus type of evaluation. The horizontal axis locates methods for evaluating an object in time (predicted performance ← evaluated object → measured performance), while the vertical axis relates the evaluation to the environment and occupants, that is, the social and psychological dimension of the environment (environmental systems ← relationship → human systems; Coleman et al., 2018).

In the SRL findings, some PrOE reports indicate working with forecast gaps and expectation gaps. This occurs because PrOE meets assessment demands made before the environment occupation. Considering this reference topic, PrOE methods, which were initially designed to evaluate operations, evolve into design evaluations, when the learning opportunities can contribute to the current case (Shin et al., 2017). However, if we maintain this benchmark, the forecast gap will widen and will change the concerns, reliability, and purpose of the application in EBD.

Thus, if considering that the “evaluated object” is a new environment, the “predicted performance” of a PDE is defined by the POE recommendations of similar cases, and the measured performance needs to be simulated. Redefining that the reference point changes from “actual occupation” to “projected occupation,” clarifies the limits of action of each assessment instrument and guides the definition of what strong evidence is by using the most suitable one in the different BLC stages.

Synthesis of the Frameworks and Examples

Based on the presented SLR, we suggest an alternative framework (Figure 3), consisting of programming, design, and evaluation cycles for innovation. The initially cited EBD cycle steps were listed in the right column, just as a description of EBD steps. The middle column presents a conceptual EBD cycle. The left column distinguishes available resources and the EBD process, showing evidence of existing conditions and the research/design process.

OPEN IN VIEWER

Figure 3.

Proposal of evidence-based design steps cycle. Source: The authors’ proposal, based on Joseph et al. (2014), Lygum et al. (2018), and Sidenius et al. (2017).

This framework can be used in “objects” of different BLC contexts and times, as illustrated by three examples that demonstrate the adequacy of use in identified studies:

Example 1: (1) An organization that wants to know if the available space and layout are suitable for the new occupancy conditions, (2) this organization that seeks reports, specialists, methods, and criteria available to start the (3) projection of possible scenarios and (4) tests them; finally, (5) that publishes the findings and deposits the data in a database, and (6), depending on the result, requests a new EBD schedule (this time to start the project or another measure; based on Riratanaphong, 2021).

Example 2: (1) A design team wants to decide and justify to the clients what the best spatial arrangement for movement and use is, or what the best building envelope composition is to prevent burglary, (2) the team researches what the appropriate performance simulation criteria are to evaluate what they want to verify (3) they design possible spatial arranges or facade composition based on existing reports about similar cases and necessary characteristics to simulate and (4) test the alternative solutions; finally, (5) depending on the result, they publish the findings and deposit data in a database, and (6), define the baseline performance measures to request a new EBD programming (based on Major et al., 2019; Shin et al., 2017).

Example 3: (1) A research and healthcare POE team wants to understand the impact of healing gardens on patients in a specific healthcare facility, (2) the team members search available reports of similar evaluation methods and criteria, as well as design, recommendations already evaluated in real use conditions; then (3) they define a hypothesis and adjust the selected perception capture instruments to this specific condition, and (4) test the hypothesis with the collaboration of patients, defining comparative guidelines circumscribed for the case; finally, the findings need (5) to be published and the data deposited in a database, and, (6) depending on the outcome, the responsible organization recommends staring a new preproject evaluation EBD schedule (based on Lygum et al., 2018; Paraskevopoulou & Kamperi, 2018; Valente & Marcus, 2015).

Note that POE is necessarily linked to the BLC occupancy stage, and therefore, that POE is not part of EBD but contributes to it with a database and much more: The intersection between EBD and POE procedures for the collection and analysis of evidence depends on the adequacy of the POE instruments, as well as instruments borrowed from other disciplines to the conditions of evidence admitted by the EBD.

The proposed framework eliminates the overlap between the EBD and BLC processes. Instead, an auxiliary framework is presented to retrieve the relationship between methods and application times and the characteristic of the data involved (Figure 4).

OPEN IN VIEWER

Figure 4.

Evidence-based design method mapping. BPE = building performance evaluation; PBD = performance-based design; PDE = predesign evaluation; POE = postoccupancy evaluation; PrOE = preoccupancy evaluation. Interpreting “EBD method mapping” framework: Dots indicate generic instruments. Each point is located on the chart according to its properties. Thus, “dot N” is an instrument oriented to the predominantly human system (one-domain), more qualitative, and is a predicted performance. For example, in an opinion interview without a consolidated occupation. The “dot I” indicates the user’s perception interviews’ of the occupied environment (multidomain) is qualitative and is a measured performance. For example, the degree of thermal comfort according to users. It is qualitative, and it is a measured performance. N.B. = In a future deployment, a weight will be assigned to the properties, enabling us to measure the location of each evaluation procedure and instrument in the framework. Source: Authors’ proposal, based on Coleman et al. (2018).

We change Coleman’s approach, redefining the horizontal quadrant, in which the more proximity of the central vertex indicates methods of greater proximity to the object experience. Moreover, in the opposite direction, greater proximity to the limit of the circle indicates methods that depend more on the accuracy of the means of prediction and measurement and, therefore, require more attention in the instrumentation and quantification of quality metrics. Vertically, the location of a method indicates a gradation between a focus on human systems (in a narrow sense, psychological and sociocultural) and on environmental systems (in a narrow sense, material and physical). In this case, greater proximity to the central vertex means that the method includes more domains of the environmental relationship, for example, thermal comfort involves physical characteristics and user perception.