Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Introduction

Opportunities for Adaptive Architectural Systems

In the context of sustainable building design, novel material innovations are shifting the performance capabilities of building envelopes towards adaptive systems that can respond to the changing energy demands of buildings while addressing to occupant preferences for comfort and control (Krietemeyer et al. 2015). Adaptive architectural systems include building skins or surfaces that can mechanically, electrically, or chemically alter their state to adapt to changing external or internal stimuli, such as outside temperature, sunlight, or building inhabitants (Schnädelbach 2010). In contrast to fixed all-glass building facades, where uncontrollable solar gains and little consideration for occupant control were the result of architectural ideologies of the twenty-first century, emerging glass building facade technologies maintain a different focus. Smart films and shading devices are incorporated into glazed facade systems to combat incoming solar energy (Baetens et al. 2010), and adaptive facade systems are increasingly focused on user control for better privacy, thermal comfort, views, and visual effects (Loonen et al. 2013) (Fig. 11.1).

Fig. 11.1
figure 1

Examples of building facade systems illustrating the trajectory towards adaptive architecture and increased degrees of user control over the building facade’s appearance and behaviors

Full size image

The degree to which user control has been integrated into building envelopes has changed over time. While the early 1900s made use of Venetian blinds for solar shading and privacy, this common shading device allowed for a range of visual variation with which occupants could individually control the amount of diffused light and views at windows. In contrast to this variation, all glass curtain wall systems of the mid-twentieth century no longer controlled incoming solar radiation at the building facade; instead, mechanical cooling systems were used to maintain a consistent level of indoor thermal comfort despite changes in weather or occupant preferences. As a result, minimal control was available to building inhabitants for modifying views, daylight, or glare within the perimeter glazing areas of the building. Conventional building systems created a homogeneous indoor environmental standard for visual comfort, a sociocultural construct of modernity that has in turn led to occupant dissatisfaction and overall decrease in well-being (Shove 2003). With the energy crisis of the 1970s, the excessive use of mechanical building systems generated a greater awareness of energy use, thus spurring the development of glazing technologies and facade shading devices to combat unwanted solar gains. Glazing technologies such as tinted or Low-E glass are aimed at mitigating solar energy and were engineered to block heat gain and reduce glare. However these glazing technologies do not necessarily solve issues associated with the lack of individual control since they are typically fixed tinted glazing systems applied around the entire building. They still face limitations with visual discomfort. As new materials have been integrated into building facade systems, the performance criteria driving their design have expanded to include both solar control and increased user control over the facade’s appearance and behaviors.

Numerous contemporary design research projects and installations demonstrate ways in which intelligent materials and building technologies could alter the relationship between the user, building system, and interior and exterior space. Interdisciplinary research groups are investigating responsive architectural materials and environments along with ways in which building technologies can incorporate a range of inputs into their dynamic response. The Material Dynamics Lab at the New Jersey Institute of Technology experiments with the integration of electro- and thermo-responsive smart materials for systems like the Homeostatic facade that can adapt to their local environment (Decker 2013). Similarly, collaborators in interactive and responsive design at the Swiss Federal Institute of Technology (ETH) in Zurich explore organic kinetics in architectural applications using electroluminescent screens, electroactive polymers, and flexible audio panels to generate emotive and responsive environments (Kretzer et al. 2013) The Sabin Design Lab at Cornell explores the integration of passive materials, sensors, and imagers into responsive building eSkins (Sabin 2015). The Center for Architecture Science and Ecology (CASE) at Rensselaer Polytechnic Institute is developing Electroactive Dynamic Display Systems (EDDS) to address energy performance goals of building facades while simultaneously allowing for a range of information patterning and user control (Dyson et al. 2013). Each design research group investigates various material prototypes for high-performance building envelopes that can respond to a range of stimuli. The ability to scale up these physical prototypes and integrate their behaviors with other building sensing and control systems is a critical step in determining their feasibility and overall performance relative to balancing quantitative and qualitative criteria. Systems like the EDDS offer many opportunities for user engagement and control over the environmental, visual patterning, and spatial effects. Because of the multivariate parameters, it is necessary to digitally simulate the numerous possibilities to understand the energy performance impacts before investing in the physical prototyping phase of research.

Until recently, there haven’t been building envelope materials and technologies that have provoked the engagement of occupants to the degree that they offer now. For the EDDS and many of the featured projects and research groups, emerging material breakthroughs are redefining the meaning of performance for building envelope technologies, transforming their role as static and sealed enclosures to fluctuating membranes mediating energy and information. Novel building envelope systems being explored by architectural designers are becoming increasingly legitimized in the building science community by their ability to address quantitative energy performance benchmarks. At the same time they are challenging traditional architectural notions of boundary and space, physics and energy, experience and perception, and author and interpreter. The remarkable material malleability and responsiveness of new systems will transform buildings from fixed enclosures to flexible interfaces that effortlessly capitalize on local environmental flows while inviting a participatory dialogue with the people who reside in their presence. It is critical that the architectural design, simulation, and prototyping methods are able to adequately consider occupant interaction with responsive building skins. Furthermore, occupant interaction should be understood as integral to methods for predicting the energy performance of adaptive architectural systems.

Computational Design Tools for Adaptive Architectural Systems

There are several different methods for the design, prototyping, and simulation of novel building envelope systems. In the case of systems like the EDDS and others mentioned above, physical prototypes are a necessary step in the research process for measuring quantitative performance metrics such as electromagnetic response, cycles of durability, and fabrication feasibility. Yet relying solely on lab-scale prototypes during the design and development phase risks overlooking valuable qualitative characteristics that could more effectively be examined at various scales, such as visual comfort, perception, interactivity, and control. Since physical prototyping can be costly, time consuming, and limited in scale, computational simulations are often used in the design process to visualize the architectural or daylighting effects and to measure the predicted energy performance of these adaptive systems. Simulations also provide exciting opportunities to visualize and test the interactive potential of adaptive systems.

3d computational modeling, simulation, and energy analysis tools typically utilize a linear workflow in which a design option for a building facade is modeled in one software for visualization and then imported in a separate program for analysis. The designer must manually manipulate the building geometries and parameters, export the fixed model, and then analyze the design separately in simulation software to test for building energy impacts (Lagios et al. 2010). The disconnected workflow makes it challenging to test various configurations of adaptive systems quickly and according to both external and internal stimuli.

One recent approach that utilizes the parametric modeling tool DIVA for Rhino builds on the linear method of exporting a model for energy analysis through direct links to EnergyPlus and Radiance for a seamless daylighting simulation workflow. This method allows the rapid visualization of daylight and energy impacts from an architectural design model where users can easily test multiple design variants for daylight and energy performance without manually exporting to multiple softwares (Jakubiec and Reinhart 2011). While this simulation workflow speeds up the daylighting analysis process and integrates occupant comfort models to determine the status of shading systems such as venetian blinds, it lacks real-time capabilities for analyzing the impacts of more complex adaptive facade systems according to both internal and external stimuli. This real-time analysis is essential for understanding how adaptive architectural systems respond simultaneously to fluctuating environmental flows and variable occupant preferences, which can often pose conflicts with regard to desires for views, privacy, daylight, and the need to mitigate solar heat gain.

Another method to a building energy simulation for adaptive facades aims to quantify their long-term impact on building performance using genetic algorithms for multi-objective optimization. This method supports the need for simulation tools that analyze the energy impacts of adaptive conditions on a long-term basis and allows for visualization of trade-offs between two or more conflicting design objectives. It argues that seasonal facade adaptation is a more practical and reliable approach than facades that change on a higher-frequency basis (Kasinalis et al. 2014). The approach fills gaps in the field of dynamic simulation frameworks through the integration of multi-objective algorithms; however it does not yet support exploration of adaptive systems that could respond immediately and simultaneously to a range of occupant comfort needs, instead privileging longer-term external response.

Existing energy simulation frameworks remain somewhat limited to basic predefined inputs and do not always accommodate analysis at various spatial or temporal scales. Further, they do not include real-time visualization and spontaneous interaction with the inhabitants as factors to the energy analysis. Standard building simulation tools are lack dynamic, geometric and material complexity, and are unable to incorporate realistic occupant behavioral models. These limitations lead to evaluation methods that treat external environmental response and internal occupant response as separate performance goals (Fabi et al. 2011).

Immersive virtual reality (VR) environments offer alternative methods for visualizing adaptive architectural systems and for incorporating human behavior models, or real-time user interaction, for experimental testing. One example is a cave automatic virtual reality environment (CAVE), where flat panel displays or projections are directed on multiple interior surfaces of a room-sized cube. A CAVE provides true-stereo 3D and can be used to visualize large datasets of information in a 3d interactive and immersive way. CAVE systems support groups of users in a high resolution 3d shared immersive setting, but they are expensive and require a substantial amount of physical space, supporting infrastructure and hardware. Smaller VR visualization devices such as the head-mounted display (HMD) create a similar VR experience that is less expensive and more mobile than a CAVE. HMD devices allow stereo viewing through small monitors mounted in front of each eye and head tracking hardware for 3D immersion.

Various scales of CAVE and VR HMD technologies are becoming increasingly popular visualization tools for the architectural profession (Kim et al. 2013). Although most of the research related to immersive simulation has been conducted in fields other than architecture, it can have a direct parallel and can be used to advance the work in immersive building simulation. Potential applications include the post-processing of Computational Fluid Dynamics (CFD) Data, building and data representation, building performance visualization, and immersive visualization for structural analysis (Malkawi 2003). An early example of a fully immersive CFD visualization enabled users to visualize various building thermal analysis data using a CAVE. Users could change the space parameters such as window size or materials and visualize the resulting thermal conditions. This study was one of the first aimed at building a system that allows a user to perceive different environmental factors in a three-dimensional space (Malkawi and Choudhary 1999). Various combinations of VR environments for architectural applications have been explored over the last decade, such as wearable systems for the design process, mixed reality systems for archiving historical building information, augmented reality systems on construction sites, and integration of mixed reality in education and design studios (Wang and Schnabel 2009). Architectural researchers and practitioners continue to explore opportunities for evaluating designs, improving 3D models, facilitating remote collaborative design, and studying human preferences in virtual environments that represent real-world settings. One application of a VR HMD for studying human preferences in architectural applications creates an Immersive Virtual Environment (IVE) to understand the relationship between human comfort, daylighting, and lighting controls in an interior space. The IVE provides flexibility in creating environments with different control settings and in evaluating end users’ behavior and preferences given different design and operation scenarios (Heydarian et al. 2015). Similar to the aims of the research presented in this chapter, the IVE design process seeks to ensure that architectural proposals not only meet the end-users’ preferences but also encourage more energy efficient behaviors.

With the integration of new adaptive material technologies and virtual reality systems into architectural design, questions of design authorship and agency are raised: what types of information patterning will be expressed on and within buildings, and who will curate this information? How can a building envelope system move beyond an automatic response to external forces and instead engage in an interactive dialogue between external and internal stimuli—between itself, energy and people? The interactive simulation environment presented in this chapter combines new material technologies, hybrid reality visualization systems, and energy simulation software into a design tool for architects and end-users to experience the many performance possibilities of adaptive systems.

Objectives

Visualizing the energy performance of adaptive architectural facade systems is important for understanding their architectural effects and energy performance. However, current methods for visualization and analysis of the relationship between energy flows, building demands, and occupant control remain limited because commercial software tools and virtual reality environments are not yet integrated into a seamless feedback loop. In order to keep pace with rapidly advancing research towards responsive building envelope technologies on multiple fronts, new design tools are needed to address the multiscalar complexity and socio-cultural performance possibilities inherent within emerging material behaviors. For designers integrating next-generation adaptive architectural systems into buildings, interactive simulation environments are necessary to anticipate the fundamentally new environmental, social, and spatial implications of their dynamic and responsive potential. This is particularly important in response to inevitable conflicts between user control, aesthetic desires, and environmental performance criteria.

The following sections of this chapter present the development of an interactive visualization and simulation tool that combines real-time energy analysis visualizations with hybrid reality techniques to support user interaction with adaptive architectural skins and systems early in the design process. Computational algorithms and virtual reality visualization tools are integrated into a simulation environment for real-time interaction and analysis of adaptive architectural systems and their impacts on energy performance. Using the EDDS as a facade testbed system, the goal is to utilize the interactive simulation environment as a design tool that informs the physical prototyping of novel architectural facade systems. Developing computational simulation tools to support new facade material opportunities such as the EDDS is a critical step concurrent to ongoing physical prototype developments.

The challenges that this approach begins to address are threefold: first is the ability to design adaptive facade systems according to unpredictable environmental and human inputs simultaneously; second is the ability to integrate human perception and behaviors into the evaluation and decision-making process based on the various degrees of observation and interaction that can be experienced at full-scale; third is the ability to visualize and experience the architectural effects and dynamic potential of emerging material systems like the EDDS that aren’t physically scalable at this point in time, particularly in generating synergistic relationships between the human desires and environmental response. Critically, this research is producing a design decision-making tool that both measures and visualizes dynamic architectural conditions while receiving real-time energy feedback based on users’ engagement. In the process it establishes exciting opportunities for the fields of architecture, materials science and engineering, and human-computer-interaction to inform each other in multidimensional ways.

Methodology

Constructing Hybrid-Reality Simulations for Interactive Design

The setup for the simulation environment uses multiple digital projectors, sensors, large flexible screens, VR displays, and customized algorithms for interactive design. This approach uniquely utilizes a combination of digital projection and VR display technology as a hybrid method for experiencing and interacting with the full-scale effects of dynamic facade systems like the EDDS. The approach is considered hybrid since it combines a large-screen semi-immersive projection environment with a fully immersive VR environment using a head-mounted display (HMD) (Fig. 11.2).

Fig. 11.2
figure 2

Diagram illustrating the hybrid-reality simulation setup at the Interactive Design and Visualization Lab at Syracuse University. Users can interact with the large-screen projection (top right), or use the VR HMD to view architectural design proposals (bottom right)

Full size image

The purpose of the large-screen projection is to create a full-scale visualization of an adaptive facade system where multiple users can experience and modify its behaviors. The physical setup supports the visualization and interaction with the facade’s dynamic patterning, changing views to the exterior, and ambient daylight and shadows within the space. Similar to a CAVE, the large-screen projection environment uses digital projectors and sensors for position and perspective tracking. Two projectors are used to simulate the facade and its daylighting effects: one rear-projection throws an image of the simulated facade onto a large flexible screen, and a second ceiling-mounted projector throws an image of the daylighting and shadow effects onto the floor. Kinect motion and infrared sensors located in the corner of the screen and connected to a desktop computer track the physical positions and gestures of users as they interact with the dynamic facade systems’ behavior. Projectors and sensors are wired to one desktop computer. This type of environment provides a collaborative design space with real-time visual and analytical feedback unlike standard 3d architectural modeling tools. The setup is more adaptable and cost-effective than a standard CAVE and can be installed in most spaces using one or two projectors without the need for specially designed rooms and extensive infrastructural support. The flexible fabric projection screen stretches across large room widths and heights without the spatial restrictions of typical CAVE systems, and can be adapted to different architectural offices or studios for designers and clients to visualize architectural proposals.

The purpose of incorporating the VR HMDs is to create additional flexibility and full immersion for interactive, multi-user design at a range of architectural scales. The integration of HMD devices like the Oculus Rift, combined with motion sensors and the gaming engine Unity3d, offers a number of exciting possibilities for the design process. First, a user can visualize, meander, and interact with a dynamic building system or architectural space in a completely immersive 3d visual environment without concerns for the physical lighting or spatial requirements. Depending on the extent of the modeled environment, the boundaries are essentially limitless, whereby one can explore multiple scales of architecture within the virtual environment. Second, with a state-of-the-art combination of VR HMDs and motion sensors, an interactive design concept developed by collaborators Noirflux (2015), users wearing the Oculus Rift can physically walk around while viewing their virtual environment through the headset. This physical movement reduces the effects of simulator sickness, which is caused by the visually-induced perception of self-motion when the body isn’t actually moving. Third, the large-screen projection can display the view from the Oculus Rift, or display supplemental environmental information that can be accessed by a group simultaneously. Further possibilities for collaborative, remote architectural and urban simulation are discussed later in the chapter in ongoing work.

The simulation software uses VVVV, a live-programming environment for quick prototyping and development. VVVV is designed to integrate large datasets and media environments with physical interfaces and real-time motion graphics, and audio and video that can interact with many users simultaneously (VVVV 2015). In our interactive design simulation, VVVV provides an immersive visualization platform and graphical user interface (GUI) for 3d architectural modeling software tools such as Rhinoceros and Grasshopper. By importing 3d geometric data into VVVV, architectural designs and their energy performance analyses can be viewed and experienced in a dynamic way either through web/App-based user interfaces or through VR HMDs (i.e. Oculus Rift, Google Cardboard, etc.). Users can visualize and interact with a simulated architectural space or adaptive facade system and experience both exterior and interior conditions for any 3d geometry at multiple scales. Alternative dynamic facade materials, geometries, and building designs can be imported and viewed interactively, which is enormously beneficial for architects testing different design proposals in various climate and site scenarios.

Gestural Interactions for Controlling Facade System Behaviors

Users can interact with the simulation in one of three ways: the first is through a custom graphical user interface (GUI), which is accessible through a monitor, the second is through gestural interaction with the large-screen, and the third is through gestural interactions with the Oculus Rift. The GUI provides access to modify the parameters of the simulation, such as the geographic location, solar position, material composition, and library of facade patterns and user interactions. It also provides access to a user’s point cloud position data, which is recorded for data collection on user’s interactions. Both the Oculus Rift and the large screen and motion sensors allow users to interact with the simulation through position and perspective tracking, as well as through gestures that change the pattern or visual effects of the facade’s behaviors.

The facade’s dynamic behaviors include opening locally for viewing portals, closing for personalized privacy screens, and morphing into customized pixilated patterns or animated videos across the facade. The motion sensors and customized algorithms identify a user’s presence by creating a point cloud, and then locate an individual’s head, hands, feet, and body for gestural interaction. Users can swipe their hands and arms left to right or top to bottom to change the appearance of the facade, or they can use both hands simultaneously to switch the pattern, portal, image, or animated effect they wish to see on the facade. Personalized images or videos can be ‘uploaded’ to the facade as pixilated versions, creating dynamic shadow effects on the interior, and individualized expressions along the exterior of the facade. Combined, the simulation environment creates a full-scale interactive visualization of an adaptive building facade system and its perceived effects on views to the exterior as well as daylighting and thermal conditions (Fig. 11.3).

Fig. 11.3
figure 3

Multiple users can simultaneously interact with the dynamic facade simulation (top row). Point cloud data allows designers to document and record positions and gestural interactions with a dynamic systems’ behavior (bottom row)

Full size image

Point cloud data viewed through the GUI on the monitor anonymously records gestural interactions in order to analyze the tendencies and degrees to which users modify and adapt to a dynamic systems’ behavior (Fig. 11.3). This data is currently used in several ways: one is to observe how quickly users adapt to the gestural control settings. This allows us to identify which gestural interactions are most intuitive. Another is to examine how adaptive building envelope systems negotiate potential conflicts between groups of users (i.e. how to program the facade to adapt to different user gestures within the same area). Lastly it is to program and test how a facade adapts to users’ control preferences while still meeting energy performance goals for reducing unwanted solar heat gain.

Moving beyond typical architectural modeling and analysis tools, users of our hybrid reality simulations have the ability to interact with and modify adaptive building skins while receiving measured feedback as to their predicted energy and daylighting performance. There are multiple ways to receive energy performance feedback. The first is by viewing performance data related to the glazing assembly’s ability to mediate solar heat gain and daylight. Users can hold up their arms to trigger a pop-up data panel that displays real-time measured energy performance feedback of the glazing at that specific frame rate (Fig. 11.4, left). Numeric values representing visible transmittance (Tvis), U-value, and solar heat gain coefficient (SHGC) are calculated for the building envelope assembly and visualized in real-time through dynamic charts that continuously update as the patterns shift based on solar position, pattern changes, and privacy or viewing portals. Another method for real-time energy performance feedback includes a full-scale daylighting analysis map that is displayed on the floor of the simulation environment (Fig. 11.4, right). This allows the user to interact with the facade and be semi-immersed in a dynamic pseudo-color analysis showing illuminance levels. For the first time, users of the simulation environment—especially architects and engineers—get an interactive and immersive experience of performance data that is typically only viewed as graphs, image stills or an animation through a computer monitor. Instead, the real-time full-scale daylighting analysis creates a stronger and more intuitive connection between the design and data analysis workflow, simultaneously folding in user input directly into the process.

Fig. 11.4
figure 4

A pop-up data panel displays real-time performance values for the adaptive facade system (left). The large-screen simulation environment displays a full-scale analysis map on the floor in colors representing daylighting or heat gain (right)

Full size image

Experimentation with User Interactions and Energy Performance

Initial experiments with participants examined the ability of the dynamic building envelope to negotiate its response for both solar tracking and user preferences for certain patterning effects or views. Using the EDDS as a facade testbed, these studies tested the ability of the simulations to allow for the design of system behaviors that matched glazing energy performance goals without compromising the dynamic visual effects designed by individual users (Krietemeyer et al. 2015). In the process, individual participant designs overlapped with others’ preferences for viewing portals, privacy screens, or sunshades, which materialized or disappeared based on one’s proximity to the simulated facade. When environmentally-responsive patterns were combined with participant interactions, an unanticipated series of optical effects, or biomorphic expressions, emerged at the intersection of human desires, material behaviors, and energy flows (Fig. 11.5). The interactive simulation as an open platform for participation and observation demonstrated how moments of collective ideation and design enabled participants to extend individual knowledge and contribute to a spatial assemblage that produced unexpected outcomes through localized inputs. As a result, the blended outcome of multiple participant designers satisfied a range of performance demands, both in terms of environmental performance and aesthetic effects. Participant feedback of designs further demonstrated a collective preference for hybrid visual effects that allowed for interrupted interactivity, regardless of the final blended appearance (Krietemeyer et al. 2015).

Fig. 11.5
figure 5

When environmentally-responsive patterns (top) are combined with participant interactions, an unanticipated series of optical effects emerge (bottom) at the intersection of human desires, material behaviors, and energy flows

Full size image

Another series of experiments investigated the impacts that multiple users interacting with the same dynamic facade had on a building’s energy consumption. Algorithms and customized scripts were developed to link the interactive simulations to the energy simulation software EnergyPlus, which is an open source building energy modeler available through the U.S. Department of Energy to calculate a building’s energy consumption. The goal was to understand the environmental impacts that multi-user interaction with the facade had at the scale of an entire building. The EDDS dynamic facade was again used as a material testbed for the interactive energy performance simulations. First the EDDS was programmed to respond to changing solar positions to provide adequate shading to maintain a certain level of daylight and heat gain on the interior. Next, the EDDS was programmed to adapt or ‘compensate’ its surface patterning in order to respond to users’ desires for views or other visual effects while still maintaining the required solar control or daylighting levels on the interior. For example, the EDDS adjusted its pattern density as users engaged or ‘interrupted’ the default solar tracking state of the system. The resulting pattern configurations were then translated to glazing information that was integrated with the EnergyPlus software to measure the impacts of user interactions on the heating, cooling, and lighting loads of a whole building (Krietemeyer and Rogler 2015) (Fig. 11.6).

Fig. 11.6
figure 6

Methods for real-time feedback incorporate full-scale daylighting analysis maps into the simulation environment (top row), which are then linked to a whole-building modeler to measure energy consumption (bottom row)

Full size image

Results were measured as values for daylighting and heat gain and were visualized as an analysis map on the floor of the full-screen simulation environment. An optimized facade baseline pattern was programmed to block out direct sunlight to reduce solar heat gains within the interior space. Then facade adaptability was introduced through the option of user-controlled viewing portals, whereby motion sensors tracked users throughout the simulation space and the facade simulation opened or closed based on proximity to the screen. This user interaction caused a deviation from the optimal baseline pattern and an increase in daylighting and heat gain levels. In order to adapt to both the users’ positions and to the performance goals for controlling heat gain, the facade’s response was programmed to redistribute its pattern so that viewing portals were provided but the facade still blocked out the necessary percentage of incoming solar radiation. The results of one interior space with the adaptive facade were then multiplied and simulated within a larger building model to measure the effects on an entire building’s energy consumption.

The computational workflow between interactive simulations and building energy performance software examined how adaptive facade systems can reduce a building’s energy consumption while simultaneously responding to occupant interactions and overrides. Preliminary analysis results demonstrated that systemic compensation for occupant interaction with the EDDS had positive impacts on the daylighting and thermal performance of a building (Krietemeyer and Rogler 2015). They also demonstrated the ability of the interactive simulations to visually scale up a dynamic building skin system, to experience and measure its daylighting performance, and to simulate its ability to compensate for multi-user interactions in order to meet goals for both occupant desires and environmental response.

In sum, current experiments combining hybrid-reality simulations and energy analysis software examine the ability of an adaptive architectural facade system (the EDDS) to negotiate potential conflicts between external and internal demands. In order to understand the implications of this methodology and its implementation in related design fields, it is important to discuss the benefits and challenges of hybrid reality simulations and ongoing work in adaptive architecture.

Discussion

Benefits and Challenges with the Hybrid-Reality Simulations

Hybrid-reality interactive simulation methods provide support to the research and development process on several levels. Simulations are critical for understanding the impact on energy and information performance from user interaction and behavior patterns, as well as on overall system performance. Dynamic decision-making design tools and shared visualization spaces are crucial for the growing field of adaptive and sustainable architecture where visual real-time communication is the primary tool for collaborating across disciplines and with clients. By constructing immersive visualization environments that simulate the responsive behavior of intelligent materials at full scale for multi-user interaction, the feedback and analysis can inform the physical prototyping process with valuable user input early on. This significantly reduces risks associated with physical prototyping new material technologies in the research and development phases while allowing for an iterative co-design process to occur between material and computational experimentation.

There are exciting areas for ongoing work that aim to address current challenges with the hybrid-reality simulations, namely those that focus on structured human factors empirical studies with the interactive simulations and physical prototypes, accurate calibration of the computational simulations with physical material prototype performance, and advancement of algorithms for more precise energy analysis at the system and building scales. Computational work is important in the development of algorithms for an entire building management control system, which will streamline communication between different spaces and types of building systems to maintain optimal energy performance.

The methodology must include the calibration of more precise material spectral properties of the physical prototypes with the computational simulations. The energy analysis methodology currently simplifies the material properties and dynamic range of movements to accommodate the limitations of the whole building modeler, EnergyPlus. The parallel development of the physical material prototypes with the interactive simulation is pushing widely used software like EnergyPlus to support higher resolution characterization of emerging materials into its tool palette. Finally, exposing the simulation methods to a wider audience is important for incorporating diverse user feedback.

Ongoing Work: Expanding Audiences and Scales

Advancements in physical prototype testing, computational development, and human factors studies all present important yet distinct areas for ongoing work that will inform each other in significant ways. The interactive simulations provide an interdisciplinary framework within which seemingly disparate areas of study can co-exist and where collaborative innovation is fostered. One of the challenging elements of this collaborative work that aims to address user needs, preferences, and desires is to include a diverse range of user input into the design and testing process. This involves increasing access to these tools and environments to remote locations and to the public in order to expand audiences and scales.

The hybrid use of large-screen projection and VR HMDs creates a flexible virtual design space for collaboration. The increased mobility and freedom from physical spatial constraints provides opportunities for designers and users to collaborate from different geographic locations for remote interdisciplinary design using HMD devices such as the Oculus Rift. With the Oculus Rift, challenges of multiuser perspective tracking can also be addressed, whereby multiple people could be wearing an Oculus Rift and occupying the same virtual space, much like a shared gaming environment. In a shared virtual environment, users can simultaneously interact with adaptive building envelope systems and spaces and still receive the same real-time measured energy performance feedback of ambient lighting or thermal flows, visualized as three-dimensional pseudo-color matrices or computational fluid dynamic analyses. Unlike many typical architectural applications of HMDs, users have the ability to gesturally interact with the adaptive facade system and could potentially engage in a learned dialogue between the material, energy flows, and other people in the same virtual space. Because of the limitless scale of virtual worlds such as Unity3d, a multitude of architectural proposals can be designed and explored at the building skin, building, or urban scales. As the simulation environment is further developed to support interaction at multiple scales, users will have the opportunity to modify not only the behavior of dynamic systems, but also participate in designing and interacting with energy flows across the facade, building, and city (Fig. 11.7).

Fig. 11.7
figure 7

Preliminary examples of user interactions with simulated energy flows at the building facade and urban scales: a Simulation of facade’s energy flows from the interior perspective, b User interacting with the facade simulation using a combination of the large-screen projection and the Oculus Rift VR HMD, c Simulation of urban energy flows from an aerial perspective, d Two users interacting with the urban energy flows through the large-screen simulation environment using the hybrid-reality environment

Full size image

There are several aspects of this methodology to consider relative to designing, analyzing, and observing user interaction with adaptive architectural systems in real-world settings. Social behaviors and preferences for interacting with dynamic building systems are likely to differ in real-world settings versus those that are simulated in a laboratory. Exposing the interactive simulation methods to a wider audience and in different locations will be important for getting diverse user feedback, especially to better understand the various tendencies and preferences of people when engaging emerging systems for the very first time or over long periods of time. Our ongoing work includes designing algorithms for various degrees of interaction with dynamic systems, buildings, and cities that can be demonstrated and tested outside the lab and in the broader public realm – through public exhibitions, museum installations, and facade testbeds.

Moving forward, improved computational methods and sensing algorithms for multiuser interactivity will also generate opportunities for more comprehensive participant experiments that explore a greater range of human factors issues. The integration of alternative sensing technologies for multi-modal interactions can heighten the perceptual experience and learned capabilities of the building system or environment, such that a dialogue continually takes place between multiple users, responsive architectural systems, and energy flows. For interactive artist Usman Haque, accounting for underspecified and observer-constructed goals enables the collaboration and convergence of shared goals in connecting with our environmental systems (Haque 2007). In the context of highly-responsive building envelope systems that are open to the inputs and preferences of many different people from diverse backgrounds, this convergence could result in an unanticipated performance between extremely complex and dynamic systems. What’s critical is that criteria for ecological design enables multiple readings, interpretations, and degrees of user engagement, and that these methods are exposed to a wider audience, where people become players in the development of these systems. Regardless of the enabling technologies for these emergent interactions and assemblages of knowledge space, it seems inevitable that maintaining degrees of choice in the ways that people participate, engage, and observe the aspects of environmental performance will be essential in developing the criteria for responsive architecture.

Adaptive Architecture: Toward User Empowerment

Simulation environments that support interaction with adaptive building materials and envelope systems enable sustainable architectural design practices to expand beyond energy performance criteria to include multiuser desires for diverse comfort preferences, degrees of interaction, and overall aesthetic effects. In the case of the EDDS and many adaptive architectural skins and systems in development, energy performance goals of modulating light and heat have the opportunity to blend into extensions of human performer, expressing emotion and desire, whereby one’s decisions fluctuate according to ambient energy flows or the interactions of other people with the system. In this case user empowerment comes with degrees of participation that adapts with the material and computational developments. Personal preferences, needs, and ideas might evolve based on their exposure to the technology and to exposure of others’ choices. In architectural discourse on sustainable design, the focus no longer needs to associate an architectural design intention with either energy-driven or aesthetically-driven criteria, but rather adaptive building skin systems such as the EDDS offer a both/and condition, where environmental mediation is an expression of user empowerment and interactivity at multiple scales. In the process of expression, individual and collective identities emerge for a diversified experience that is at once sustainable and empowering.

The authoritative role of the designer or architect becomes ambiguous as primary author and instead is transformed into a choreographer of material, energy, and information. Within the simulation, certain material parameters are pre-assigned by the architect, but the behaviors and visual outcomes are a result of a negotiation between solar- and occupant- responsive interfaces, atmospheres, and effects. Design agency is not limited to the intentional actions performed by a system or by people; instead it embodies people, material responses, and energetic flows, and the architectural outcome is temporally emergent. The interactive simulation environment allows us to stage dances of agency as a way of exploring how we get along with these new materials, our environment, and with each other. Performance criteria don’t rely solely on quantitative benchmarks, but rather are an entanglement of qualitative and quantitative characteristics, human, and non-human agents. Variability, choice, and learning from the architecture and from each other could lead to greater occupant satisfaction while reducing energy consumption in buildings. Introducing individual agency—and perhaps most importantly, various degrees of engagement—to the expression of the architecture expands design opportunities for building-integrated energy performance and for redefining cultural expectations for environmental comfort.

Impacts on Materials Science: Criteria for Material Behaviors

In returning to the iterative co-design process introduced in the Discussion section, the design feedback loop between experimental physical prototypes and the interactive simulation environments are especially important in the context of the EDDS prototype development. For example, as a result of the interactive architectural simulations of the EDDS micro-scale material assembly, a new set of architectural criteria embodying environmental and user-driven performance is pushing for far greater adaptability of these materials at the nanoscale. Current research at CASE/Rensselaer is focusing on multifunctional nano-structured materials for energy harvesting and environmental mitigation at the facade, but with an increased emphasis on user interaction, environmental comfort, and information display (Thomas et al. 2015). Nano-material prototypes are now considering criteria for user interaction, environmental comfort, and information display alongside criteria for energy harvesting and mitigation. This expanded set of criteria was introduced during the research process because of the possibilities discovered through interactive simulations.

Based on these material innovations we will increasingly be able to program precise mechanical, electrical, and optical behaviors of materials to respond to a range of environmental inputs, building demands, and physiological needs and individual desires. This multiscalar approach is leading to technical strategies for solar tracking and spectral selectivity for improved glazing performance, and it’s also leading to design strategies that amplify the potential for variable patterning, information exchange, and biomorphic expression of buildings. Environmental and aesthetic criteria at the building scale are informing the design and engineering of new material behaviors at the micro and nanoscale. An extension of the research at the Interactive Design and Visualization Lab aims to develop simulations that support the higher resolution characterization of systems like the EDDS. This is a primary example of how the development of the simulations alongside the testing of multi-scalar physical prototypes is creating an iterative co-design process between physical and computational experimentation.

Impacts on Human-Computer-Interaction: A Co-design Research Process

In designing environmental building envelope systems, engaging both energy metrics and user experience approaches what Felix Guattari has referred to as a “triple ecological vision” (Guattari 1989), merging the intertwined registers of social, mental, and environmental ecologies. Self-determination and individual conceptions of personal preferences for environmental quality and visual effects cannot be disregarded or relegated as secondary to energy performance benchmarks. Previous attempts have approached the building envelope technocratically as an isolated problem to be solved. The false dichotomy established between energy performance and user engagement is one which must be challenged. By testing new material innovations and interactive design tools in action and according to broader audiences, a more encompassing vision of ecology is possible.

Advancing immersive and interactive simulation environments for emerging architectural materials and technologies could provide radically new interdisciplinary opportunities at the intersection of architectural design, materials science and human-computer-interaction. The ability to experience and test dynamic visual, aural, or haptic perception within shared physical environments entails an inherent exploration into the social organization and politics of space. Combining innovative design processes into synthetic testing environments that utilize distributed interactive computing and/or big data allows for architects, computer scientists, and interaction designers to participate in the making of multifunctional material behaviors. Simultaneously this allows them to explore the ecological, spatial, and social implications of these compositions through immersive experimentation.

Future developments in adaptive building technologies and spaces will continue to inform the need for new computational design and interactive prototyping methods for predicting the technology’s performance according to a range of architectural, social, and environmental criteria. By focusing on the spatial and cultural potentials at the intersection of human desires, material behaviors, and energy flows, material technologies and human-computer-interaction (HCI) methods will support broader visions of sustainable architecture and ecological design. HCI methods will be especially important in the increased use of VR as an effective and usable design, visualization, and analysis tool. Within the built environment HCI research can facilitate interactions between users and VR systems and support iterative prototyping and testing outside of lab environments to figure out the best way to build user-friendly interfaces (Kim et al. 2013).

Through the co-development of hybrid physical-computational design simulations, the formation of individual identity simultaneously occurs for both the designer and end-user through the process and product. This approach, however, cannot be achieved by architectural designers alone. Experts in HCI can contribute significantly to the systemic, analytical, and navigational knowledge as it relates to the interface design and user experience with emerging technologies that typically don’t fall within the architectural material palette. Because of these different approaches, the collaboration of architects and HCI designers can have profound impacts on how the simulation parameters and workflows are organized, defined, and implemented within our built environment in environmentally and culturally productive ways.

Rather than solely operating in isolated vacuums, each field engages each other’s methods of research. Whereas the sciences typically decouple variables allowing for the testing of hypotheses, architectural design processes focus on simulating complexity (Latour 2008). Unlike the premise of the scientific method and its rational procedures, architectural design is not linear. Creative processes between multi-disciplinary researchers and methods are messy, iterative, and informed by sometimes illogical choices. Inevitable conflicts ensue in the exchange of ideas and the negotiation of value systems. Despite these challenges, the combined team of faculty members, students, professionals, and end-users disrupt traditional hierarchies of contribution and credit, allowing for a transparent exchange of ideas found typical in architectural design. The presumed boundaries of these knowledge spaces reveal themselves to be porous and transmissible. This collaboration demonstrates that the entanglement of sociological factors, while typically characterized as barriers, can be catalyzing rather than paralyzing constituents in the production of a synergistic co-design research process. They can generate constructive tensions and pivotal moments within the ‘messy’ production of shared knowledge space (Turnbull 2009). Critically, this work demonstrates that access, experimentation, and observation within a shared space—physical or virtual—is necessary to expose each other to alternative methods, to identify overlaps in research, and to invite new design methodologies that expand beyond typical disciplinary boundaries.