Computing has become an established part of the built environment augmenting it to become adaptive. We generally assume that we control the adaptive environments we inhabit. Using an existing adaptive environment prototype, we conducted a controlled study testing how the reversal of control (where the environment attempts to influence the behaviour of the inhabitant) would affect participants. Most participants changed their respiratory behaviour in accordance with this environmental manipulation. Behavioural change occurred either consciously or unconsciously. We explain the two different paths leading participants to behavioural change: (i) we adapt the model of interbodily resonance, a process of bodily interaction observable between, for example, partners engaged in verbal dialogue, to describe the unconscious bodily response to subtle changes in the environment and (ii) we apply the model of secondary control, an adjustment of one's own expectations to maintain the pretence of control, to describe conscious cognitive adaptation to the changing environment. We also discuss potential applications of our findings in therapeutic and other settings.
We describe an experimental study of the control relationship in biofeedback environments.
Inhabitants on average synchronize their physiological behaviour when they lose control over the environment.
Inhabitant behaviour is explained through a form of interbodily resonance and secondary control.
We see potential therapeutic uses of combining biofeedback and automated control.
The confluence of the Internet of Things (IoT), the Quantified Self movement and the Digital Turn in Architecture (cf. Carpo, 2013; Kolarevic, 2003) is making Weiser's vision (Weiser, 1991) of the computer of the 21st century (almost) a reality. In an effort to know more about their home environments and to be able to control more aspects of their homes, a growing number of people are installing IoT technologies, such as the Nest (Nest Labs, 2016) and Eve (Elgato, 2015) home automation product ranges, or the Canary security system (‘Canary,’ 2014), to name but a few. Such smart home technology now interfaces with mobile phones, for example through the Apple iOS HomeKit. This framework enables the communication with and control of IoT devices, such as remotely being able to read the room temperature and turning the heating on or off. Some of these can also be automated in response to the home owner's behaviour, such as turning the heating or air conditioning on when s/he is starting the commute back home. In addition to an increasing interest in data about our homes, many people have begun to measure their own performance, especially during exercise, framed by what has been coined Quantified Self movement (e.g. discussed in Parviainen, 2016; Perakslis et al., 2014). This reveals an interest of people in the measurable aspects of the physical body, such as steps taken and floors climbed per day, or the current heart rate (HR). Some argue that the feedback sought from measuring physiology and comparing it to a behavioural goal helps with controlling behaviour (cf. Hermsen et al., 2016), for instance exercise habits. In the context of this development, software infrastructures are emerging, for example the iOS HealthKit, and these allow the collection of personal activity data and communication of that data with third-party applications and devices, such as FitBits (Fitbit Inc., n.d.) among many others.
Technically, it is a simple step to connect data about the home and the built environment more broadly with data about the self and the body more generally. Subsequently, this would allow people's bodies to be ever more closely linked to the spaces that they inhabit. This is already developing, despite us not always being aware of it. The increasing number of sensors (IoT devices) distributed in the built environment connect us to architectural space more intimately and more directly than ever before.
1.1. The Digital Turn in Architecture
While the consumer markets for IoT for buildings and for quantified wellbeing might not yet have fully merged, some of the possible interactions between real-time behavioural occupant data and the responding architectural space are already being explored in the form of unique, playful and provoking experiences in architectural and artistic settings. This occurs in the context of the Digital Turn in Architecture, as for example mapped out by Carpo (2013). For the last few decades, this has resulted in a deep integration of digital tools into the design and production process, first via CAD and now via Building Information Modelling. More recently, it has led to an increase in experimentation with Adaptive Architecture, defined by Schnädelbach as concerned with buildings that are specifically designed to be adaptive to their environment and to their inhabitants (Schnädelbach, 2010). The diversity of approaches, interpretations and emphases of what architectural adaptivity could and should be now and in the future is evident in the variety of terms used to describe it. They range from digitally driven (Bier and Knight, 2010), to responsive (Bullivant, 2006), to interactive (Fox and Kemp, 2009), to dynamic (Kolarevic and Parlac, 2015), to robotic (Green, 2016) and more generally to adaptive environments. Prototypes reported in the literature above also span a variety of interactive approaches. Many implement a relatively simple, reactive interaction loop. For example, the motion of inhabitants is coupled with the motion in an architectural prototype. Even simpler, direct interaction like a button press triggers a change in the architecture (Green et al., 2009). There are also proposals that draw on machine learning approaches, mining recorded behavioural data over time to enable better profiling and decision-making, see Mozer's Adaptive House for an early example (Mozer, 2005). Finally, researchers have proposed providing environments with computational agency, so that the environment can follow and instigate behaviours. One of the very few functional and evaluated prototypes is ADA, providing multiple inhabitants with a playful and engaging experience, developed for the Swiss Expo 2002 (Eng et al., 2006).
1.2. Adaptive architecture that responds to the body
Most relevant for the discussion in this paper are those prototypes that use real-time physiological data to respond directly to the bodily behaviours of their occupants. Currently, a recurring approach to link people's bodies and adaptive environments is to make that link via respiration (RSP), as for example demonstrated by projects such as Lungs [the breather] (Guerra et al., n.d.), Breathe (Jacobs and Findley, n.d.), ExoBuilding (Schnädelbach et al., 2010) and Sonic Cradle (Vidyarthi et al., 2012). Heart rate has been used in Khut's work (Khut, 2007) to change 2D projections, while electromyography enabled Space Trash (The Principals, 2014) to respond to its inhabitant. A commonality between these projects is that the interaction is computationally very simple. They directly translate an input signal, such as inhalation and exhalation, into an output signal, such as an upward and downward motion, i.e. there is no machine learning employed nor computational agency given to the prototype itself. Exemplifying the simplicity of respiratory interactions and environment, Breathe both reflects and records the breathing of a single inhabitant. Ceiling-hung strings of one colour sway in response to the inhabitant. Simultaneously, Breathe replays the breathing pattern of the previous inhabitant using different coloured strings, allowing time-shifted inhabitant interaction. Compared to Breathe, Sonic Cradle (Vidyarthi et al., 2012) and ExoBuilding (Schnädelbach et al., 2012), the only real-time adaptive environments of which we are aware that have been empirically studied (though user studies of, for example, robotic environments (cf. Green, 2016) have been reported), provide a more visceral experience. Sonic Cradle, a dark chamber in which the user sits, creates an adaptive soundscape in real-time response to inhabitant RSP. ExoBuilding, described in detail later, translates inhaling and exhaling in real-time into upward and downward movement of a tent-like fabric inside which an inhabitant sits.
As explicitly described by the authors, the interaction with both ExoBuilding and Sonic Cradle is based on biofeedback, which is a training method to gain control over involuntary/automatic bodily functions. It uses physiological sensors and output devices to make trainees aware of their physiological behaviour to enable them to learn how to control it (cf. Schwartz and Andrasik, 2003). Both groups of researchers report reductions in user RSP rates (RSPRs) when engaged in this form of feedback. All environments that reflect real-time bodily behaviours to their inhabitant establish an interaction cycle or loop with their inhabitant, as described by Schnädelbach (2011). This (feedback) loop can temporarily couple an architectural environment with the human body.
The above-described trends in the IoT spanning the home and the body in the context of the Digital Turn in Architecture result in a general environment and the specific technical capabilities to make digital links between people's behaviour and that of the built environment around us. This is set to create increasingly responsive and adaptive architectural spaces, which we generally assume to be in control of. Our own experimental work (Schnädelbach et al., 2012) has triggered us to query this assumption and pose the following question: To what extent do we control adaptive environments and to what extent do they assume control over us? In what follows, we briefly frame this question by presenting the relevant research background with regard to embodied interaction and control in architecture.
1.3. Embodied interaction
Architects have long been engaged with embodiment theories and its relevance for the experience of static, non-adaptive architecture, as publications by, for example Mallgrave (2013, 2011), Pallasmaa and Holl (2008) and Pallasmaa (2009) illustrate. However, behaviour responsive architecture now offers new interaction paradigms, as discussed in Architecture and Interaction (Dalton et al., 2016) in general and our chapter Embodied Interactions with Adaptive Architecture (Jäger et al., 2016) in particular.
Being able to sense and respond to inhabitants in real-time provides buildings with what in humans is called a perception–action loop, a form of ‘dynamical attunement of organism to environment’ (Gallagher and Bower, 2014, pp. 241/242). Establishing such a loop would enable real-time communication between building and inhabitants. And in turn, buildings equipped with a perception–action system allow inhabitants to exert both indirect and direct control over the emerging adaptive architectural spaces surrounding them. Thus, as Haque (2015) explains ‘We no longer think of architecture as static and immutable; instead we see it as dynamic, responsive, and conversant.’ Indeed, a result of the echoing of behaviours is the forming of an interaction system between inhabitant and their adaptive environment similar to that between a person and their mirror image: architectural space reflects human behaviour in real time. This reflection of their own behaviour enables inhabitants relate to the environment in an embodied way by using the physicality of the body.
Embodiment is a concept to explain cognition that rejects the Cartesian separation of mind and body. Currently, a synthesis of four different aspects of embodied cognition, the so-called 4E approach (Menary, 2010), is the most comprehensive explanation of embodied interactions with our world. It consists of the interdependent concepts of embodied, extended, embedded and enacted.
Embodied describes cognition as depending on the body and even occurring in its physicality. An example is our stereoscopic vision, which enables us to see and interact with three-dimensional objects because of the specific distance the eyes have to each other. As Rowlands (2010) explains, this allows us to judge the size and distance of objects in relation to us. Through our haptic, gustatory, olfactory, visual and audial senses our body reaches out and extends itself into the surrounding physical world. Subsequently, extended describes the body's interaction with the environment in order to extract inherent information from objects (Rowlands, 2010). Extended adds intentionality to the pure physicality of the body's senses described in the embodied concept. The extension of the body also applies to our use of digital technology, as Clark (2004) explains. He argues that digital technology extends our body's abilities in a similar fashion as traditional hand tools do. The body is also situated in its environment—or it is embedded. The human body accesses specific parts of the surrounding environment in order to reduce mental load on the brain. Haselager et al. (2008) explain that the location of the body within an environment facilitates ad hoc access to information in the environment without the necessity for the brain ‘to maintain any internal models of the world’. In contrast to the extended perspective in which the body probes its environment for information, embedded describes information as easily accessible to the body, which is immersed in a specific context. Finally, the enacted concept refers to the body acting on the world. Rowlands (2010) describes this acting primarily as exploration of the world's physical elements with the ‘visual modality’. In contrast, De Jaegher and Di Paolo (2007) emphasize the multi-sensory coupling between agents and the environment and their continuous embodied interaction. This is congruent with McGann et al. (2013) who argue that the core of the enactive approach is ‘the dynamic interaction between the agent and the environment.’
Especially, the enactive approach is crucial for investigating our interactions with Adaptive Architecture. As we described above, inhabitants of Adaptive Architecture that reflects physiological behaviour are coupled to their environment through an interaction loop. As long as they, for example, breathe to interact with their environment, they engage in a continuous embodied interaction and, thus, exert control over the environment and the interaction.
1.4. Multiple approaches to control in environments
So far, most of our understanding of how we relate to and exert control over architecture stems from physically static environments. For example, psychological research has shown positive effects of having and exerting control over common environmental features, such as lighting or levels of aversive noise. Having control (usually as a single call-and-response) reduced stress (Glass et al., 1969), increased attention-to-detail and task persistence (Sherrod et al., 1977) and increased performance and creativity (Green et al., 2008) in participants.
Influencing effects can also be observed in the opposite direction, with researchers documenting the effects of environments on people's perception and behaviour (Purcell, 1987). This includes research of and applications in, for example, retail environments where atmospherics (cf. Turley and Milliman, 2000), such as music, scent and spatial arrangement are being deliberately manipulated to affect customer perceptions and behaviour. Similarly, the disciplines of environmental psychology and architecture co-developed the intensely debated concept of architectural determinism. This concept originally claimed that it is possible to predict human behaviour in architectural space, for example discussed by Lee (1971) and Marmot (2005). Related but less controversial is the architectural research practice of space syntax. It aims to show the effects on movement patterns and social interaction of a given spatial layout through spatial analysis (cf. Hillier, 1998).
Digitally driven environments now allow for actual, continuous control (rather than mere influence) to exist bi-directionally (from human to environment and from environment to human) in the form of reciprocal control. However, this flow of control within dynamic, and particularly, kinetically adaptive environments as well as its effects on human perception and behaviour is not yet sufficiently studied and understood. The only example we are aware of that has investigated the link between deliberate architectural motion that controls inhabitant movements is a study by Lee and Kalmus (1980). In it, the researchers investigated the optical flow field and found that motion of an analogue environment can indeed ‘control’ inhabitant movement, such as swaying in in-phase synchrony with a moving room surrounding them.
Recently, the control relationship between inhabitant and environment (re)entered the discussion of adaptive architecture. For example, Sterk (2015)—though not yet including real-time physiological data—argues for adopting a hybridized model of control in the context of user-centred robotic architecture. This model consolidates work by Eastman (1972) and Friedman (1972) by combining direct manipulation (participatory) and automation (self-regulating) features of interaction. On the other hand, Fox (2015), arguing for bio-robotic architecture that is both controlled and self-controlling, briefly discusses novel methods of control to interact with and rearrange architectural space. Part of this novel interaction with architecture, he argues, will be sensor based including cognitive control (brain wave sensing) and gesture-based manipulations of architectural space. Thus, Fox implies a direct link between architecture and inhabitant that might be bi-directional. However, neither Sterk nor Fox validate their theories empirically.
1.4.1. Reciprocal control—interpreting two existing prototypes
Out of the examples of digitally driven kinetically adaptive environments, so far only OpenColumns and ExoBuilding offer concrete examples of the potential benefits of reciprocal control between inhabitants and adaptive environments.
OpenColumns are ceiling-mounted poly-elastomer meshes that respond to CO2 concentrations in an interior space (Khan, 2010). CO2 levels increase as a result of exhalation. When CO2 levels near a column reach a pre-defined threshold, the column slowly drops to the floor, encouraging the dispersal of the inhabitant group underneath, who had caused CO2 levels to rise through their RSP. Thus, OpenColumns dynamically ‘control’ inhabitant distribution in an interior space and establish a slow-acting feedback loop between inhabitants and environment. To the best of our knowledge, no study of OpenColumns has so far been reported.
In contrast, a more intensely studied feedback environment is ExoBuilding (Schnädelbach et al., 2010). It is a single inhabitant, tent-like structure that changes its height, volume, shape, colour and sound based on its inhabitant's real-time physiological data. For example, when inhabitants breathe in, the structure expands and when they breathe out, the structure contracts. The feedback loop generated can have profound effects on inhabitants. Schnädelbach et al. (2012) have shown that in this feedback mode, inhabitants tend to reduce their RSPRs, breathe more deeply and more regularly without having been instructed to do so. In both a no-motion condition and a condition in which the environment moved automatically, no such effects were found.
1.5. Possible applications of embodied control in adaptive architecture
A few application areas for embodied control in Adaptive Architecture are already apparent. For example, they can easily be imagined in settings where relaxation and mindfulness are being sought. This link has already been shown by Vidyarthi and Riecke (2013) who used Sonic Cradle to mediate mindfulness, while Schnädelbach et al. (2014) experimentally showed that being immersed in ExoBuilding more easily allowed participants to be mindful than experiencing the interaction from without.
We also envision applications in yoga, certain kinds of which already make the environment a central feature in its practice. For example, Iyengar yoga uses props, such as chairs, benches and blocks, to help yoga practitioners achieve perfect postures. Similarly, in anti-gravity yoga practitioners use ceiling-hung hammocks to facilitate and support yoga postures. With the support of digital technology, the environment could become a more active participant in the practice of yoga. It could for example provide feedback on breathing, as our previous work in this area (Moran et al., 2016) shows. We found that respiratory biofeedback delivered by the adaptive environment influenced the choice of postures during yoga sessions and positively affected group cohesion. But digital adaptivity could also predictively support postures to avoid injuries and possibly challenge practitioners to achieve better performance.
In addition, the calming effects on participants described by Vidyarthi, Riecke, and Gromala (2012) and Schnädelbach et al. (2012) would make such environments suitable as therapeutic spaces, making use of their biofeedback properties. Using technology in therapy is a common approach and has been employed in a variety of settings, such as stroke rehabilitation (Chen et al., 2010; Matamoros et al., 2009), in physiotherapy (Kousidou et al., 2006; Lam et al., 2015) and generally to reduce stress (Chung et al., 2009; Muench, 2008). Thus, extending this approach to include architectural space is a logical step. Understanding if an adaptive architectural environment can principally control physiological behaviour of its inhabitants, for example to provide therapy, would enable designers and healthcare professionals to create supportive spaces and interactions with such spaces.
1.6. Article structure
The remainder of the paper discusses our experimental investigations of reciprocal control in adaptive environments. We first introduce the experimental methodology. This includes a description of the experimental environment, the aims and objectives of the study and the physiological principles used in the study. We also explain the experimental design, the manipulation, participants, procedure, hypotheses and measurements. The following two sections contain the results of a pilot study, which investigated extended exposure to our experimental environment, and the formal study for which we compare the two experimental trials and analyse the effects of the manipulation. Further results include a questionnaire on perceived control as well as an analysis of interview responses after the experimental trials. In the discussion, we interpret our results by relating physiological effects to participants’ self-reports. We apply the concept of interbodily resonance, a pre-cognitive response to the experimental manipulation, and adapt it to the context of adaptive architecture. We also discuss a cognitive response to the experimental manipulation termed secondary control, before concluding with a summary of our findings and an outlook to future work.
2. METHODOLOGY OF STUDYING RECIPROCAL CONTROL
2.1. Experimental environment: ExoBuilding
We used the aforementioned ExoBuilding as experimental environment. In contrast to the brief description above (see Section 1.4.1), colour and sound adaptations were disabled for the study described here to avoid confounding effects. ExoBuilding consists of jersey fabric that stretches over an aluminium spine, which hangs from two ceiling-mounted servomotors. The motors move the spine vertically by about 30 cm (see Fig. 1). Combining physical structure, sensing technology and a custom middleware platform called Equator Component Toolkit (Egglestone et al., 2006) allows direct physiological interaction between inhabitant and environment. ExoBuilding is mainly a dark space: the ceiling lights are extinct and only the projection of a circular blue graphic and residual light remain visible.
2.2. Aims and objectives—physiological control
In order to investigate how inhabitants perceive their control relationship with a biofeedback environment, we aimed to examine how the reversal of control in an adaptive environment—firstly, the participant being in control, and then secondly losing it—would affect participants’ physiological behaviour and psychological state. Would participants’ respiratory behaviour fall into synchrony with the environment after control had been transferred to the environment?
Leaving this question unanswered would potentially lead to unintended, possibly negative physiological and psychological effects of adaptive environments on their inhabitants. Not understanding the effects may result in detrimental effects, such as the ‘over-awareness’ mentioned by Schnädelbach et al. (2012) or even direct harm as the environment may cause some inhabitants to physiologically behave in unnatural and perhaps unhealthy patterns. For example, they might begin to hyperventilate, which means they would breathe too quickly, which can cause dizziness, and fainting. Or they might hypoventilate, i.e. breathing too slowly to provide enough gas exchange.
To answer the question whether participants would align their behaviour with the environment, our objectives were to (i) lead participants to believe that they would always be in control over the environment, (ii) use a manipulation entailing an unnoticeable transfer of control to the environment and (iii) utilize a physiological phenomenon, called HR variability (HRV, explained below), that required participants to focus both on their own physiology and the environment as feedback device. As part of the manipulation, the environment initiated a change in its own behaviour, attempting to make participants follow this new behavioural pattern. Through the use of HRV as interaction mechanism with ExoBuilding, we created a more sophisticated interaction with the environment than a call-and-response relationship. We intended this interaction mode to both increase the participants’ awareness of their own body and, simultaneously, make it less obvious how to control the environment. This ambiguation of control would hopefully help us disguise the transition of control.
2.3. The physiology of controlling the environment
We mapped HR to the upward and downward motion of the structure. Humans can voluntarily control their HR, but only indirectly. HR is, among other processes, linked to RSP. When HR varies (HRV) as a result of breathing this is termed Respiratory Sinus Arrhythmia (RSA; Cacioppo et al., 2007), as shown in Fig. 2. RSA is desired in, for example, biofeedback training (cf. Schwartz and Andrasik, 2003) since ‘prominent RSA’ indicates a healthy, well-exercised heart (Yasuma and Hayano, 2004).
Figure 2 shows the stepped HR curve and the RSP trace below it in near perfect alignment: as the RSP curve rises (inhalation), the HR curve also rises (faster heart beat). Generally, at low RSPRs (below eight breaths per minute) the difference between fastest and slowest heart beat in one cycle is maximized: HRV is strongest (Song and Lehrer, 2003). Thus, through slow abdominal breathing, it is possible to maximize and, crucially, control HRV, which reduces stress (Prinsloo et al., 2011), for example. We gave participants instructions (see below) allowing them to achieve high and regular HRV through self-paced breathing close to their resonant frequency (Lehrer et al., 2000). Executing our instructions allowed participants to achieve self-control over their HR and enabled them to indirectly control upward and downward motion of the environment: slowly inhaling caused HR to increase (more beats per minute) and the environment to move up. Slowly exhaling caused HR to decrease (fewer beats per minute), resulting in a downward motion of the environment. The more HR and RSP were aligned, the more directly participants would perceive their physiological control over the environment.
Two preparatory studies and 1 pilot study (totally 13 participants) indicated that most participants would achieve the state of indirect control. Our designed trial sequence also allowed all participants ample time to learn how to achieve self-control, control ExoBuilding and establish a close physiological link to it. Participants were likely to perceive relaxation after the study due to our instructions. But this was not an explicit goal of the study.
2.4. Experimental manipulation—taking control away
To study whether an adaptive environment can indeed control its inhabitants, we designed a 2-fold manipulation that would (i) unnoticeably transfer control from the participant to ExoBuilding at a specified point and (ii) alter ExoBuilding's behaviour. The latter would allow us to measure if the environment, in effect, controlled participants. When participants synchronized with this new environmental behaviour, it would indicate that the environment had controlled them.
We achieved a seamless transfer of control by first tracing the participants HR behaviour (ref. Fig. 3, stepped curve), adapting an automated data stream (ref. Fig. 3, smooth curve) to participant data and then blending both data streams until only automated data were driving the motion of the environment. Once only automated data controlled ExoBuilding's motion, a deceleration of the data took place resulting in a slowing motion pattern of the environment until this reached 80% of the original motion frequency. We chose a reduction of 20% as it provided enough measurable difference to the starting frequency and would not cause physiological stress, such as ‘under-breathing’, if participants followed this reduced rate. Since slow RSP is associated with relaxation (Song and Lehrer, 2003), a reduced breathing rate seemed a generally desirable goal.
In technical terms, a middleware component traced the participant's HR curves and, using zero-transitions, calculated cycles per minute (c.p.m.) of the normalized curve. With HR mapped to the motion of the environment, these c.p.m. were equivalent to ExoBuilding's motion frequency. At 4m30s, another component started identifying the next upward zero-transition of the HR curves, upon which a 15-s (Cubic-Bezier-based) blending of live participant data and automated data (Fig. 3: P/E), a sine wave of the same frequency as ExoBuilding's current motion frequency (Fig. 3: 100%), set in. Due to the tracking of zero-transitions, the exact point of the control transfer varied per participant. Thus, Phase A of Trial 2 (Fig. 4) lasted from 4m30s to 4m49s (M = 4m34.5s, SD = 4.68 s). After blending the data, only automated data drove the structure. Hereafter, the starting frequency of the automated data will be referred to as system rate (SR). Subsequent to the blending period, SR decelerated by 20% over a period of 3m00s. For example, if a participant's HR curves had a frequency of 10 c.p.m., the environment would move 10 times per minute, the SR. The transition, then, decelerated SR to 8 c.p.m. (Fig. 3: 80%) within 3m00s. Thereafter, ExoBuilding's motion frequency remained constant.
2.5. Experimental design
The repeated measures, within-subject experimental design consisted of two not counter-balanced trials (Fig. 4). Participants were run individually and experienced each trial as a 9-min period of sitting inside ExoBuilding following minimal breathing instructions. Trial 1 consisted of participant biofeedback control over ExoBuilding. Trial 2 was split into two overall phases: A and B (Fig. 4). Phase A lasted at least 4.5 min and was characterized by participant HR biofeedback control. Phase B, characterized by ExoBuilding control, consisted of transition and post-transition.
After the ethical review board had approved the study, we recruited 31 participants (16 Female and 15 Male) between 18 and 50 years old with a mean age of 26.5 (SD = 6.26). They were undergraduate and postgraduate university students; with the majority (16) being Caucasian. We advertised across campus (posters) and through department-wide email distribution. Conditions for participation were to have no heart conditions, no respiratory problems and not to be claustrophobic. Participants were compensated for their time with a £20 online retail gift certificate. Members of our laboratory were not eligible to avoid biased results.
Figure 4 shows a graphical representation of the experimental session. The vertical axis on the left represents progress through the session, while the horizontal boxes labelled (1) and (2) represent the two experimental trials. The figure also includes a legend in the top right corner, labelling all items of the figure.
First (Fig. 4: I), participants received an introduction to the experiment including written and verbal information about HRV and RSA, as well as ExoBuilding's function. Participants were always given the option to terminate the experiment at any time. None exercised this option. The experimenter did not reveal the true nature of the study at this stage, as this would have alerted participants to anticipate a change in the environment. Instead, if asked, the experimenter explained that the purpose of the study was to investigate differences between first and second time exposures to HRV biofeedback through an environment. Additional measures to avoid participants expecting a change between the two trials were that the two trials were designed to look and feel identical to participants. They performed the same walking exercise before both trials, the trials had the same length, same instructions, same interaction with the environment, and participants filled out the same questionnaires before and after each trial. The vast majority of participants were computer science students who are not used to participate in experiments in general. Moreover, they are not familiar with experiments that include any form of deception. Accordingly, we are positive that they did not expect a manipulation in Trial 2. To assess their psychological state before and after each trial, we asked participants to answer psychometric questionnaires three times during the experiment: before Trial 1 (Q1), between trials (Q2) and after Trial 2 (Q3). Q3 also contained a small drawing task and interview. After the interview, all participants were fully debriefed regarding the nature of the study.
Before each trial, the experimenter asked participants to walk ca. 45 m at a speed slightly faster than their usual walking speed (Fig. 4: E). This ensured comparable levels of alertness before each trial. Also before each trial, participants sat in the reclining chair, receiving brief and identical instructions for slow and regular breathing:
Please keep your eyes open for the duration of the trial. Breathe with your diaphragm only. Try not to raise your shoulders while breathing. Focus on your breathing. Breathe slowly and gently. During each exhale count slowly and silently to 4. Some people also count slowly to 4 while inhaling, which helps to achieve a good rhythm.
We designed these instructions to allow each participant to achieve high and regular HRV throughout the trials. This would result in smooth and regular motion of ExoBuilding. The testing phase (TP) allowed the experimenter to verbally guide the participant towards a respiratory rate that achieved high and regular HRV. It also allowed participants to familiarize themselves with the environment and the physiological control mechanism.
2.8. Hypotheses and expectations
We had two hypotheses for our quantitative data:
H1: Participants should reduce their RSPRs as a result of the manipulation (Trial 2, Phase B). The manipulation was designed to reduce motion frequency of ExoBuilding by 20%. Thus, participant RSPRs should decrease by approximately 20%.
H2: Due to the unnoticeable manipulation, perceived control scores after both trials should not differ significantly.
Based on these hypotheses, we expect that participants will reduce their RSPR (H1) but not notice a loss of control (H2). In turn, if participants do notice a loss of control, we expect it to be less likely that they synchronize their behaviour with ExoBuilding, which would be congruent with the results of the automated motion condition in Schnädelbach et al.’s (2012) study, as described above.
During the interviews, we anticipated participants to elaborate their relationship with ExoBuilding, which might have included articulating whether they had noticed a loss of control without prompting them to do so.
The experimenter fitted each participant with electrodes (electrocardiogram, galvanic skin response) and an RSP belt. These sensors (sample rate: 32 samples per second), including the software BioTrace, are part of the NeXus-10 bio-sensing unit by MindMedia (2015).
We administered the Perceived Control scale, a 10-item Likert-style scale, after each trial (Fig. 4: Q2 and Q3). This scale assesses the level of control participants perceive to have over the motion of ExoBuilding, the environment in general, and the session. The questions of this scale were modified based on an existing questionnaire by Veitch and Gifford (1996). In Q3 (Fig. 4) we conducted semi-structured, audio-recorded, interviews to investigate the relationship between inhabitant and ExoBuilding. By neutrally asking about anything unusual or noteworthy, we gave opportunities to voluntarily mention the loss of control.
We collected additional psychometric measurements, which we did not further analyse for this study as they explore aspects of adaptive environments unrelated to control.
3. RESULTS I—PILOT STUDY
3.1. Extended exposure to ExoBuilding
Before running the described study design, we tested how extended exposure (20 min) to HR biofeedback through ExoBuilding without any environmental manipulation would affect participants’ respiratory behaviour. This pilot study (n = 8) was as similar as possible to the main study. Participants received identical instructions and answered the same questionnaires as described for the main study.
For analysis, consistent with the analysis of the main study, we discretized the 20 min of raw RSP data into 13.3 segments of 90 s for each participant. Based on the trial design of the main study, we were interested in time segments nine (S9) and 12 (S12). These correspond to the absolute time participants of the main study had spent inside ExoBuilding until pre- and post-transition (cf. Fig. 4) in Trial 2. To examine whether the duration of exposure to ExoBuilding would cause participants’ RSPRs to vary between S9 and S12, we conducted a paired-samples t-test for these values. The t-test showed no significant difference between the RSPRs of S9 (M = 7.75) and S12 (M = 7.33): t(7) = 0.46, P > 0.05, r = 0.67 (Table 1).
|Time period||S9 (12m00s to 13m30s)||S12 (16m30s to 18m00s)|
|Time period||S9 (12m00s to 13m30s)||S12 (16m30s to 18m00s)|
We aimed to ensure that if RSPRs decreased between the time segments, that this reduction would not exceed or approach 20%, the value used in the designed manipulation in Trial 2. The reduction from S9 to S12 is only 5.42%. Therefore, we are confident that participant behaviour is stable over time and RSPRs vary only marginally as long as no manipulation occurs. The minimal overall decrease of RSPR is probably caused by the breathing exercise itself, which stems from standard practice biofeedback breathing exercises (cf. Schwartz and Andrasik, 2003) and the low-light, low-stimulus conditions inside ExoBuilding.
4. RESULTS II—EXPERIMENT
4.1. Physiological data
For the analysis of all physiological data, two participants of the total of 31 had to be removed for (i) failure to follow the given instructions and (ii) a miscomputed transition, leaving 29 participants (n = 29).
4.1.1. Comparison between trials
We compared average RSPRs of Phase A of both trials, T1.A and T2.A, to test whether participants achieved self-control over their RSP resulting in low enough RSPRs to establish a close physiological link to ExoBuilding, as explained above. Both values (T1.A = 6.94 c.p.m. and T2.A = 6.49 c.p.m., Table 2) are well below average resting RSPRs of 12–15 respiratory c.p.m. (Cacioppo et al., 2007, p.238).
T1.A, first 270 s (4m30s) of Trial 1; T2.A, first 270 s (4m30s) of Trial 2; T1.B.90, last 90 s of Trial 1.
To show that RSPRs did not reduce over time as a result of our instructions or exposure to the environment, we compared RSPRs for period T1.B.90, corresponding to the last 90 s of Trial 1, to RSPRs in T1.A. This matches period post-trans in Trial 2. The values were normally distributed and the dependent-samples T-test showed no significant difference between T1.A (M = 6.94) and T1.B.90 (M = 7.15), t(28) = −0.638, P > 0.05, r = 0.63.
4.1.2. Effects of the manipulation
The second trial included the manipulation of transferring control from participant to the environment. To measure if the environment had control over participants throughout Phase B of Trial 2, we calculated RSPRs for the two 90-s periods of the transition (trans.1 and trans.2) and the post-transition (post-trans) time segment (≤90 s) as presented in Table 3 and illustrated in Fig. 5. SR was derived from participant HR cycles just before the transition was initiated (pre-trans).
|SR (c.p.m.)||RSPR (c.p.m.)||RSPR (c.p.m.)||RSPR (c.p.m.)|
|K–S||P> 0.05||P> 0.05||P> 0.05||P> 0.05|
|t-Test||SR and RSPR: t (28) = 6.07, P < 0.001, r = 0.26|
|SR (c.p.m.)||RSPR (c.p.m.)||RSPR (c.p.m.)||RSPR (c.p.m.)|
|K–S||P> 0.05||P> 0.05||P> 0.05||P> 0.05|
|t-Test||SR and RSPR: t (28) = 6.07, P < 0.001, r = 0.26|
Pre-trans, pre-transition (90 s before start of transition); trans.1, first 90 s of transition period; trans.2, second 90 s of transition period; post-trans, post-transition (≤90 s); SR, system rate computed ad hoc via Fast Fourier Transform based on the participant's HR fluctuations in c.p.m.; K–S, Kolmogorov–Smirnov test, Lilliefors corrected; t-test, dependent-samples T-test between pre-t and post-t values.
Kolmogorov–Smirnov's test of normality revealed normal distribution of all Trial 2 data. The dependent-samples T-test (Table 3) showed that the participants’ mean RSPRs during post-transition (M = 5.72 c.p.m.) were significantly lower than the mean SR (M = 6.64 c.p.m.), t(28) = 6.07, P < 0.001, r = 0.26. On average, participants breathed 14.20% slower during the post-transition phase compared to the SR. To confirm that participants adjusted their RSPRs to the decreasing motion frequency of the environment, a one-way repeated-measures analysis of variance was performed for pre-trans, trans.1, trans.2 and post-trans.
Mauchly's test indicated that the assumption of sphericity had been violated, χ2(5) = 17.65, P = 0.003, therefore we report Greenhouse–Geisser corrected tests (ε = 0.72). The results show that RSPRs decreased significantly over time, F(2.15, 60.04) = 6.12, P = 0.003, ω2 = 0.11.
Overall, 25 (out of 29) participants decreased their RSPRs. Seventeen participants decreased their RSPRs within 1 SD (14.3%), that is ±7.15% of the predicted 20% RSPR reduction. This corresponds to a range between 12.85 and 27.15% overall reduction. Three participants decreased their RSPRs beyond 1 SD (more than −27.15% reduction). Five participants decreased RSPRs <1 SD.
In summary, participants on average achieved consistently low RSPRs in Phase A of Trial 1 and Trial 2, indicating that they were able to exercise self-control. In turn, this means that the physiological link between participant and environment was maximized for our design of interaction with the environment. Importantly, participants reduced RSPRs according to the transition of control as the means across pre-t, T1, T2, and post-t indicate (cf. Fig. 5). Furthermore, participants reduced RSPRs significantly and 17 of them (59%) within 1 SD of the target value of 20% RSPR reduction.
4.2. Perceived control questionnaire
The questionnaire on perceived control was adapted from Veitch and Gifford (1996) substituting ‘lighting’ with ‘movement’ or ‘motion’ where applicable, e.g. ‘I had some control over the movement of ExoBuilding.’ It included 10 questions (see
On average, participants reported significantly less control over the motion of ExoBuilding after Trial 2 (M = 10.79, SD = 3.97, SE = 0.74), than after Trial 1 (M = 13.31, SD = 2.35, SE = 0.44). This difference, −2.52, BCa 95% CI [1.30, 3.89], was significant t(28) = 3.58, P < 0.001, r = 0.37, and represents a large effect, d = 1.07.
The interviews were semi-structured and contained 11 pre-formulated questions, asking about the participant's relationship to ExoBuilding. Additional questions appeared if the experimenter asked for clarification, elaboration or had to re-phrase questions. Interviews lasted on average 16m00s including debriefing.
One of the dominant themes emerging from the interviews was control. Participants introduced control in connection with their immediate experience of ExoBuilding.
4.3.1. Control in the relationship to ExoBuilding
When asked about their relationship to ExoBuilding in the final trial, some participants reported that they felt more in control in Trial 2 compared to Trial 1. The following statements show how participants discussed their control relationship with the environment and the perceptual shift towards the environment's behaviour:
P06: It was more symbiotic, it responded better to the way I [behaved], and I felt [that I had] more control over it [in Trial 2]. It was better than the first [trial]. I was less worried about it because [I] knew what was going to happen.
P30: It was like it listens to me. […] It does the same thing that I do. Whenever I breathe it feels like the environment breathes.
Among participants who detected a loss of control statements such as these were common:
P08: For me, there was no relationship at all. […] I didn't feel like the tent was moving with me. […] But in [Trial 2] I think it just followed a pattern.
P31: I think I had less control than before [Trial 1]. I had slightly less influence over it. I just sensed that it was slightly less controlled by me. And it seemed more regular.
Further comments about the control relationship to ExoBuilding referred to the fabric touching arms or hands of some participants (depending on their size). Three participants explicitly mentioned how this affected them, for example:
P01: When I felt it touch my hand, it gave me a sense of [being in] control.
One participant talked about how understanding the control mechanism changed the attitude towards the environment:
P06: The environment is a little bit close when you don't know what it's going to do […] and when you don't know how it's going to respond to you. It feels a bit more threatening […]. But when you know that you can control it, […] you feel a lot more comfortable, it's a little more reassuring.
Finally, for 21 participants having some control over ExoBuilding or the session was so important that they made it a condition to use it again or even buy it:
P07: Yes, I'd be interested. Especially, if I felt that I was in complete control of it.
4.3.2. Controlling one's body
Eleven participants explained their methods of interacting with ExoBuilding by focusing on their physiology (4/29), mental state (3/29) or in terms of new kinds of input (4/29):
P26: I tried to concentrate [more] on breathing. From the first trial I understood that the walls move according to my breathing. So I tried to concentrate more on breathing with my stomach, breathing in and out. […]
P27: It's a very strange experience I haven't tried before. It's like motion control but this is about breath-control. […] it affects the whole environment around me – and actually the changing of the environment affects my mind and it affects my [breathing] input.
The results reveal a more complex picture than either quantitative data or qualitative data alone could provide. In terms of the quantitative data, confirming our main hypothesis (H1), participants, on average, slowed their breathing in alignment with the environment once control was transferred to this environment. We showed that this occurred throughout the transition phase (Table 3). This indicates that participants indeed matched their breathing behaviour to the slowing motion frequency of the environment. Against our second hypothesis (H2), participants reported significantly less perceived control over the motion of ExoBuilding in Trial 2 compared to Trial 1. This signifies that, on average, they had noticed a loss of control despite our efforts to create a seamless transition of control and make both trials appear the same. Based on just these two quantitative results, one would think that participants noticed a loss of control and intentionally adjusted their behaviour to match that of the environment.
The qualitative data offer a different angle. The interviews showed that the intentional decision to follow the environment was made by some of the participants. Another group of participants revealed in the interviews that they were unaware of losing control. For example, they described the relationship to ExoBuilding as more symbiotic compared to Trial 1. In the following, we try to interpret these differing results.
5.1. Physiological effects and self-report
To be able to interpret the results, we first interpret the physiological effects and self-report separately for the time when participants had control over the interaction with ExoBuilding and then for the time when they had lost control.
5.1.1. In control of the environment
While participants were in control of the environment through HR biofeedback in Trial 1 (Phases A and B) and Trial 2 (Phase A), they exhibited remarkably low RSPRs during the same phases (A) of the two trials. Biofeedback practitioners usually aim for rates between six and eight breaths per minute (Schwartz and Andrasik, 2003) but need multiple sessions with their clients to achieve this. The reported rates (T1.A = 6.94 c.p.m. and T2.A = 6.49 c.p.m.) show that participants achieved this self-control over their RSP quickly, with very little practice. These low and consistent RSPRs resulted in smooth and regular, thus, predictable environmental behaviour.
The consistently low RSPRs suggests that participants synchronized their breathing with the environment. This behaviour might suggest that during the biofeedback phase, while being in control, participants had established such a firm bodily connection with ExoBuilding that their personal body acted as one with it. Describing this entwined relationship during biofeedback phases, a participant explained:
P15: It felt like it was making me breathe rather than I was making it move. […] I was almost waiting for it to do my next breath.
The example quote reveals the strong bodily bond between participants and environment (waiting for an artificial environment to execute a normally voluntary physiological behaviour), while it also indicates an uncertainty over the agency of behaviour (it only felt like ExoBuilding initiated the breathing).
To summarize, for our specific design of interaction with the environment, the consistency of RSPRs across the first phases of both trials and the perceived bond between inhabitant and environment meant a high likelihood of a maximally established physiological link between participant and environment before the start of the transition phase.
5.1.2. Losing control over the environment
During the transition, the physiological data showed that participants, on average, slowed their breathing frequency in accordance with the behaviour of ExoBuilding. While most participants had decreased their RSPRs after the transition phase (post-trans), they also reported less perceived control over the motion of ExoBuilding in Q3. The following quote indicates how some participants were aware of the loss of control and continued to align their behaviour with that of ExoBuilding:
P07: I had some control over it first, but everything I did it slowed down linearly: It was making me breathe slower.
This quote shows how P07 noticed the altered relationship to ExoBuilding (P07 having control first but then something changed) and P07 adjusted their behaviour to the environmental change (ExoBuilding making them breathe slower). Concordant with P07’s description, the questionnaire showed that 21 of 29 participants indicated that they perceived to have had less control over the motion of ExoBuilding in Trial 2 compared to Trial 1.
The final group of participants did not notice a loss of control. Indeed, during debriefing, after we revealed the nature of the study, a good third of participants were genuinely surprised that they had been manipulated. As the following statement indicates, some participants were completely unaware of the manipulation. Their behaviour was in such close alignment with ExoBuilding during and after the transition that they never questioned the control relationship:
P19: I guess I followed [the decelerating movement of ExoBuilding]. – I feel a bit betrayed that I wasn't doing it [i.e., controlling ExoBuilding in Trial 2, Phase B].
In summary, (i) one group of participants noticed a loss of control, against our second hypothesis (H2), and continued to breathe alongside the environment, confirming our first hypothesis (H1) and (ii) the other group of participants did not notice the loss of control, which confirms H2, and automatically breathed so closely aligned with ExoBuilding that they perceived to still be in control, also confirming H1.
First, we will look at those participants who did not notice a loss of control and who continued to breathe in synchrony with ExoBuilding.
5.2. Out of control?—pre-cognitive bodily interactions
Concentrating on the group of participants who were unaware of losing control, their behaviour seems to suggest that environment and participant interacted through a form of pre-cognitive bodily communication. Pre-cognitive describes the phenomenon of events or behaviours that occur without (or prior to) rational thought processes guiding them.
It appears as though participants interacted with the environment purely through their bodies, without intentional reasoning.
In the following, we seek to explain this behaviour by applying two models of embodiment, which are both framed by the 4E view of embodied interaction described in the ‘Introduction’ section. Specifically, they belong to the enacted paradigm, as they both explain continuous, dynamic interaction processes. The related but subtly different concepts of interbodily resonance (Froese and Fuchs, 2012) and mutual incorporation (Fuchs and De Jaegher, 2009) describe human interaction through bodily interplay of human interactors. Following an overview of these, we apply them to interaction with ExoBuilding.
5.2.1. Interbodily resonance and mutual incorporation
Froese and Fuchs (2012) argue that interbodily resonance between people is a pre-cognitive process in which each bodily expression of one interaction partner equals an impression for the other. The authors further explain that the interaction between two humans occurs in the form of continuous physical micro-adjustments in response to the bodily expressions of an interaction partner: for example, Partner B senses expressions of Partner A through his/her body, resulting in B's physically expressing reactions to the perceived emotional state of Partner A, a form of bodily resonance. This continuously adjusting interaction describes a phenomenological and embodied feedback loop between two individuals. The interaction eventually establishes an autonomous process, which none of the interactors directly control. Froese and Fuchs argue that the interactors feel as though both their bodies are dynamically connected. Thus, both interactors’ bodies extend to and coordinate with each the other, seeking mutual embodiment in a continuous, shared interaction, which also matches Gallagher and Bower's description of the perception–action system in which the organism (interactor) dynamically adjusts to the environment, in this case the other interactor.
In turn, mutual incorporation describes the process of reaching out for shared embodiment. Due to the active, dynamic and continuous nature of this process, mutual incorporation relates to the enacted paradigm of the 4 Es. Mutual incorporation can be observed, for example, during a tennis match, where each player incorporates both the moving ball and the opponent (position, movements and posture) into their own body image. Coordinating the body with ball and opponent then triggers a reaction in the opponent even prior to the ball being hit. This might suggest that the ability to incorporate may improve with experience. Fuchs and De Jaegher (2009) also explain that, for example, faking a shot breaks the established coordination due to the mismatch of anticipated and performed action. This illustrates how mutual incorporation couples the body to its environment consisting, in this case of objects (ball) and other human agents (opponent), as argued by McGann et al. (2013).
While mutual incorporation and interbodily resonance appear very similar, the former tends towards action and acquired skill, while the latter leans towards perception.
5.2.2. Interbodily resonance between human and environment
Having explained the interpersonal enactment process, we now substitute one of the human interaction partners with our prototype (see Fig. 6). First, through the real-time mapping of inhabitant data to the building fabric (biofeedback), ExoBuilding establishes a predictable physiological coupling with the inhabitant very similar to the concept of mutual incorporation. It quite literally incorporates the participant's data into its physical structure and behaviour. Participants incorporate ExoBuilding similar to knowing the distance and relationship to a mirror image of themselves: their behaviour is being reflected through the environment. Given enough time (Trial 1 + Trial 2, A), this interaction evolves to establish ExoBuilding as predictable interaction partner. It also enables the emergence of the autonomous process between the two interactors, which both only control indirectly. Similar to interbodily resonance, both interactors react to small changes in this process through continuous split-second micro-adjustments of their behaviour. Through very subtle but constant behavioural changes in the interaction, one interaction partner, such as a digitally driven kinetic environment can control its inhabitant through an automated behaviour.
To summarize, participants interacted with the environment first through biofeedback control for an extended period (one and a half trials). The slowly established predictability of this interaction, lead to mutual incorporation. Over time, a certain autonomy of the interaction emerged, which meant that the interaction continued without a clear leader. Inhabitant and environment resonated with each other. Once the direction of control unnoticeably reversed so that the environment initiated movement, interbodily resonance continued to function, but reversely: the participant micro-adjusted to environmental behaviour. This explains the results of participants reducing their RSPR as per the transition and not noticing a loss of control.
We can speculate that interacting through interbodily resonance and mutual incorporation is essential when attempting to directly affect a participant's pre-cognitive physiological behaviour through an adaptive environment. Confirming this speculation is a study by Moraveji et al. (2011) who showed such unaware behaviour in a study in which participants adjusted their respiratory rate to a peripheral on-screen feedback stimulus while focusing on their work. An indirect confirmation of workings of interbodily resonance and mutual incorporation can be gained from situations without such feedback-based manipulations in which participants did not change their behaviour. For example, it was observed both in the results of the extended exposure pilot study (reported here) and in Schnädelbach's original study (2012). In the extended exposure study, participants received feedback and no manipulation occurred. This resulted in overall consistent RSPRs. In Schnädelbach's (2012) study, participants experienced regular, automated motion of ExoBuilding without prior biofeedback and transition of control. As a result, no physiological effects (slowing of RSPR) on participants were observed.
5.3. Maintaining control—a cognitive act
A subset of participants did notice the loss of control but aligned their behaviour with ExoBuilding despite perceiving the relationship as fractured. To explain the behaviour of this group of participants, we draw on the psychological model secondary control (Rothbaum et al., 1982). This is in contrast to, what Rothbaum, Weisz and Snyder call primary control (1982), which refers to people seeking and having an affinity towards control. For example, White (1959) highlights a general human motivation to gain control over an environment. Also, having control seems to be beneficial. Lefcourt (1976) explains that external stress is reduced when people have ‘some degree of control’. In our study, participants had lost primary control in Trial 2 (Phase B). The model of secondary control by Rothbaum et al. fits both the above literature (affinity and motivation to control), and our findings of participants reporting less perceived control (Q3) but syncing with ExoBuilding. Rothbaum et al. argue that people have a tendency to align their wishes and expectations with environmental forces in order to maintain the pretence of control over the situation, such as gamblers believing to be able to influence the outcomes of a bet, thus aligning themselves with chance. One participant expressed such a behaviour of alignment:
P27: […] it [ExoBuilding] just kept doing that [being non-responsive but moving steadily]. […] At the end I tried to follow it and I had changed my decision [to ‘break the game’]. But my instinct [was] to break it.
According to Rothbaum et al., then, adjusting to an environment is an act of secondary control on the part of the participant: after losing actual control, the participant makes a conscious decision to align with the environment, thus maintaining the pretence of control. Hence, secondary control is a cognitive, rational process requiring awareness of losing control. Secondary control explains why those participants who perceived a loss of control still aligned their behaviour with that of ExoBuilding. This contradicts our initial expectation that participants who would notice a loss of control would not synchronize their behaviour with ExoBuilding.
5.4. Two paths of control, one destination
Both the pre-cognitive process of interbodily resonance and the intentional process of secondary control equally lead participants to synchronize their behaviour with the changing environment (see Fig. 7). Thus, participants reached the same goal of reduced RSPR despite following different experiential paths. While the experience after the manipulation (M) appeared seamless for participants on the pre-cognitive path, participants on the conscious path experienced a moment of cognitive awareness (C) of the changing condition followed by a conscious decision to synchronize with the environment.
Both interbodily resonance and secondary control contribute a way of understanding interactions with existing prototype environments, especially those that provide continuous interaction in real-time. For example, Muscle Tower II (Oosterhuis and Biloria, 2008) is a metal-frame structure that leans and bends towards passers-by, following their movements as if it were curious about them. Similarly, InteractiveWall (Hosale and Kievid, 2010) flexes in response to approaching people. This new class of real-time adaptive environments particularly resonates with enacted embodiment in architecture, such as interbodily resonance. While such environments are designed to respond, they could also actively engage people in more complex, reciprocal and potentially playful interactions through the concept of interbodily resonance and secondary control, leading people to new behaviours.
Beyond the interpretation of existing adaptive structures, our study also concretely addresses some of the issues and theories of control being currently discussed in the field of adaptive architecture. Specifically, we experimentally showed a form of hybrid control similar to the hybridized control concept elaborated by Sterk (2015), partially letting inhabitants participate in controlling the space, partially automating control over the features of the space and in turn controlling inhabitant behaviour. Thus, our research adds to Sterk's perspective on control: while he focuses on the design of architectural space, we emphasize the interaction mechanism between inhabitant and environment and the effects their interactions have on the inhabitant. Similarly, we add the dimension of experimental research to, for example, Fox's (2015, 2009) work who also emphasizes design regarding interactions with architecture, without specifically and experimentally testing the effects such interactions have on inhabitants of adaptive spaces.
For the remainder of the discussion, we will briefly reflect on the current and future applicability of the findings of this work outside the experimental setting described here. Firstly, the interactions described in this paper seem already suitable for specific contexts like therapy rooms, where patients could experience biofeedback through the architecture itself. They may, then, unconsciously or consciously follow the structure and by doing so assume healthier behaviours. Similarly, environments that are likely to accommodate people who are stressed, anxious or tense could offer spaces that reciprocally interact with their inhabitants. Some of these environments might be airports, hospitals, hospices, care homes and potentially even schools.
Secondly, we already explored the general applicability and agency of an adaptive environment leading its inhabitants to pre-defined behaviours. In the context of group yoga sessions, we showed that such an environment can be a useful guide for physiological behaviour, such as breathing (Moran et al., 2016). We examined how both biofeedback and automated movement of ExoBuilding affected groups of yoga practitioners. We found that when participants chose a movement frequency for the automated mode, they appreciated ExoBuilding as an additional member of the yoga group. Similar to following a human instructor, participants were matching their own behaviour to the rhythm expressed by ExoBuilding. They explained: ‘[…] it makes everything extremely easy, to focus, to concentrate and to improve your practice.’ Thus, ExoBuilding provided an ideal state, which allowed participants to follow a training goal. And illustrating ExoBuilding's role in the yoga session they said: ‘It feels more like a trio than a duet, which is nice [..] [ExoBuilding] being the master, a conductor.’ With this statement, participants attributed significant agency to ExoBuilding as third group member. By accepting ExoBuilding as master or conductor and consciously following the behaviour of ExoBuilding, the yoga practitioners exhibited behaviour similar to secondary control described above. Furthermore, participants thought that interacting with the environment in this way would provide a training effect: ’[…] we were both using that to stretch ourselves a bit. […] I can see this as being a really useful teaching aid or learning aid, in order to learn how to deepen and lengthen the breath.’ This statement also indicates that the reduction of respiratory frequencies as described above is a useful and even desirable environmental behaviour. And although not yet tested in the context of yoga, using a seamless transition from biofeedback to decelerating automated motion appears to be a viable training tool in yoga to increase the training effect of longer and deeper breathing. In summary, the responses of the yoga practitioners indicate that adaptive environments have the potential to directly and positively contribute to and participate in their inhabitants’ activities. It also shows that participants are both aware and appreciative of such contribution and participation by an adaptive environment.
In future, two specific strands of investigation seem most promising, separated by the scale of the interventions. Concretely, we are in the planning phase of installing a desk-mounted version of ExoBuilding in an office environment. This will test both the use of biofeedback architecture in this context as well as the above-described control mechanism during daily office activities and whether this can have positive effects on office worker's breathing patterns. Thus, it will expand Moraveji et al.’s (2011) by exploring peripheral adaptively paced RSP in an immersive three-dimensional environment compared to their desktop interface. This investigation will seek to address the usability of the approach in a more common, day-to-day context. Its scale will mean that multiple adaptive features could co-exist in the same environment (e.g. on multiple desks), even though interaction might remain limited to one-to-one mappings between a person and an adaptive feature.
More long-term plans include explorations of this approach in multi person environments, where multiple adaptive features are linked to multiple co-present people. This clearly presents a challenge, and we have only taken some small steps in that direction in the outlined yoga work. Exploring this space is very timely, as the fundamental infrastructure to enable such interactions has already started emerging around us. As outlined in the ‘Introduction’ section, fitness trackers and health monitors (e.g. the Quantified Self) are becoming widespread, as are sensor and actuator infused buildings (e.g. Adaptive Architecture). Technically, interfacing the two is straightforward, while the non-technical challenges that we see lie ahead are many. They concern at least the continuum of organizational versus individual control, the legibility and ‘scrutinisability’ of both the data collected and the interactions mapped, and the overall interactional aims of such mappings between people and environments in possibly very different contexts (e.g. from homes to offices, to entertainment venues and sports facilities)
We showed that in specific conditions a digitally driven kinetically adaptive environment can manipulate the physiological behaviour of its inhabitants. We argued that the unconscious bodily interaction between inhabitant and environment is very similar to that between two human partners. We modified this concept of interbodily resonance to reflect the human–environment relationship. We also observed participants consciously aligning their behaviour with the environment, which we explained through the cognitive process of secondary control. Applying these models improves our understanding of bodily interactions with kinetically adaptive environments. Methodologically, our findings also substantially expand past controlled studies of biofeedback environments as our study included a deliberate manipulation of the physical environment. Our study adds an experimental, inhabitant-centred perspective to the general discussion of control in adaptive environments and shows the importance of considering the potential effects of reciprocal interactions between adaptive structure and inhabitant. Furthermore, we have shown the viability of applying this research in the context of yoga environments, while we continue to investigate additional application areas and new designs of kinetically adaptive environments.
We thank all participants as well as the reviewers of this paper for their invaluable insights and comments.
This work was supported by EPSRC Grants EP/P505658/1 and EP/M000877/1 as well as the University of Nottingham via the Nottingham Research Fellowship ‘The Built Environment as the Interface to Personal Data’.