Proj17_hero image.jpg

Psychology of interactive experiences 

University of Technology Sydney / Beta Space (Power House Museum)
Dec 06 - May 09

How do people experience interactive installations and what can designers learn from it?

Today we are mostly designing digital products or service experiences - however in the future we will design for emotional break-throughs, mindfulness and creativity. 

Context of the study

In my research, I investigated how people experienced installations (interactive artwork) in an experimental space. Beta-Space in the Power House Museum, in Sydney, provides a public context for artists and researchers to conduct research into artworks that may be at various stages of completion, from early draft to fully functioning work. The room has a large screen on one side, surrounded with speakers, sensor floor pads and cameras. Depending on the installation the designer uses any of those inputs to create sensory experience for the visitors. 
 

Beta-Space in Powerhouse Museum
Beta-Space in Powerhouse Museum
Installation in Beta-Space
Installation in Beta-Space

In a typical scenario, the visitors often walked with curiosity into the room to see the installation. We want them to be actively engaged with the work and sustain this engagement.One of the research challenges was to do systemic observations in a public space  which is essentially an uncontrolled set up  — where families, kids, group of teenagers randomly walk into the installation space. So we limited our research approach to individual sessions. 

During the sessions, we observe the user and video record their experience as they walk in to the space. Then we take the user upstairs to another room, show the video of their episode and ask them recall their memory of the experience - we call that video cued recall. 

The outcome of my research was a model to describe engagement with interactive installations — representing the intentions, expectations and movements of participants who walk into the exhibition space, interact with the work, stay engaged throughout the different experience phases. The model is based on long-term analysis of direct and lateral audience observations and qualitative analyses of verbal reports and interviews — experience data of nearly hundred sessions from ten different artwork installations in Beta Space over a three- year period between 2004–2007.

Examples of installations

Iamascope is an interactive kaleidoscope that creates images triggered by user’s movements in front of a video camera. The kaleidoscope image reflects back a portrait of the user, accompanied by music created by the change and speed of user’s movements. With this installation users get into the flow and the playful experience lasts for few mins, until they start anticipating what will happen next, and then they often easily loose interest. At this stage they may leave the room, and move on to something else in the museum. 

Iamascope by Sid Fels
Iamascope by Sid Fels

Cardimorphologies (by George Khut) is comprised of one large floor-to-ceiling video projection, that is controlled by heart and breath signals from the participant who is seated a few meters way. The core focus with this work was on translating heart and breath information using concentric circles – to create a kinaesthetically compelling but extremely simple visual design that could be experienced in varying states of attention (i.e. with soft gaze), inspired by mandala imagery and tunnel like visuals often described in connection to near-death experiences – vision at a threshold.

Cardiomorphologies by George Khut
Cardiomorphologies by George Khut

Absolute 4.5 (by Ernest Edmonds) is comprised of a large screen with a changing grid of colours accompanied by a complex sound track and controlled by a generative set of rules carried out by a computer. As the audience approaches the screen Absolute 4.5 detects their presence through sensors in the floor. Aspects of the system’s behaviour, such as its rate of change, are influenced by audience behaviour in the space.

Absolute 4.5 (by Ernest Edmonds)
Absolute 4.5 (by Ernest Edmonds)

Behavioural analysis

We collected observational data from experiences of ten different installations in Beta Space over a three- year period. Within all the diversity of the data, I was curious about finding a consistent and descriptive model to explain the interactive behaviour in different contexts, relationships and timelines.

Using the detailed narratives of the users, I was able to code the cognitive components of their experience. We used a video analysis software, where the video is parsed into of 5-15 second segments - where one can assign cognitive codes to what is happening in their experience. As the data accumulates, the software givex us the patterns of movements, interpretations and thinking of the participants. 

CodedBehaviours.jpg

Patterns of Audience Behaviour: Video Analysis

Creative engagement model

At the end of my research I have produced a creative engagement model which defines interactive experience as a transformative dialogue between the audience and the interactive (art) system. Through studying different art systems and audience experiences I have identified interaction modes, phases and states of audience engagement and used these as basic elements of the engagement model.

 

I developed this model through long-term analysis of direct and lateral audience observations and qualitative analyses of audience verbal reports and interviews. We collected observational and qualitative data from audience experiences of ten different artwork installations in Beta Space over a three-year period between November, 2004 and November 2007.

Engagement Phases.jpg

The key to understanding interactive engagement is to learn about participants’ intentions in performing certain actions, how they interpret the outcomes of their actions and make sense of their experience.

Creative Engagement Phases

Four main engagement phases that drive an interactive experience

Here is a summary of the four main engagement phases that I found that drive an interactive experience:
 

1: Uncertain Encounter

There is often uncertainty in the first-time interactions, as the participant has little or no idea about what the installation does — so she interacts without an intention — in an unintended mode.
Here are a few design principles to support “uncertain encounters”

  • Set clear and encouraging expectations: In this phase, it can help to inform participants in a clear and encouraging way but not prescriptive of what is going to happen (not to spoil it).

  • Invite to uncertainty: Consider simulating the initial interaction in an inviting and obvious manner, making the initial interaction easy. For example, it is easier to walk into a room with dynamic lights, rather than a dark room. This may also set the expectation that the installation will engage participants with some dynamic lighting later on.

  • Make it obvious: Provide visible and noticeable outcomes from the unintended initiation and just enough for the participant to realise the interactivity (visual, sound, touch).

2: Anticipation

Early interactions with installations are often about trial and error, learning how the system works, and what it does. Participants often want to learn quickly and feel they are in control of the system and their interactions. Anticipation is not always an end state, there are situations where interactions might still be engaging and lasting for participants; for example, kids might be watching the same video clip over and over without getting bored — just because they purely enjoy it.

Here are the design principles to support “anticipation”

  • Make it intriguing: Try creating situations where the participant initiates an interaction unintentionally and is surprised about the outcome.

  • Stage the interaction: Provide a chance for the audience to anticipate the interaction one stage at a time — rather than puzzling them continuously

  • Facilitate learning in situ: Provide multiple ways of learning the interactivity and the environment — if the interaction cannot be simple and inviting

 

3: Adaptation to the Uncertainty

Once the participants learn the rules of the interactions and predict what is to come, they might not find more meaning or enjoyment in repeating the same interactions. One way to maintain engagement is to introduce ‘unexpected’ changes to the experience, where the participant might be challenged. For example, the system might start to respond in a new and unexpected way to the same input — while interactions still feel familiar, the unpredictability creates curiosity to maintain engagement.


The key is to provide sufficient ‘adaptation’ time for the participant to practice and learn the new rules interactions. If there is a bombardment of feedback and changes in the environment at once, they probably won’t have enough time for adaptation. This is similar to getting frustrated being around an erratic person, one might immediately disengage and avoid any further interaction. However, a mildly unpredictable person might be interesting to explore. As designers, we should consider introducing one change at a time, allowing time for participant’s adaptation, then maybe introduce another one — depending on participants appetite for a challenge.


Here are the design principles to facilitate “adaptation to uncertainty”

  • Allow time for adaptation: Provide sufficient ‘adaptation’ time for the participants to practice the interactivity

  • Provide consistent feedback: Adaptation is easier when there is clear and consistent feedback from the system. When interactions are consistent, participants can maintain focus and adapt to the system easier.

  • Reward participants: Reward them with a sense of achievement or control

 

4: Deeper Understanding

In phase 3 — during the uncertain mode, because the participants’ intentions and expectations are misaligned, they might interpret the interactivity as unconventional or uncomfortable. In order to stay engaged, they need to shift their thinking of the new situation. This shift can, in return, initiate a new way of making sense of their experience.

As the participant starts seeing new ways of interacting, they reach a more complete understanding of the installation and what their relationship is to the work. In this phase, they judge and evaluate at a higher, conceptual level. They may discover an aspect of the interaction, a new meaning or an exchange she was not aware of before.

 

Here are the design principles to facilitate a “deeper understanding”

  • Delay Anticipation: Anticipation sustains engagement until it is perceived as boring. To avoid boredom, you may purposefully delay anticipation however, the key is to understand how long a participant would stay engaged to reach that state.

  • Create a balance for predictability: Switch between a ‘sense of uncertainty’ and ‘sense of control’ to keep participants challenged and interested

 

A summary of what I learned

To understand and decode engagement, designers have to observe and understand participants’ intentions in the context of the interactive system. Once you understand the most common and intriguing intentions, then you may design interactions that support those intentions.


As a designer, you will need to explore the consequences of your design decisions e.g. whether to align or not to align system feedback with the participants’ intentions. You may Introduce ‘unexpected’ changes to the participants’ experience, where they might be challenged. If you go down this path, then you may provide guidance for the ‘unexpected’ situations or continue to challenge participants with uncertainty — especially when participants seek closure.


I’ve found in many cases, as participants continue to think about the interaction episodes they tend to appreciate their experience more. The feeling is similar to thinking about an event in a movie days after seeing it and recognising patterns in life that were not obvious to you before seeing the movie. Similarly, a deeper understanding phase occurs if the interactive experience triggers new conceptual layers and realisations into the future.


Hopefully, the engagement phases and design principles may guide the artists and designers who have the ambition to design for engaging, influential and creative experiences.