Kinect Interactive Installation
Installation mixing sound with visuals gathered from an Xbox Kinect to be used as a marketing campaign for high-end speakers and TVs.
Client: Bang & Olufsen (hypothetical)
Scope: Kinect Installation
Role: Concept, Design, Implementation
The Challenge
Creating a unique marketing campaign based on interaction and graphics for a company to show off their sound system and TVs and interact with potential customers. It needed to be engaging, cutting edge, and have a takeaway that users could then share on a social media platform.
The Solution
Using an Xbox Kinect, Synapse, Quartz Composer and Ableton Live, I programmed different body parts to respond to musical elements which would influence an image on the screen. Some elements respond to volume, others respond to movement. The unique images created by the user could then be shared on social media.
Tools
Xbox Kinect
The Kinect has an infrared camera and depth sensor so a users motions and position can be detected.
Synapse
Synapse can send joint information from the Kinect via OSC calls to various different programs.
Ableton Live
Ableton allows for musical composition and arrangement. Using Midi music clips, I was able to manipulate pitch, reverb, songs and anything in between. This was the audio portion.
Quartz Composer
Every joint movement was able to be mapped to display something. I could also map images that responded to sound. This was the visual portion of the installation.
The Setup
For the proof of concept, I setup a laptop running Quartz Composer and Ableton Live. The Kinect, in conjunction with Synapse, would send the joint data to the computer and see which ones were being called by Quartz and Ableton and display or play the images and sounds. A projector displayed the images on a wall of white boxes while a speaker system connected to the computer would play the sounds.
Joint Mapping
The breakdown of the joints were as follows:
Head: Nodding head would create cymbal sounds. Legs: Jumping would play a MIDI snippet of a song. Left arm: Moving the arm up or down would change pitch. Right arm: Moving arm up or down would create more or less reverb.
Visuals
Using a rutt etra patch, the users body would distort a series of lines to create an appealing and unique pattern. There were dots running down the left side of the screen that responded to decibels of the songs created by the different body parts.
Installation
Using a wall of white cardboard boxes, I was able to project the images onto them with a projector. The users would then stand in front of the Kinect and computer to see how it was all working.
The Campaign
The use of images and sound in a campaign made sense for a company that sells speakers and TVs. The best way to showcase their merchandise is a window display where potential customers can use the installation to test their equipment.
Every user creates a unique design with their body movements and sounds. This becomes an identifier only a single person can have at any time. The idea of such a unique image created would drive social media content sharing and create more enthusuiasm for the company.
Conclusion
The users that tested thereof of concept all loved it. They suggested being able to tag team with a friend would be a great next step. In the end, I was proud of the project and felt it could very much become a real campaign in today's age where people are craving more and more unique interactions.