For the new game I am developing, I am creating a played soundscape and AV experience, which allows the player to abstractly conduct the sounds they are listening to, through movement, interaction and exploration. I have been making adventurous instruments for sound and visualization before and I really wanted to make a more playful and interactive experience. I want to portray a speculative approach to sound making and AV experience, encouraging the viewer no matter their musical ability or knowledge to form and compose sound in new and experimental ways.
I want to make the piece procedural and random to produce an experience that can reform and recompose itself for every performance, meaning that the experience is only felt by the user. In some ways referencing my interest of speculative realism and the Anthropocene, reflecting the way we see only our projected surface of phenomena and never a true depiction, by itself ever changing form and intra-active composition.
For the interface I decided to develop an experience for Leap Motion, a hand tracking hardware that allows the user to interact in virtual space with their hands. As a new and not hugely distributed hardware, I wanted to use the possible naivety of the player to create new and adventurous virtual spaces, somewhere they felt they could test the possibilities and not guess the result.
Outputs will be ambisonic audio (surround sound replicating virtual space) and a screen or projection displaying the visuals and play.
You are placed in outer space, a cosmos of possibility in front of you. Sounds call from the planets surrounding you in the void of space. You guide yourself through space with your hands, directing your movements through fluid gestures. As you turn you hear the sound turn around you, as you move towards the sound, you make out the form of a planet and its orbiting moons.
As you finally get to the planet its sound crescendos into center stage and it fills the screen, you reach out and touch it, and it moves and rotates. You reach out and touch one of its moons, grasp it in your hand, as you do the sound starts to change. Each discovery pushing you on to try new interactions and compose new soundscapes through play.
There is no real end point just when the person has explored enough and may come back again another time as it will change again and again.
Move around with your hands, in a similar fashion to flying or a 3D joystick, investigation is the primary driver. Search with your eyes and ears to sense the next oasis in the void, using ambisonic sound and good graphics to lead you around. Using abstract interfaces like sound planets and modulating moons to allow the user to play with the sound in new fun ways. Create some other visualizers and sound makers to vary and open the experience.
Potentially allow the player to add their own samples/sounds.