KIMA: "The Wheel" at Curtain Call
As part of a collaboration with the Analema Group, we created an interactive experience for Ron Arad's Curtain Call, an immersive 360 degree screen. The experience was built in Unity as a custom particle system driven by 7 neural networks which interpreted audio input from 7 microphones to visualise an avant-garde choir piece.
The piece draws inspiration from the combination of the space of the Roundhouse and the poetry of Evgenia Emets. Built as a particle system on a vector field, the piece directly maps 6 microphones assigned to the singers of the choir to 6 different neural networks (referred to as brains) that control the characteristics of 6 particle emitters, with a further central network connected to a main microphone in the centre of the circle. The installation consisted of a two-part performance by the choir, with an intermission in which the audience was encouraged to interact directly with each of the microphones.
The visual layer consisted on two Unity 5 applications running on two 7thSense Delta Infinity servers, each creating a 6480 x 1920 pixel texture that was sent via Spout to the main video system of the roundhouse. These textures were mapped onto 12 HD projectors, each spanning 30 degrees of the circular screen.
The visual aesthetic of the installation is heavily inspired by hexagons, which required me to write a custom algorithm to place the forces that made up the vector field in a hexagonal grid.
By coaxing the forces to only take on three directions, we were able to force the particles to follow hexagonal paths, which could then be made more or less dramatic by adjusting the magnitude of the forces.
The field is divided into 6 emitters which had their own settings (force, speed, friction, magnitude) which allowed to create the conceptual idea of 6 worlds. While initially we were building mappings manually on the unity layer, it quickly became apparent that for higher expressivity we would have to search for a better method.
I had recently came across wekinator, a tool created by Rebeca Fiebrink that allows to create expressive interfaces by using machine learning techniques. I suggested we moved to using neural networks to control the emitters, due to the increased expressivity, and the ability of training the networks to respond on different singing patterns.
Each network was thus trained to analyse OSC input from an audio analysis application and respond differently to different sound stimuli. We were able to easily train certain "worlds" to respond to tonal or fricative sounds in a really intuitive manner, mostly by setting the output that we wanted from the network, and then providing it with enough test data for the network to be trained. This was done in a most natural way, by speaking onto a microphone.
The usage of these networks was vital in creating a compelling and expressive interface between the artists and the visuals. Moreover, it became a sort of visual instrument, responding instantly to the choir's voices, and enabling the artistic vision to be easily realised. By not having to build the mappings directly, non-technical members of the group were able to independently adjust and explore the behaviour of each world.
KIMA was one of the most ambitious projects I've taken part in. We developed the entire system in about a month, with minimal testing time on the final hardware. This, together with the complexity of the setup and the live nature of the installation (which lasted for a really short time) put a lot of pressure on the team. However, the project was a resounding success, and as a testament we will be showing permanent version of the piece at Fondation L'Abri as part of a residence in November 2016
I wish to thank everyone in the AnalemaGroup for the opportunity to work side by side with you guys!