updated iphone gestural system December 3, 2010Posted by onyxashanti in Uncategorized.
This is one of those really REALLY geeky posts. forewarned…
My gestural system has reached a milestone in beatjazz technical development. i’ve updated my iphone control system to do much more than before. the system is MAD unstable, but that never stopped me before. yaaaaay!!!
In the beginning, all of 3 1/2 years ago, after years of playing clubs, i had a few requirements for my next phase as an artist. no footpedals, dynamic soundsets and the ability to control parameters easily and wirelessly using a handheld device and wifi or bluetooth. in the old days, back in 2005, i used an HP ipaq running a terminal server wirelessly from my laptop, located in the DJ booth. it wasn’t very elegant but it was very freeing. no need to have to run back to the DJ booth to stomp some footpedal or screw with the trackpad on the laptop. i could just do it from the ipaq from anywhere in the club. my soundset was loaded and i could select different sounds by program changes sent from my wind controller.
it was at this time,while living in NYC that i also discovered max/msp by attending the SHARE parties in the east village. although i didn’t really need this type of software at the time, i knew that it was going to be the future. eventually i settled on its open source sibling, Pure Data and set about a 2 year journey to understand what it was, how it worked and why i would even want to use it. and it was about a year and a half ago after adapting a gamepad controller to “play” my FX in performance, that i started to realize how this software could allow me to add very personal functionality to my system.
since that time, i shifted my focus from the game controller to the iphone for two reasons; an app called TouchOSC and its ability to allow me to easily make use of its accelerometer. and accelerometer is the sensor in iphones that tell it what direction you are holding it in. for me, a light went off in my head! I could control things without having to mouse-around on the laptop! so last winter i holed up in my flat in Berlin and set about making use of the combination or the touch screen as controller interface and accelerometer as dynamic controller. most of my music from the last 8-9 months showcases that work. as a concept, using accelerometers attached to one body to control computer functions is called “gestural control” because it make use of bodily “gestures” to manipulate computer data in a fluid manner.
in that first iteration, i was able to use directions to control functions. left to right is the x-axis. tilting the hand front/down up/back is the y-axis and turning the hand upside down is the z-axis. so i had 3 axi to play with, in addition to customizable touch screen knobs and sliders. so i create a simple means of controlling volume and patch selection with the touch screen and FX with the accelerometer. the results were much more fluid and interesting than i had anticipated. so much so that i wont perform without my iphone anymore (at least not until i finish construction of my custom controller).
this system has accompanied me to all of my gigs this year and it is because of this intimate relationship with the system that i started to see the weaknesses.
- the accelerometer was underutilized. way too much “touching”. i wanted a means of controlling and adjusting some parameters without having to constantly tapping the back of my right hand, where the iphone is located.
- i wanted to use the power of pure data to make the wind controller more powerful by giving it the ability to control things as well,instead of just sending musical midi data.
- push the boundaries of the gestural concept by controlling many more parameters from the accelerometer and using midi and touchOSC to determine which parameters got edited and when.
- create the first in a series of gestural sound design templates. to be able to not have to use the same sounds but to design the sounds to be dynamically controlled by accelerometer data.
So upon returning from my summer sojourn i set about working on what this system might be like and i just finished the first prototype of that system last night.
the most important part of the system is what i have termed as the B-A-G system. up until just recently, my usage of pure data was as the interpretation layer between my iphone and fl studio, my synth rack. my wind controller worked as normal as a midi controller playing the synths and nothing else. now, i have managed to shoe-horn pure data in between the wind controller and flstudio as well. this is a very big deal and it will define everything i do from this point forward in my music and career. now pure data interprets everything i play before it goes anywhere so my wind controller is a very malleable number stream generator. each note being a number, some notes can now be buttons and functions and switches. B-A-G stands for the top 3 notes on a woodwind instrument…B,A and G. so now i can play a high B and for as long as i blow into the horn i can use the accelerometer to send data somewhere ( and remember that the accelerometer has 3 streams of data), then again with the A key and again with the G key. so that is 9 streams of control data! BUT,wait….theres more!!
now each instrument in my system has its own dedicated midi channel which means that each channel has its own B-A-G functionality including a new FX channel just for playing the FX like any other instrument! so this is an order of magnitude greater control than i’ve ever had. now,for instance, i hold a note and turn my hand as if i am turning a knob for volume, and adjust my volume. or assign synth parameters to certain axi and design a completely original sound from almost scratch,onstage, without even glancing at the computer!
and finally…FINALLY I have added a chord creation system that will allow me to create rather than simply play chords! its simply a breath controlled midi sustain, but it is very powerful within this context.
it seems that it would be unwieldy, but quite the contrary. the parameters i programmed are systemized so they are memorizable. such as that high B/x-axis is always volume, no matter what channel,for instance. and I have made intensive use of many Native Instruments and fl studio plugins that utilize built-in x-y control systems! how perfect is that!?!? and it just so happened that N.I. updated their FX suite with DJ FX so i’m pretty much in heaven right now on the sound/fx front.
why? Beatjazz is my answer to the atomization of genres in music culture. today, as soon as a genre establishes itself, it is atomized and disintegerated into 1000 sub-genres and each of those goes thru the same process. the only way to survive in this world is be genre-less. to exist in a malleable state where you are -using the definition for quantum non-location- here AND there simultaneously. to be rock AND hip hop AND dubstep without being any or them. i theorize that a dynamic gestural computer based improvisational system achieves this and if it doesn’t, its a step in that direction and someone else can pick up from where i leave off. the next step is…well i don’t want to say but lets just say that its everything above without ANY pre-created sounds. totally in the moment sound and music creation,but you’ll have to wait a few months of rmore details on that one 😉
“is your system stable?” um…no. in fact, its is wildly unstable. ” are you going to perform with it?” Hell yes!! I will be debuting the system and the new sound designs at my show tonight, Dec 3rd @16doors (reutestr 47, Neukoelln-Berlin) the system is seriously “alpha” and i have to hit a midi panic button every 2-3 minutes or the midi data clogs up the pathways and freaks out, but hey…that never stopped me before. Pure data doesn’t like my wind controller right now but in time they will learn to love each other. and i’m pretty sure that remembering all those gestures will make my frontal lobe implode but i LOVE this shit! if its too unstable, i’ve got a really nice alternative beatjazz soundset with new dubstep/8-bit sounds. but the first 20-30 minutes is dedicated to pure experimentation,warts and all. I will film it for those of you who can’t make it. for those who can make it. i hope to see you tonight. cheers!