protfood

    The last time my aunt Judy visited Chicago, she reminded me of the blisters I acquired on my thumbs from playing Ikari Warriors on the NES. I also spent about five times the magnitude of time becoming muscle memory monster at Street Fighter II on SNES than learning to play piano on my velocity insensitive Casio CT-640. I doubt I could actually contend on any meaningful regional level besides my basement, but the hours were put in, and controllers like the Atari joystick, NES pad and Advantage, SNES pad, and Xbox pad have become extensions of me, much more so than the fluetophone, piano, clarinet or guitar.

With my Kaoss Pad collecting dust and my Photon X25 phasing-out, as the vast majority of my sequencing and musical articulation is done on a 1984 IBM clicky with very little mouse usage, I was obliged to find a more inviting and available way to manipulate sound parameters; Something always ready-to-go, that I can spontaneously grab and use without altering my physical position or mental zone I'm in; A midi controller with no excuse not to use that becomes part of me, and not part of a desk, or a desktop itself, covered in Cheeto particulate.
    The multitude of >$200, sometimes >$1000 pieces of gear I've gone through attempting to become better in-touch with my sound is slightly staggering, and how little I ever put them to use, embarrassing. One looks at these things and thinks, "Wow, I think these things look like they should be making some interesting sounds!" But looks deceive, and the lack of sound produced signifies the need of something different.
    The first soundcard I ever owned was a Gravis Ultrasound (GUS) MAX, also a trusted maker of fine computer game controllers, and manufacturer of my ten button usb GamePad Pro, purchased to make better use of NES, Atari, and SNES emulators. Performing some speed-runs of muscle-memory-monster-maiming as I kept an ear out for interesting 4-bit percussion sounds to sample, I decided that I should absolutely be able to use this controller as a midi drum trigger in Ableton Live.
    Googling the idea brought up some joystick related apps I'd found in the past, like the softsynth x-crusher live, and eventually I found a discussion on Create Digital Music with a number of enthusiasts posting their findings on the same matter. Testing out a bunch of shareware, joystick-to-midi translation applications, the contenders became Rejoice, and (some japanese thing). (some japaneze thing) was designed to support musically oriented macros, and allowed one to get quite deep into implementing combinations, as well as importing an image of one's controller to map graphically. I spent a lot of time on this trying to get it to work logically, and look cute. One of it's downfalls is that it would often crash; also that it wasn't free. It was a time and save limited crashing demo. Oops.
   Rejoice is quite pure in functionality. You simply assign a game controller input to a midi note or control command, set it to function like a toggle or button, as well as setting incremental ranges of CC data per press. It also has flexibility in its timing, so as not to overload the midi throughput, dead-zone, to make it easier to configure touchy analog controls, and some combo capability.
    Also necessary to get the midi generated by Rejoice or most any other joystick-to-midi application is a midi host like MIDI-Yoke, or my favorite LoopBe1. I've now purchased the full LoopBe30. These applications let you route the midi out of any application to the midi-in of any other application. LoopeBe has the added advantage of throttling the data as well. Without this, many analog game controllers can wind up sending far more data in a short period of time than the midi buffers, at least in Windows XP, can handle.
    Combining Rejoice with LoopeBe1, and feeding the midi into Ableton, I was recording live drum sequences with my gamepad in just a few minutes, simply assigning note triggers to each button. I don't really mind mousing the velocities later on.

Here is a brief how-to which I will elaborate and hyperbolize on once I figure out what needs fleshing-out:

For Windows, you will need the following:

- puredata - http://puredata.info/downloads
The Vanilla or Extended package should be fine.

- pd_joystick by Joseph Sarlo - http://crca.ucsd.edu/~jsarlo/pd/
This external is what I have been using to retrieve game device data for midi translation. There are other ways to perform similar, such as GriPd , but I prefer the more minimal approach with pd_joystick. It also comes with some example .pd files to get you started.

- LoopBe1 - http://www.nerds.de/en/loopbe1.html
This is needed to provide a virtual midi device to route midi from one application (puredata) to another (Abelton Live). After installing, a device "Internal MIDI 'in/out' '#'" will appear which can be selected in any audio/midi application. In puredata you would select the midi out as "Internal MIDI out", and in Ableton Live select "Internal MIDI in". Now any midi puredata generates will be sent to Ableton Live.
I also recommend paying for loopbe30 as it has an overload protection feature. Otherwise, without lowing the input rate in puredata, analog inputs might inundate Window's midi buffer with too much data and make it nervous.

For those running Windows who want to inspect all the MIDI data going in and out, you can monitor your midi ports with MIDI-OX. I used this for some troubleshooting when I was using Rejoice.

These are all the tools you will need to get started. In part two I will get into more details with using/configuring puredata, and more details about what I have implemented.


Also, the Chicago Reader recently interviewed me about this topic. Here is a link to the article entitled "New Ways to Play".   http://www.chicagoreader.com/features/stories/sharpdarts/080417/

And, for your enjoyment, here is the original unedited interview:

MR: Tell me about the gaming interface you use to play music. What's it made up of? How does it work? How long have you been using it? Did you design it?
 

P: I am using Pure Data, similar to Max/MSP, to translate the game controller input to MIDI data. There are some shareware apps out there that already do this, which were what I started off using, but I found them limiting. Initially I was using an application called Rejoice which allowed relatively simple mapping of game controller data to midi data. The limitations in this were what inspired me to investigate puredata, and it was what I used as a basis to start making my own application in Puredata, which made it easy to keep objectives clear. Once I had the basic functionality working, it became easy to augment. As far as hardware goes, all of the controllers I've used have been USB or easily convertible to USB. I haven't needed to solder connections, worry about voltages or any of that malarkey - yet.
    The controllers are used both in composition and performance. In composition they are used to generate and resample sounds I use for sequencing. For performance they are used to manipulate sounds, trigger loops, and manipulate effects in Ableton Live. Its like having a dozen Korg Kaoss pads in my hands at once. I can even do things like use the slect/start buttons to flip between pages of "virtual controllers".
    Starting about a week ago, I am doing everything under linux; tracking, composing, puredata, controllers, editing, mastering, etc. This means no more Ableton. Pure Data gives me an entirely empty canvas whenever I want though. Since high school I have been sequencing with amiga-style DOS trackers as the basis for 90% of my composition. Since I am ditching Ableton Live, I am now focusing on using a midi-capable tracker. Perviously using Ableton Live for live performances, combined with my laziness, restricted me to keeping the functionality of the game controllers pretty static during a performance, but with the new tracker I am learning, every module will have it's own unique set of parameters to manipulate upon being loaded. Each piece of music will in-effect, be considered its own "game" I get to play and will be able to learn to get better at, as these tracks will also be much more abstract at their base, forcing me to learn to play them to get what I want out of them. This will also be the first time I've had everything set up the way I wanted on a single ultraportable linux laptop without having to use an external soundcard or any other cumbersome gear. Carrying a game controller in my bag, I'll be able to make tracks and perform totally untethered with very little difference between the act of composing and the act of performing.
   

MR: About the "better at Street Fighter than the piano" thing you said last night: do you think different approaches to instrument interfaces might encourage people without any training on traditional instruments to make music anyway? How has it affected the way you make music? What about things like Guitar Hero, which let users sort of almost play music but not quite?
 

P: Guitar hero? Guitar zero! Last summer I was given the opportunity to give a presentation at the Classic Gaming Convention up in Oconomowoc, WI for a bunch of kids and parents. I showed them how I could with the controller, take a very simple beat, and apply some methods with effects and game controllers to make some much more oblique and exciting sounds.
Most computer musicians I know employ a piano based midi controller which also employs a few knobs, pads, sliders, etc. I've probably gone through a couple thousand dollars worth of these trying to find something that encourages me to use it and make more music with it, but they always wind up only looking quite while collecting dust and not sounding musical. When I do try to employ them, I can never really do what I want. Everything is spaced so far apart, you can maybe control two or three parameters simultaneously depending on the interface, and they often require such exaggerated, sloppy motions to get anything interesting done with them; using them becomes gratuitous, and good luck biking around with one comfortably. Game controllers are designed to maximize ergonomics and perform with great immediacy. Punch now! Kick now! Tweak now! Transpose now! Throw that snare drum into the delay chain now! Swap between your choice of random toy instrument samples now!
    My sentiment is that I spent easily a thousand, if not several thousand times the amount of time improving my video game skills than practicing the Casio piano my family bought me when I decided to take piano lessons. With an Xbox 360 controller next to my laptop, and all the conditioning I've put myself through, it entices to grab hold and start tweaking sounds unlike any piano based midi controller.
    Many people I talk to about it are interested in the kitsch factor of using vintage/retro game controllers. Recycling is great, but these are usually pretty low on functionality. I'm more interested in the cyborg aspect of it.  The more buttons and axis within finger's reach, the better; at least for the type of music I am currently interested in making. Cyborg, cyborg cyborg.
    For tracking, btw, my piano is about two octaves: zsxdcvgbhnjm q2w3er5t6y7ui9o0p.
    I can't say that I am much for Guitar Hero, or Rock Band. I read about a hundred blog posts a day about people hacking their controllers to output midi. It's sort of like putting duct tape on a flip-flop to make it more like a shoe, but midi, programming, software, synths, sequencers, and practice aside; if someone handed you an NES game pad where every button and combination of controls triggered a crazy sound you could then manipulate further with the d-pad (hat) while keeping tempo and applying logical variations on a power-pad, who would say no?
   
 

MR: What other different interfaces have you come across that you find interesting?

P: I've got a Wii now, too, but I haven't yet implemented the Wii controller and nunchuk. I've seen a lot of news about people DJing and performing with those. A DJ application called Deckadance supports the Wii controller natively now. I have some plans for the Wii, but for now I really like having six axis and fifteen buttons at my disposal rather than waving and gesturing into the air. I've also been implemeting a Dance Dance Revolution pad during live performances to make myself more of a spectacle.
Next on my list, since I already have game controller to midi translation globally cracked, is to build my own game controller using some gaming hardware modules. In this instance I will be designing my own interface with pots, sliders, buttons, axis options, etc. I just need to decided on an interesting format and chassis. Wireless controllers are great as I currently have the wireless xbox 360 controller as my favorite to use for music.
    My friend Ipaghost has been using a Nintendo DS and it's touchscreen as a midi controller which is pretty rad. I appreciate interfaces that are the most portable, available, usable, and familiar. Also, Pure Data and the way I am implementing it allows one to translate ANY kind of data into midi. I could plug the thing into the OBD port of my car and use it to create generative dance music. I don't really see much purpose in that though. I'd probably crash.

MR:  Do you think that some sort of instrument that uses a non-traditional playing method could become as widespread as, if not the electric guitar, maybe the synthesizer keyboard? 

P: The synth-keyboard is just an piano one can augment the sound of. A lot of people play the highly ubiquitous rubber-band whenever given a chance at the workplace. That Beamz laser-harp-abomination has been moreover rejected and mocked by the mainstream as well as the underground. My friend Right-eye-Rita's sweated-double-trumpet is a bit cost prohibitive. I think the Korg Kaossilator is quite interesting as a potable music machine, though maybe a bit constraining musically. I'm sure they'll sell tens of thousands. I think my answer is no.

Also, my freind ipaghost showed me this rad youtube video last night where a computer musician uses a steering wheel as a very spectacular midi controller:

Younnat - Wheel Experience

http://www.youtube.com/watch?v=243X056p9-0

+!