Yo-yo tuning; tangible audio

I’ve been thinking about augmented reality and tangible stuff in relation to music recently. A while ago I hacked together a RFID reader and rotary encoder (using cheap-ish off the shelf USB kit from Phidgets) into a virtual knob. Ian captured me giving a quick (“it’s me knob demo!“) demonstration in the office a couple of weeks ago. The idea there was that one rotary encoder could act as more than one controller if it knew it had moved between different positions. In this case, using RFID tags.

More recently, I have been playing with reacTIVision (the software behind the reactable, and incidentally what the SLorpedo team used in their Hack Day entry). It’s incredibly fun and ridiculously easy. To avoid my hands getting in the way of the tags, I threw together a picture frame, a cardboard box, a 25 watt table lamp and a Logitech quickcam (actually, the LEGO Vision Command camera, which is a nasty manual-focus quickcam with nifty LEGO extrusions).

Yo-yo, camera, Altoids tin Side-on Fiducials viewed from below Yo-yo tuning (video thumbnail) The (trivial) setup in Reaper

For tangible objects, I grabbed three things relatively close at hand. Here’s a video of me having fun in C major with a red wooden yo-yo, a tin of Altoids and a hidden surprise.

Having installed reacTIVision, I run it like this:

reacTIVision.exe -m midi\demo.xml –midi-device 10

An argument of -l midi will list all the available MIDI devices. Something like MIDI Yoke is handy here (which is device 10 for me). The MIDI output is optional, and the default OSC output is more flexible, but for today I wanted to play directly with MIDI and this made it really easy.
To use the three different inputs shown in the video, I first described the controls I wanted in the xml configuration file. I just edited demo.xml to include

<map fiducial="0" type="hfader" channel="1" control="1"/>
<map fiducial="0" type="note" note="72"/>
<map fiducial="1" type="knob" channel="1" control="2"/>
<map fiducial="1" type="note" note="76"/>
<map fiducial="2" type="vfader" channel="1" control="3"/>
<map fiducial="2" type="note" note="79"/>

Which, as you’d expect, means…

  • tag 0 (the tea bag) plays a C when visible, as well as treating its vertical position as MIDI controller 1 on channel 1,
  • tag 1 (the yo-yo) plays an E, as well as treating its angle as controller 2, and
  • tag 2 (the Altoids) plays a G, as well as treating its horizontal position as controller 3.

Next, I needed something to handle the MIDI notes and commands. I love Reaper for this kind of thing. (Annotated screenshot which explains a little more of what is going on here). For an instrument, I used the lovely Tapeworm from Tweakbench.

Finally, I trained Reaper (though you might prefer Ableton or whatever) with the controls I planned to manually twiddle. I set controller 2 to affect the fine-tune knob in Tapeworm and controller 1 to change the volume for the track.

The possibilities here seem endless. Throw in a Monome 40h, a couple of Wiimotes, an P5 Glove and the RFID reader / rotary encoder knob I was playing with before, and I have more physical controllers than I could ever need. All of which talk MIDI and/or OSC. Expect more demos (hopefully with some actual music rather than just proof of concepts) as I continue to experiment. I already like reacTIVision a lot, and it makes me want to buy a better camera.

15 replies on “Yo-yo tuning; tangible audio”

  1. Hey Roo – this is not only fun – it’s very interesting … especially with all that low-tech stuff ;-) Keep up those creative ideas !! greetz, Tom

  2. Neato! A classmate and I are using Reactivision for our final and am having a lot of trouble with the camera’s calibration. We’re using a unibrain i-fire and plexiglass, both of which seem to be causing problems. We’re doing it with visuals in Processing instead of Midi. Any advice you could provide would be much appreciated! Again, good work. That’s awesome.

  3. Ok, ive gotten reaper to recognise thw midis and play music, but how fo i get it to react to the turning and sliding????

    hit me up:


  4. MIDI notes and contollers can be assigned to all sorts of actions in Reaper. First, make sure the MIDI device you’re using is set up for use as a control input, as well being enabled.

    Options | Preferences | Audio | MIDI Devices

    Right click on the MIDI input device you’re using (in the list in the top half of that screen) and make sure it’s enabled for control messages.

    To assign actions to controllers, go to

    Options | Preferences | General | Keyboard/Control | New…

    Then, with the ‘Shortcut’ field highlighted, [move a MIDI control so a MIDI message is generated and the editor knows which controller (or note) you’re talking about. Then you can pick an action in the ‘Action’ dropdown. For example, “Track: Set volume for track 01”.

    The other area to explore is in Reaper’s FX chain. Almost every action in every plugin can be assigned to a MIDI note or controller. Look for the ‘Learn’ button in the top right of the FX plugin editor. Pick a control for that plugin, and teach it will invite you to teach it which MIDI controller you want to use by changing that control. It should be pretty easy once you get the hang of it, but fiddle with the different settings to see what works for you.

  5. Ok that worked, thanks.


    Any idea on how to use it wor computer control? Volume control, etc?

    Sorry, I don’t know what you mean. If you’re having trouble with REAPER, try the manual. If you’re struggling with reacTIVision, the post at the top of this page is what I did, and it worked for me. After that, the user forum is probably a good place to start.

  6. hey Roo

    Cool stuff and great fun. I have all I need now to put a small setup together.However when I invoke reactivision with my midi device the microsoft midi mapper device keeps opening.As you know this is device zero.My device is 19 and midi yoke (which I could also use) is 10.I am invoking with the command >reactivision.exe -m midi\demo.xml -midi-device 19 but only the midi mapper opens.

    Do you have any suggestions?


  7. Not really. If I were you, I’d try it with a lower number, just to see if that’s causing the problem. After that, I’m afraid you’ll need to try the reactivision forums. Good luck with it though.

Comments are closed.