Yo-yo tuning; tangible audio

I’ve been thinking about augmented reality and tangible stuff in relation to music recently. A while ago I hacked together a RFID reader and rotary encoder (using cheap-ish off the shelf USB kit from Phidgets) into a virtual knob. Ian captured me giving a quick (“it’s me knob demo!“) demonstration in the office a couple of weeks ago. The idea there was that one rotary encoder could act as more than one controller if it knew it had moved between different positions. In this case, using RFID tags.

More recently, I have been playing with reacTIVision (the software behind the reactable, and incidentally what the SLorpedo team used in their Hack Day entry). It’s incredibly fun and ridiculously easy. To avoid my hands getting in the way of the tags, I threw together a picture frame, a cardboard box, a 25 watt table lamp and a Logitech quickcam (actually, the LEGO Vision Command camera, which is a nasty manual-focus quickcam with nifty LEGO extrusions).

Yo-yo, camera, Altoids tin Side-on Fiducials viewed from below Yo-yo tuning (video thumbnail) The (trivial) setup in Reaper

For tangible objects, I grabbed three things relatively close at hand. Here’s a video of me having fun in C major with a red wooden yo-yo, a tin of Altoids and a hidden surprise.

Having installed reacTIVision, I run it like this:

reacTIVision.exe -m midi\demo.xml –midi-device 10

An argument of -l midi will list all the available MIDI devices. Something like MIDI Yoke is handy here (which is device 10 for me). The MIDI output is optional, and the default OSC output is more flexible, but for today I wanted to play directly with MIDI and this made it really easy.
To use the three different inputs shown in the video, I first described the controls I wanted in the xml configuration file. I just edited demo.xml to include

<map fiducial="0" type="hfader" channel="1" control="1"/>
<map fiducial="0" type="note" note="72"/>
<map fiducial="1" type="knob" channel="1" control="2"/>
<map fiducial="1" type="note" note="76"/>
<map fiducial="2" type="vfader" channel="1" control="3"/>
<map fiducial="2" type="note" note="79"/>

Which, as you’d expect, means…

  • tag 0 (the tea bag) plays a C when visible, as well as treating its vertical position as MIDI controller 1 on channel 1,
  • tag 1 (the yo-yo) plays an E, as well as treating its angle as controller 2, and
  • tag 2 (the Altoids) plays a G, as well as treating its horizontal position as controller 3.

Next, I needed something to handle the MIDI notes and commands. I love Reaper for this kind of thing. (Annotated screenshot which explains a little more of what is going on here). For an instrument, I used the lovely Tapeworm from Tweakbench.

Finally, I trained Reaper (though you might prefer Ableton or whatever) with the controls I planned to manually twiddle. I set controller 2 to affect the fine-tune knob in Tapeworm and controller 1 to change the volume for the track.

The possibilities here seem endless. Throw in a Monome 40h, a couple of Wiimotes, an P5 Glove and the RFID reader / rotary encoder knob I was playing with before, and I have more physical controllers than I could ever need. All of which talk MIDI and/or OSC. Expect more demos (hopefully with some actual music rather than just proof of concepts) as I continue to experiment. I already like reacTIVision a lot, and it makes me want to buy a better camera.


RSS feed for comments on this post.

  1. […] Toutefois et en attendant la suite, une démo sympathique par l’Institut Fatima avec Live (collectif hispanico-germanique) : * 24 août *J’ai lâché le monde du gratuit et l’univers de l’OSC (Open Sound Control) pour celui du payant (en version démo) et du Midi.Je me suis finalement rabattu sur Live et la sortie MIDI de ReacTiVision grâce aux blogs de Èffe et de Roo Reynolds qui ont travaillé aussi sur leurs propres tables maison. Après avoir installé le merveilleux logiciel MidiYoke qui permet de router le flux Midi en interne sans interface physique, j’ai pu – au bout de 24h – trouver la bonne config stable et efficace entre la webcam et LiveTM.Dès que les résultats auront un peu plus d’allure je posterai une petite vidéo de démonstration. […]

    Pingback by Marmuz, le webzine de chuchumuchu » Blog Archive » Reactable : interface tangible pour musique collaborative — August 23, 2007 #

  2. […] Another link for corsair2 – Yo-yo tuning; tangible audio. […]

    Pingback by In High Gear » Blog Archive » Another Barrage — August 24, 2007 #

  3. […] I don’t suppose anyone here reads this, and doesn’t also read roos blog but if you don’t, you’ve just missed some really cool stuff on tangible interfaces, including whiteboard pong and yo-yo tuning. […]

    Pingback by kybernetikos.com » Blog Archive » Shout-Outs — August 25, 2007 #

  4. […] http://rooreynolds.com/2007/08/17/yo-yo-tuning-tangible-audio/ […]

    Pingback by after todays seminAR « SIF90 TangibleMedia 2oo7 — September 17, 2007 #

  5. Hey Roo – this is not only fun – it’s very interesting … especially with all that low-tech stuff ;-) Keep up those creative ideas !! greetz, Tom

    Comment by Tom Sperlich — September 26, 2007 #

  6. Thanks Tom. I appreciate the comment.

    Comment by Roo — September 29, 2007 #

  7. Neato! A classmate and I are using Reactivision for our final and am having a lot of trouble with the camera’s calibration. We’re using a unibrain i-fire and plexiglass, both of which seem to be causing problems. We’re doing it with visuals in Processing instead of Midi. Any advice you could provide would be much appreciated! Again, good work. That’s awesome.

    Comment by Heather R. — December 1, 2007 #

  8. Ok, ive gotten reaper to recognise thw midis and play music, but how fo i get it to react to the turning and sliding????

    hit me up:


    Comment by Tyler — December 2, 2007 #

  9. MIDI notes and contollers can be assigned to all sorts of actions in Reaper. First, make sure the MIDI device you’re using is set up for use as a control input, as well being enabled.

    Options | Preferences | Audio | MIDI Devices

    Right click on the MIDI input device you’re using (in the list in the top half of that screen) and make sure it’s enabled for control messages.

    To assign actions to controllers, go to

    Options | Preferences | General | Keyboard/Control | New…

    Then, with the ‘Shortcut’ field highlighted, [move a MIDI control so a MIDI message is generated and the editor knows which controller (or note) you’re talking about. Then you can pick an action in the ‘Action’ dropdown. For example, “Track: Set volume for track 01”.

    The other area to explore is in Reaper’s FX chain. Almost every action in every plugin can be assigned to a MIDI note or controller. Look for the ‘Learn’ button in the top right of the FX plugin editor. Pick a control for that plugin, and teach it will invite you to teach it which MIDI controller you want to use by changing that control. It should be pretty easy once you get the hang of it, but fiddle with the different settings to see what works for you.

    Comment by Roo — December 2, 2007 #

  10. Ok that worked, thanks.

    Any idea on how to use it wor computer control?

    Volume control, etc?

    Comment by Tyler — December 2, 2007 #

  11. Ok that worked, thanks.


    Any idea on how to use it wor computer control? Volume control, etc?

    Sorry, I don’t know what you mean. If you’re having trouble with REAPER, try the manual. If you’re struggling with reacTIVision, the post at the top of this page is what I did, and it worked for me. After that, the user forum is probably a good place to start.

    Comment by Roo — December 2, 2007 #

  12. […] while back, Roo Reynolds posted about his experiments with Reactivision. Reactivision is a really neat way of building what’re called tangible user […]

    Pingback by Unilever Centre for Molecular Informatics, Cambridge - Brighten the Corners » Blog Archive » Sentinel — January 10, 2008 #

  13. hey Roo

    Cool stuff and great fun. I have all I need now to put a small setup together.However when I invoke reactivision with my midi device the microsoft midi mapper device keeps opening.As you know this is device zero.My device is 19 and midi yoke (which I could also use) is 10.I am invoking with the command >reactivision.exe -m midi\demo.xml -midi-device 19 but only the midi mapper opens.

    Do you have any suggestions?


    Comment by flashfoto — January 21, 2008 #

  14. Not really. If I were you, I’d try it with a lower number, just to see if that’s causing the problem. After that, I’m afraid you’ll need to try the reactivision forums. Good luck with it though.

    Comment by Roo — January 21, 2008 #

  15. […] talked about MIDI, and it’s trendier younger brother OSC, here before. Since these items showing up in ControllerMate, (including Wiimotes via Bluetooth and Guitar Hero […]

    Pingback by Roo Reynolds - Rocking Outside the Xbox — August 5, 2008 #

Sorry, the comment form is closed at this time.

Powered by WordPress with GimpStyle Theme design by Horacio Bella.
The postings on this site are my own and don't necessarily represent my employer's positions, strategies or opinions.