Virtual Drums via Real-Time Stereovision 3D Tracking

Matthieu Aubry, Julien Rouvière and Xavier Maurice have developed a drum system that uses two cameras to track drum sticks in real-time. This allows the player to drum away on virtual drums, the computer analyzes what the player is hitting in the virtual world and plays the corresponding sounds.

Video after the jump.

“Video acquisition of the 2 cameras simultaneously, at a frequency of 30Hz. At this step, image data are huge: 640 * 480 * 3 bytes * 2 cameras * 30Hz = 55Mb/s transferred on the firewire bus! Because we have the 2d positions of the sticks on the screen, and because we know the position of the cameras in 3 dimensions, we can determine via geometrical algorithms the 3d positions of the sticks. Using a positioning algorithm, we determine if the stick is hitting a drum element. We then compute the sound localization and the volume, depending on the speed.”

 

9 Comments


  1. Oops dead link (may be just the server temporarily down)
    This guy, Johnny Chung Lee (http://procrastineering.blogspot.com/) has been doing some amazing 3d tracking work using the Wii Remote. Using the Wiimote would have saved these guys a whole lot of effort.


  2. Cool. I really like that. almost makes me want to make one… 🙁




Comments are closed.