VJ System – Vocal Joystick

The University of Washington is working on an amazing voice recognition system called VJ which stands for Vocal Joystick. No it doesn’t do text to speech like everyone else is doing, it allows simple vocal sounds control your computer mouse. After watching the videos it looks much easier than the other technologies I have seen in the past.

More videos after the jump.

“The goal of this project is to develop a novel system that we call the Vocal Joystick (VJ). This device will enable individuals with motor impairments to use vocal parameters to control objects on a computer screen and ultimately electro-mechanical instruments.

The VJ system consists of three main components: acoustic signal processing, pattern recognition and motion control. First, the signal processing module extracts short-term acoustic features, such as energy, autocorrelation coefficients, linear prediction coeffients and mel frequency cepstral coefficients (MFCC). Signal conditioning and analysis techniques are needed for accurate estimation of these features. Next, these features are piped into the pattern recognition module, where energy smoothing, pitch and formant tracking, vowel classification and discrete sound recognition take place. This stage involves statistical learning techniques such as neural networks and dynamic Bayesian networks. Finally, energy, pitch, vowel quality and discrete sound become acoustic parameters to be transformed into direction, speed and other motion related parameters. The application driver takes the motion control parameters and launches corresponding actions.”

Thanks to Michelle from Investor Spot for the tip.