Microsoft Karaoke Technology Learns, Enunciates Language

Microsoft is hoping to encourage Chinese web users to try the Bing dictionary with an added feature that appeals to the country's karaoke fans.

Karaoke (now commonly referred to as "KTV") is one of the more preferred activities in China, with KTV clubs a staple on many street corners in the Far East. With the new "KTV function", Microsoft is hoping to introduce cutting-edge technology to the core principles of karaoke in making the experience of learning a new language fun for all parties.

Microsoft Technology Artificially Maps Movements

The feature works by first extracting sample sentences from the online dictionary and displaying accompanying videos with a human model that repeats the sentences for pronunciation purposes.

But rather than having the same model repeat different sentences over and over again, Microsoft has found a way to synthesize the sounds and artificially mouth the facial movements of the model speaking in the video. In doing so, the language learner is able to see where their mouth movements should be, in addition to enunciating each word much more effectively. (Source: yahoo.com)

Computer Program "Says" Sentence without Uttering a Word

The process behind the KTV function is a bit complex. First, images are captured of all the different pronunciations the human model can make in a five-hour window. In that time, the new technology focuses on lip movement and finds the best match between the lips and what word the programmer wants the model to say.

The end result is a video of the model mouthing words of the sample sentence while a computerized voice reads the words aloud. Therefore, the model "says" the sentence with her mouth without ever having uttered the actual sentence in the first place. (Source: pcworld.com)

The dictionary currently holds over 10 million sample sentences with video accompaniment.

With more than 1 million page views a day, Microsoft should be pleased with their early success. The company is already planning to create a similar Japanese-to-English dictionary sometime in the future.