Skip to main content

Sign language recognition for Kinect?

Unearthed Microsoft patent also reveals motion controller may lip-read and map toe movements

Microsoft's original patent for Kinect suggests the Xbox add-on can recognise and translate sign language.

The Gesture Keyboarding patent (as found by Slashgear) includes a clause covering technology that can read American Sign Language (ASL).

"Each gestured character is converted to its spoken equivalent or text equivalent," the filing reads.

While the 2009 patent does not specifically name Kinect (or its previous codename, Natal), it clearly reflects the motion control device's capabilities.

The ASL clause provides an example of how such a system might work. "For instance, where the user is playing an online multiplayer video game, such as a first person shooter, the game may also support voice chat. Where the user is unable to speak, he may be prevented from joining in the voice chat."

However, "The user's input could be converted to voice locally, or by each remote computer. In this situation, for example, when the user kills another user's character, that victorious, though speechless, user would be able to tell the other user that he had been 'PWNED.'"

The patent also suggests lip-reading support. "A user may be able to speak or make the facial motions corresponding to speaking words. The system may then parse those facial motions to determine the user's intended words and process them according to the context under which they were inputted to the system."

Also detailed is a skeletal mapping system, capable of reading individual toe, finger and eye movements, again suggesting the device's potential usefulness to differently-abled gamers.

While the patent covers theoretical features and thus should not be read as confirmation that Kinect (or any derivation of it) will support sign language and lip-reading, it perhaps implies more facets to the motion controller than the launch game line-up suggests.

Read this next

Alec Meer avatar
Alec Meer: A 10-year veteran of scribbling about video games, Alec primarily writes for Rock, Paper, Shotgun, but given any opportunity he will escape his keyboard and mouse ghetto to write about any and all formats.
Related topics