Microsoft engineers have developed a new type of augmented mechanical keyboard, sensing rich and expressive motion gestures performed both on and directly above the device. A low-resolution matrix of infrared (IR) proximity sensors is interspersed with the keys of a regular mechanical keyboard. This results in coarse, but high frame-rate motion data.
A machine learning algorithm, traditionally used for static classification only, supports dynamic, temporal gestures. The device uses motion signatures, a technique that utilizes pairs of motion history images, and a random forest classifier to recognize motion gestures.
A large set of gestures can be performed with the keyboard, including typing, hovering, and swiping.