MIT's ultrasound wristband tracks gestures to guide robotic hands

MIT engineers have created a wearable ultrasound wristband that precisely tracks complex hand movements by imaging the internal muscles, tendons, and ligaments of the wrist. Utilizing miniaturized ultrasound sensors and an AI algorithm trained on thousands of data points, the device translates anatomical shifts into digital data representing 22 degrees of freedom across the fingers and palm. This approach overcomes limitations of traditional hand-tracking methods like cameras, sensor gloves, and EMG sensors, which can be obstructed, uncomfortable, or noisy. The wristband was tested on diverse volunteers and accurately decoded intricate hand gestures, including 26 American Sign Language letters and subtle grips for objects like scissors and tennis balls. The wristband demonstrated real-time control of both virtual and robotic environments, enabling users to manipulate virtual objects with natural gestures and remotely operate a robotic hand to perform tasks such as playing piano tunes and desktop basketball. Researchers highlight its potential as a primary tool for virtual and augmented reality applications, offering more natural and precise hand tracking. Additionally,