Semiautonomous Controller for Assistive Robotic Manipulators & Remote Devices


With funding from the Mann Fund, researchers at the Johns Hopkins University Applied Physics Laboratory (APL) are developing a next-generation retinal prosthesis system that will enable the Laboratory's broader work on a system known as HARMONIE (Hybrid Augmented Reality Multimodal Operation Neural Integration Environment). The work will be conducted in collaboration with Second Sight Medical Products, a Sylmar, CA-based company that develops, manufactures, and markets implantable visual prosthetics for blind individuals. The APL researchers will be developing next-generation glasses with embedded vision and eye tracking sensors. Using these sensors and onboard hardware, the system will identify potential obstacles. The components of this retinal prosthesis will enable the lab's broader vision of a semiautonomous controller for assistive robotic manipulators and remote devices, HARMONIE. The basic idea behind the system is to combine elements of computer vision, autonomous manipulation, and a user interface.