Associate Professor Suranga Nanayakkara (left) with NUS student Mark Myres (right), who tested AiSee as a visually impaired user. (Image: NUS.edu)

Simple tasks that most of us take for granted — grocery shopping, running into the pharmacy for a single item, even picking out an outfit — are not surprisingly incredibly difficult for people with visual impairments (PVI). However, a team from the National University of Singapore’s (NUS) School of Computing is on the verge of ameliorating that burden via AiSee.

AiSee is a smart headphone equipped with an integrated camera, microphone, and an AI-powered software architecture, said Lead Researcher Suranga Nanayakkara, Associate Professor, Department of Information Systems and Analytics, NUS Computing. It “sees” what’s in front of the user, “hears” what the user says, and is able “speak” to the user.

“A user can simply ask, follow up, and get answers to questions related to their surroundings,” Nanayakkara said. “AiSee responds like a personal assistant, giving the users the information they need with the ease of a conversation.”

AiSee’s 2018 inspiration came from one of Nanayakkara’s friends, Sunish. Whilst colleagues at MIT, Sunish would use his smartphone, browse to a reader app, and take a picture of his lecture notes; the app then read the text back to him.

“For him, this worked,” Nanayakkara said. “However, it didn’t make sense to me why he had to unlock the phone, browse to an app via voiceover feature, and take a picture every single time. For a blind person, a smartphone typically requires multiple steps before an application can be used. Back then, my immediate reaction was ‘What if we take the camera out from the smartphone and put it on a wearable ring?’ — a blind person would then simply be able to point at something and ask for information (via minimal number of steps to accomplish a task).”

Since its 2018 inception, AiSee has evolved greatly, Nanayakkara added. The form factor went through several iterations, with the current version boasting a design patent. Also, the software and AI backend went through major upgrades to be production-ready. Plus, the team introduced additional interactions during which AiSee can be more than just an assistive device: It can function as a high-quality Bluetooth headset, answer calls, manage messages, and listen to music.

The nature and design of AiSee is driven by lessons from established design frameworks for both natural interaction and assistive technologies. This includes over 10 years of continuous engagements with blind and low-vision (BLV) users and their caretakers, according to Nanayakkara. This provided an empirically grounded understanding and revealed unique opportunities to effectively scaffold daily activities more satisfying experience for BLV.

There are a number of related products: mobile apps such as BeMyEyes and SeeingAI, which are inconvenient according to Nanayakkara, and smart glasses like Ray-Ban Meta and EnvisionAI. However, many blind people are hesitant to use smart eyewear due to the stigma with wearing sun- or eyeglasses. However, seeing how many BLV users already don headphones, the resemblance between AiSee and standard headphones helps shift this perspective, according to BLV user feedback, he added.

Over the past 10 or so months, the team has been working to push AiSee for commercialization, Nanayakkara said. They have a refined wearable form factor, software, and are now at the beta testing with a few BLV users in collaboration with SGEnable (the focal agency for disability and inclusion in Singapore).

“The potential impact of AiSee extends beyond an assistive device and stands to be a powerful tool that enhances our perception of the visual world,” Nanayakkara said. “This could be a companion to provide contextually relevant information to support learning, memory, etc.”

Nanayakkara said that the team faces major challenges pushing AiSee toward a product that can be handed over to users for everyday use. The issues include design for manufacturing to get a robust and scalable hardware device, developing a production level software stack, and making sure that users get information within a reasonable time limit.

“I have to constantly switch my mindset from a researcher to a product developer,” he added. “We are pushing really hard, and we hope AiSee will be available for purchase before summer 2025!”

This article was written by Andrew Corselli, Digital Content Editor, SAE Media Group. For more information, visit here  .

Topics:
Design Manufacturing & Prototyping Optics Product Development

More From SAE Media Group

Tech Briefs
10 Tech-Savvy Holiday Gifts for Engineers
Related Article Image
Tech Briefs
Automotive & Transportation Finalist: Multi-Modal Traffic Detection System
Related Article Image
Tech Briefs
Readers Select 2023 Products of the Year
Related Article Image
Tech Briefs
Shape-Changing Smart Speaker Lets Users Mute Areas of a Room
Related Article Image