Non-verbal people with disabilities often rely upon devices like grid pads to communicate. The tablet technologies display dozens of words and phrases, accompanied by pictures and preset categories, so users can form sentences.
Carlos Pereira thought this approach could be overwhelming for some users.
Pereira, who has a daughter with cerebral palsy, found a way to use machine learning to ease the communication process.
The former computer analyst developed “Livox .” The software, compatible with an Android tablet, offers a more personalized and focused set of responses.
During a conversation, Livox uses Natural Language Processing to understand context and propose possible interactions for the disabled person.
For example, if you ask, “How many spoons of sugar do you want in your coffee?” Livox knows the answer is a number, and provides a few likely responses – not dozens.
“This has the potential to speed communication by a lot!” Pereira told Tech Briefs.
The solution displays fewer response items based on time and location. When it's lunch time, for example, Pereira’s system will show likely food options.
The algorithms adjust according to a given disability. Livox’s “IntelliTouch” software, for example, corrects the imperfect touch of someone with motor disabilities. Another algorithm allows those with limited mobility to use Livox with their eyes.
The system tracks track five different categories for each user: Behavior, Cognitive, Auditory, Visual and Oral Language. The more one uses Livox, the smarter it gets, altering its interface and tailoring response choices according to each category score.
In 2018, Livox was a finalist in MIT’s Solve Challenge Finals , a live pitch event that gives inventors and researchers an opportunity to present their best ways of tackling large-scale global challenges. (Livox was also a category winner in the 2019 Create the Future Design Contest, presented by Tech Briefs Media Group.)
In an edited interview below, Pereira tells Tech Briefs about the future he envisions for Livox, and why the communication tool may even find its way into your next car.
Tech Briefs: What inspired you to make this device?
Carlos Pereira: My inspiration was and is my daughter, Clara. She is twelve-years-old and she has Cerebral Palsy due to a medical mistake during my wife's labor. From the moment I found out about her condition, empowering people with disabilities became a passion for me. Because of that, my wife and I started several different initiatives to help my daughter. One of them is Livox, the software created to help my daughter to communicate.
Tech Briefs: Can you help us visualize the device? What does it look like and how does it work?
Carlos Pereira: Livox is a software for Android tablets that enables non-verbal people with disabilities and people with learning impairments to communicate and to learn. Its interface varies according to user disability. Usually, it shows a grid of cards that can make something when they are activated. These cards can be programmed to speak a sentence, to play a song, to read a story, to teach an abstract concept. In this way, Livox can be used at home, in schools, and in therapy centers for people with disabilities for multiple purposes.
Tech Briefs: Can you take us through an example or two of where Livox is most valuable?
Carlos Pereira: Livox empowers non-verbal people with disabilities to communicate. Communication is the most basic human need, and unhappily many people with disabilities are seen as burdens of society. So, by giving them a voice we can finally understand their needs. Besides that, Livox is extremely valuable in schools. People with disabilities usually have a hard time accessing educational content, and through Livox, teachers and educators can be sure that, regardless of the student's disability, they will be able to access any educational content.
Tech Briefs: What other far-reaching applications with Livox are possible? I saw that the device could potentially be used to address distracted driving?
Carlos Pereira: Livox's algorithms can have multiple applications. For example, our Natural Language Processing algorithms can be used to improve computer-human interfaces by making them more friendly not only for people with disabilities. Since Livox can understand the context of a conversation, this has the potential of enabling user-friendly interfaces that uses less interaction with a human.
Tech Briefs: What’s next?
Carlos Pereira: We are working on new Artificial Intelligence algorithms to make Livox work better with even more disabled users – for example, people that are able to use just their eyes for communication. Currently, we are working really hard on improving our algorithms so less external factors influence the user experience in this situation.
Tech Briefs: What’s most exciting to you about this technology and its possibilities?
Carlos Pereira: For me, personally, it is to be able to talk to my daughter. It's very interesting to understand how her personality and tastes change since she is becoming a teenager now. If it wasn't for Livox, this would be impossible.
What do you think? Share your comments and questions below.