A smartphone app was developed that allows a user to easily program any robot to perform a mundane activity such as picking up parts from one area and delivering them to another. The app, called VRa, uses augmented reality and allows the user to either walk out where the robot should go to perform its tasks or draw out a workflow directly into real space. The app offers options for how those tasks can be performed such as under a certain time limit, on repeat, or after a machine has done its job.
After programming, the user drops the phone into a dock attached to the robot. While the phone needs to be familiar with the type of robot it is “becoming” to perform tasks, the dock can be wirelessly connected to the robot's basic controls and motor. The phone is both the eyes and brain for the robot, controlling its navigation and tasks.
To get the robot to execute a task that involves wirelessly interacting with another object or machine, the user scans the QR code of that object or machine while programming, effectively creating a network of Internet of Things. Once docked, the phone (as the robot) uses information from the QR code to work with the objects.
The user can also monitor the robot remotely through the app and make it start or stop a task such as to go charge its battery or begin a 3D printing job. The app provides an option to automatically record video when the phone is docked, so that the user can play it back and evaluate a workflow. The app knows how to navigate and interact with its environment according to what the user specifies through building upon simultaneous localization and mapping. These types of algorithms are also used in self-driving cars and drones.
Since creating the prototype, researchers have been testing it in real factory settings to evaluate user-driven applications. Ultimately, the app is a step toward creating future smart factories, powered by artificial intelligence and augmented reality, that complement and increase worker productivity rather than replacing them.
For more information, contact Karthik Ramani, Donald W. Feddersen Professor of Mechanical Engineering, at