App Controls Swarming Robots from Your Smartphone, Using AR

New York University robotics researchers have created an app that can control a swarm of robots from your smartphone or tablet, using augmented reality. The data from a device's cameras and inertial sensors (accelerometers, gyroscopes, and magnetometers) can render augmented reality graphics on an interface screen, as well as tracking the robots. The researchers say that through the manipulation of these augmented reality graphics, users can intuitively command the robots to perform a task. Apple's software development platform Xcode 8 was used to design the app's interface. This creates a virtual grid, in which robots and objects can be easily detected on the screen. The current version of the app is using Raspberry Pi to carry out these commands. Visual tags called Fiducial Markers are placed on both the robots and the objects they are asked to move. The device captures the scene using its camera and the app detects these tags.