A new software system turns cellphones into augmented reality portals, enabling users to place virtual building blocks, furniture, and other objects into real-world backdrops and use their hands to manipulate those objects as if they were really there. The system, called Portal-ble, could be a tool to experiment with augmented reality (AR).
Current AR apps use smartphones to place virtual objects into real-world scenes but interacting with those objects requires users to swipe on the screen. Portal-ble makes use of a small infrared sensor mounted on the back of a phone. The sensor tracks the position of people's hands in relation to virtual objects, enabling users to pick objects up, turn them, stack them, or drop them. It also lets people use their hands to virtually “paint” onto real-world backdrops.
Researchers added sensory feedback — visual highlights on objects and phone vibrations — to make interactions easier. Users feel the vibrations in the hand they're using to hold the phone — not in the hand that's actually grabbing for the virtual object. Vibration feedback helps users more successfully interact with objects.
The team is expanding Portal-ble's object library, refining interactions, and developing new activities. They also hope to streamline the system to make it run entirely on a phone. Currently, the infrared sensor requires an infrared sensor and external compute stick for extra processing power.