CHANNELS

Aerospace

Automotive

Automated & Autonomous Vehicles

Battery & Electrification Technology

Defense

Drone TV

Electronics

Manufacturing & Prototyping

Materials

Medical

Motion Control

RF & Microwave Technology

Robotics & Automation

Sensors & IoT

Test & Measurement

Aerospace
Automotive
Automated & Autonomous Vehicles
Battery & Electrification Technology
Defense
Drone TV
Electronics
Manufacturing & Prototyping
Materials
Medical
Motion Control
RF & Microwave Technology
Robotics & Automation
Sensors & IoT
Test & Measurement
Aerospace & Defense
Search Results

This Technology Is Marrying AI and AR

DARPA's Dr. Bruce Draper talks about the Perceptually-enabled Task Guidance (PTG) program, whose goal is to develop virtual “task guidance” assistants that can work with different sensor platforms to help military personnel perform their duties. PTG technology can see and hear what the user sees and hears via an integrated microphone, a head-mounted camera, and displays like augmented reality headsets to deliver accurate instructions.

“Today the commercial sector is pursuing new, useful ways to present data to the user but it doesn’t go far enough,” said DARPA program manager Dr. Bruce Draper  . “The gamechanger with PTG would be having perceptually-driven AI interfaces that can make sense of the real world, react to whatever the user is doing and provide advice. I'm really impressed at how quickly performing teams are making progress toward the goals.”

Topics:
AR/AI Electronics & Computers

More From SAE Media Group

    Transcript

    00:00:00 the idea of ptg is to have a device that sees what you see it hears what you hear and it's there to help you sort of like a co-pilot walk you through steps of a complex procedure make sure you do them correctly basically help you do a complex physical skilled task better darf is involved because it fills a really necessary military need how we're going to do more without an infinite

    00:00:24 amount of training darp is in a position to do it because we're at The Cutting Edge of both augmented reality and artificial intelligence so the technology is sort of marrying artificial intelligence so to think of an agent with augmented reality since you've got a device that is really helpful in letting you do things in the real world when you know your hands are

    00:00:43 in the middle of something and you've got to figure out what the next step is and so it's this interactive two-way audio visual help that we're looking to add we're really focusing on one Battlefield medicine when people get hurt we want to get them safe and keep them healthy sustainment by which we mean the people who are responsible for keep keeping the Air Force in the sky

    00:01:02 the tanks rolling the ship's sailing and thirdly is what we call co-piloting specifically looking at helicopters we're navigating we're communicating all these other complex tasks so those are the three specific things we're focused on but I want to be clear I think all those three are important but I think they're just scratching the surface there are so many military Specialties

    00:01:23 that could be improved with this kind of help today I'm talking to you here from the hacker reactor at MIT we're being hosted by Lincoln Labs who was one of the evaluators for this this program so what we have around the reactors are different sets of teams they're using different Hardware they've got different approaches but they're all showing their first versions and the idea is partly

    00:01:44 for us to see their progress but also for all of them to work together so we get this sort of cross stimulation of groups working with each other so this may surprise people but the demos right now are designed to do cooking we're giving people cooking recipes cooking actually is a really good example of a complex physical task that can be done many ways there are lots of different

    00:02:07 objects solids liquids things change state so it's visually quite complex there's specialized terminology there are specialized devices and there's a lot of different ways it can be accomplished so it's a really good practice domain for all kinds of other highly skilled tasks we also have members of the end user communities to come in and sort of look at these early

    00:02:32 stages of this technology give user feedback right make sure we develop the technology to solve these very real needs and at the same time hopefully get an idea of where the technology is going so they can bring that back to their leadership with regard to what future possibilities are the application I'm looking at is a medical simulation for training

    00:02:53 we definitely need to come up with systems that are objective and repeatable to get away from the subjective C1 do one teach one model that all medicine uses we need a system that's accurate enough to eventually make judgments on the competency and possibly be used for long-term longitudinal observation of the evolution of individual Medics and

    00:03:19 providers this is a really difficult task and this is a challenge but not only are the performers meeting it but I've got performers showing up early so for example no one was supposed to have to do the battlefield medicine task yet and yet we've already got performers out doing Battlefield medicine tasks so that's great so the excitement in the community and the sort of Rapid pace of

    00:03:40 the community I'm really impressed at how quickly people are making progress toward the goals