CHANNELS

Aerospace

Automotive

Automated & Autonomous Vehicles

Battery & Electrification Technology

Defense

Drone TV

Electronics

Manufacturing & Prototyping

Materials

Medical

Motion Control

RF & Microwave Technology

Robotics & Automation

Sensors & IoT

Test & Measurement

Aerospace
Automotive
Automated & Autonomous Vehicles
Battery & Electrification Technology
Defense
Drone TV
Electronics
Manufacturing & Prototyping
Materials
Medical
Motion Control
RF & Microwave Technology
Robotics & Automation
Sensors & IoT
Test & Measurement
Aerospace & Defense
Search Results

AI Leads to Advances in Robotic Dexterity

Watch this video to see an MIT CSAIL team present a general object reorientation controller that does not make assumptions. It uses readings from a single commodity depth camera to dynamically reorient complex and new object shapes by any rotation in real time, with the median reorientation time being close to seven seconds.

Topics:
Automation Motion Control Robotics

More From SAE Media Group

    Transcript

    00:00:04 - [Pulkit] We really believe that dexterous manipulation is going to be the future in robotics. What can robots do today very well, is they can move around and they can pick up objects. But just imagine for a moment, what do you do you do in your house or what happens in assembly? You might pick up a screwdriver, use it to screw in a screw.

    00:00:21 You might pick up a knife, use it to peel a vegetable or cut it. What these things involve is picking an object, but then making sure that the blade of the knife faces the vegetable. This task is what we call is reorientation. If I take an object in one pose, I try to rotate it into a different target pose. Now, this involves my fingers to make and break contact with the object,

    00:00:46 which makes it quite challenging. And also, the fact that I have my fingers move in coordination with each other. Now, what we present today is a versatile dexterous manipulation system, which can reorient many complex shapes, just with one depth camera. And everything is now open-source. So it turns out, currently, robots could pick up objects and that is why you're seeing a lot of deployment happening

    00:01:10 in logistics and in warehousing. But what robots couldn't do before was to use tools. And one of the key enablers is for the system to reorient objects. So as you see now, this robot is reorienting the object. It has four fingers which are moving to get the object in the right shape. - [Tao] It performs best when we do it on the train objects, and it has some decent generalization capabilities, so it can work to some extent on some unknown objects.

    00:01:39 And of course there's a big room for improvement. So we are still working on improving the generalization aspects of this controller. - [Pulkit] So what would it take for someone to get this up and running? How much time do you think it'll take? - So amount of time it takes to build the hardware, which takes a few hours. And then the second part is setting up the software and we provide the docker image so people can just download

    00:02:01 and then run the code directory. So in the upfront software side, it should be even less than one hour. This - Is our effort towards making robots or robotics being more democratized, being more open source. - We train everything in simulation and because in simulation we can get tons of data and so we can really address these challenging issues. And eventually we transfer this controller from the simulation to the real world.

    00:02:26 We open source the code and also pre-train model and also the CAD model so everyone can replicate our results. And this is very important to promote the dexus manipulation research. And another thing that's very different from what we did to what OpenAI did is we actually in our setup or the hand is facing downward. And in this case, the graph really matters. - My hand is protecting against gravity.

    00:02:52 But just imagine how do you use your knife? How do you use screwdrivers? Your hand is always facing on the top. So now when I'm reorienting this object, the problem is that if I make one small mistake, the object can drop down. And most importantly, we don't have to assume a fixed set of objects, which means that if you have a new object, if you're in a household in your factory, you get something new, you don't have to go and retrain the system.

    00:03:16 You can use our system to hopefully still be able to reorient the object. We can think of even more things where the technology might have potential applications. What about prosthetics style? - If a robot can use different kind of tools, then it can really help the people with disabilities to use those tools to do daily tasks. - One of the things which has restricted robotics for quite some time is if you have a new task, you have

    00:03:42 to build a new controller. It requires teams of people and weeks or months, even half a year of time to design a controller for that task. And one thing with Tao and other people in the lab we have been trying to develop is a paradigm where we can train controllers very easily with very less human effort. The system is completely centurial,

    00:04:03 but how, you know, we know that once things are centurial, they end up having some issues because simulation is always a model of reality and not reality per se. - So the way we do centurial is that we train a neural network controller in the simulation and this simulation is a digital twin of the real world system. And after we train the controller, we just transfer it to the real system and the in a zero short way.

    00:04:28 In terms of the simulation to the real world gap, there are mainly two gaps. So one is the perception gap. The images from the camera looks different when it's in the simulation and when it's in the real world. - And that is perhaps because the rendering is not as good. - The rendering in the real world is much more noisy. And in the simulation the rendering is very clean. And another gap is dynamics. This motors behaves actually different

    00:04:53 when these real motors and also the simulated motors. And while we get reasonable centurial transfer performance, there's still some room for improvement. - So of course we have not solved the reorientation problem. There are many limitations there still exist that the system is not as precise as we want it to be. Sometimes it also drops objects, unfortunately. The other thing we are really excited about is the future of Dexter hands. You know, for a long time people have been developing many

    00:05:21 different kinds of Dexter hands, but they did not exist in algorithm which could perform complex manipulation such as reorientation. But now that we have the algorithm, we think this space of Dexter's hands is going to heat up again and which might change the landscape of what people think robots are able to do today versus what they can do very shortly in the future. And that thing really is tool use, right? Not just picking objects, but using them.

    00:05:51 The future is truly exciting. The future is going to belong to Dexter manipulation. And what we're giving you today is just a preview into the future. So stay tuned and we'll be back.