You CAN Teach a Robot New Tricks
Watch this video to see researchers at MIT's CSAIL use vision models to teach robots limits — enabling more intricate and diverse task completion, done safely and correctly.
“LLMs and classical robotics systems like task and motion planners can’t execute these kinds of tasks on their own, but together, their synergy makes open-ended problem-solving possible,” says PhD student Nishanth Kumar SM ’24 , co-lead author of a new paper about PRoC3S. “We’re creating a simulation on-the-fly of what’s around the robot and trying out many possible action plans. Vision models help us create a very realistic digital world that enables the robot to reason about feasible actions for each step of a long-horizon plan.”