The Yale-CMU-Berkeley (YCB) Object and Model Set provides universal benchmarks for labs specializing in robotic manipulation and prosthetics. About two years ago, Aaron Dollar, an associate professor of mechanical engineering and materials science at Yale University, came up with the benchmark idea to bring a level of specificity and universality to manipulation tasks in robotics research. He enlisted the help of two former colleagues in the robotics community, Dr. Siddhartha Srinivasa from Carnegie-Mellon University and Dr. Pieter Abbeel of the University of California, Berkeley.
The set contains 77 objects, including things like hammers, a pan and spatula, a cordless drill, a can of Spam, and a nine-hole peg test. These household items could create a new kind of standardization for robotics. For instance, a research paper might describe a task as “robotic hand grasps hammer” without specifying whether it is a big hammer or a little one. That’s a problem for a robotics researcher looking to replicate the results. With the YCB Set, everyone would be working with the same 23.45-ounce Stanley hammer included in the set.
In addition to the objects, the project provides five examples of manipulation tasks — such as pouring water from a pitcher to a mug or setting the table — and benchmarks for each. A website for the project allows other laboratories to expand on these tasks by contributing their own protocols and benchmarks. When laboratories work solely by their own standards and protocols, Dollar said, there’s often an unconscious bias toward that lab’s particular strengths. Universal standards would provide a more impartial way to evaluate results.
The YCB objects and example tasks are just a beginning. Manipulation research progresses quickly and covers a wide range of technical interests and research approaches, so the five manipulation tests Dollar and his team provide are only examples of protocols that labs can use with the objects. That’s why on the YCB Object and Model Set website, the research team has also provided a framework for other labs to contribute their own manipulation tests and benchmarks. There, researchers can see protocols from other labs and have a forum for discussion.
The objects are divided into categories. The food group, for example, includes a cereal box, a cylinder of Pringles chips and a can of Spam. Tools range from small nails to wood blocks and a cordless drill. Dollar said he aimed for a wide variety of sizes (the smallest item is a washer, the largest a water pitcher). Some items have simple geometric shapes that are relatively easy to grasp, while the complex shapes of others pose a greater challenge for robotic hands.
The items also include various task-based objects: a “box-and-blocks test” in which wooden cubes are to be placed in a box; a toy airplane that can be assembled and disassembled; and a variety of Lego pieces for building structures. The set also comes with a digital timer to measure how quickly certain tasks are performed.
For the project to succeed, Dollar needs to convince other labs to adopt the YCB Set. About 100 robotics labs around the world now have the YCB Set, which costs about $350.
For more information, visit www.ycbbenchmarks.org.