I then asked Seegrid VP of Engineering Sean Stetson about the cameras. “We use stereo cameras, which have two imaging channels — two separate sensors and two separate lenses. The cameras are automotive-grade imaging sensors. We prefer global shutters because they produce better stereo quality than rolling shutters. We design the cameras to produce concurrent exposures on both cameras on a given board, so that you're capturing the scene at the same time. From there, there's a lot of math to understand the parallax and the cyclical shifts between specific features, between the two cameras,” he said.
I then asked him how the data is used to control the vehicle. “You can imagine all the other different sensors we use — all sorts of encoders and the other things that you typically have on vehicles to help close feedback loops, as well as motor control systems — it's pretty standard stuff because we use stock powered industrial trucks,” he said. “The math associated with the dense stereo data to produce reliable navigation is the hard part.”
Driverless Forklifts vs Driverless Cars
Another question was whether there are significant differences between these industrial vehicles and self-driving automobiles. “There's not a whole lot of difference. Obviously, there are differences in environments. The types of scenarios encountered are more constrained indoors, compared to the freewheeling world that autonomous cars have to deal with. So, from a sensor suite standpoint, you tend to have more constrained distances and certainly more constrained speed.”
“Stereo cameras in the world of autonomous automobiles are getting more prevalent, specifically in the context of real-time, environmental sensing, in order to do things like semantic segmentation, object detection, classification, and tracking. The design goal for automobiles is to give the cars a kind of human sense of what's going on in the immediate environment around them, largely independent of the navigation. We've shown that cameras can be used for navigation in addition to these.” Stetson said.
The range of interactions with pedestrians, vehicles, or objects is obviously different and more constrained — these forklifts are installed into specific travel paths, so their movements are predictable — they stay in their path. Yet, the flexibility of the system allows those paths to be changed as the need arises, for example when there are changes in production flow. That's why these forklifts function so well in a typical assembly plant, where engineers are always looking for ways to improve operations and reduce overhead and wasted effort.
The required speed for processing the navigation information is a a function of speed of the vehicle. The faster it's moving, the more quickly it needs to take in information, process it, decide on a course of action, and then initiate it. Standard, human-driven industrial trucks typically run at a maximum of eight to nine miles an hour. Obviously, in the streets, cars can go significantly faster than that. “As the technology advances on all fronts, we'll see it being used in increasingly fast vehicles,” said Stetson. “One of the challenges facing the autonomous car market is getting sensors that are reliable at, say, 250 meters, because at 80 miles per hour, 250 meters is the distance needed to reliably detect something, classify it, decide on a course of action, and initiate the action in time to stop if need be. As for our vehicles, we have a whole collection of safety standards imposed on us by various organizations, one of which is ANSI B56.5. We do extensive safety testing on our system to ensure that we can stop within those target numbers over the range of distances that we deal with.”
Safety and Reliability
Christensen added: “Although we are certainly slower in these indoor environments, we are far from light-weight. We're transporting up to 10,000 pounds at a time, and the vehicle and battery add another 3,000 pounds.” That mass requires extensive testing. “The proof is in the pudding: this past February we exceeded a million miles of fully autonomous production travel with zero personnel safety-related incidents. The forklift industry is incredibly dangerous. In fact, safety is actually one of the important factors growing the demand for autonomy. Our forklifts operate significantly more safely than humans do. There's a fatality about every week in the U.S. from a forklift accident. It's really no surprise: you're moving loads around, the weight is shifting, and you're doing repetitive tasks. Humans are really bad at repetitive tasks. We get bored, and when we get bored, we take shortcuts. When we're moving really heavy things, those mistakes get people hurt,” he said.
Outlook for the Future
Finally, I asked Christensen what he thought about the outlook for his industry. He replied: “The adoption of automation is on the rise for mobile, autonomous material handling. Manufacturing is becoming much more complex. Companies are doing smaller lot sizes with mass customization, which makes supply chains far more complicated, which makes material handling challenges far more complicated. On the warehouse and distribution side demand is growing very rapidly — the ‘Amazonification’ of the whole industry. You can buy everything online, and you expect it instantaneously. The amount of pressure that puts on the logistics and supply chain is enormous and at the same time there is a labor shortage of people who are capable of doing these jobs or want to do them.”
This article was written by Ed Brown, Editor of Sensor Technology. For more information, visit here.