Although driverless forklifts have been around for about 30 years, it's only in the last 10 or so that they've been free to maneuver anywhere around their environment. In the early days, the machines followed a current in a wire buried in a concrete floor. There have since been a handful of different approaches, but all are variants on that theme. They went from wires in the floor to paint on the floor, to tape on the floor, to magnets in the floor, to reflectors on the walls, to QR codes on the ground, but they were all just different types of landmarks. So, the vehicles were confined to a fixed path — quite predictable, but extremely inflexible.

Figure 1. The primary navigation sensors are stereoscopic cameras, 10 of which are mounted in pairs on top of the vehicle.

The infrastructure-free navigation that has commercially matured in the past 10 years enables fully self-driving industrial vehicles that adapt to their environment based on sensor input. There is no need to depend upon explicit landmark infrastructure.

Infrastructure-Free Navigation

Figure 2. A driverless forklift can easily navigate around a corner.

Lidar is the standard out-of-the box solution for infrastructure-free navigation because it's relatively easy. Unfortunately, it performs quite poorly in real-world production settings, where environments are dynamic. In factories and warehouses, everything is moving, including pedestrians, so the machines have to navigate carefully and reliably. People don't buy automated forklifts to have them work some of the time, they're buying them so they never again have to worry about moving pallets 24/7.

Seegrid Corporation (Pittsburgh, PA) produces self-driving forklifts and tow tractors that use cameras for navigation. They're fully self-driving: they sense an environment and make decisions entirely onboard the vehicle. An array of cameras has the advantage over lidar because it affords wide coverage of an environment. With lidar, any individual range measurement is highly precise, but its coverage is so limited it's almost blind. A lidar unit is like a laser pointer on a turntable, it's just spinning around and will give a precise measurement of how far a wall is from you, but it tells you absolutely nothing about that wall ½ inch below or ½ inch above the beam.

Seegrid vehicles use the cameras in pairs to capture information in stereo and use the disparity between the two cameras to obtain three-dimensional spatial data. The five pairs “see” a full hemisphere over and around the vehicle as it is driving. With such a very rich environment of data available, the navigation is extremely reliable in real world conditions. Wide, accurate coverage is more important than having high precision but with only a small number of points. The cameras are used as data sensors — they're not doing landscape photography.

Programming the Forklift

Figure 3. The 360-degree views of the environment are processed by software to identify and locate thousands of unique features on every snapshot.

Programming the vehicles is very straightforward. Because Seegrid's forklifts are built on a manual chassis, you can drive them like a regular truck. In order to program it, you just put it into record mode and drive the route you want it to take. While you are driving, all 10 cameras — the five stereo pairs — are taking pictures continuously. As they find points of contrast in the images, the disparity between the pairs of cameras provides a dense 3D point cloud that covers the full space around the truck. Those images are captured every few centimeters as you drive. Every time the vehicle navigates that path thereafter, the system makes mathematical and geometric probabilistic comparisons against what it “sees” now versus what it saw before. The system then makes small navigation adjustments every fraction of a second to compensate for any differences. Since there's much more information that is persistent compared to what is dynamic, there's more than enough to navigate safely and reliably. It's similar to the teach/repeat model used elsewhere in robotics. Seegrid calls its patented system Evidence Grid technology, which uses probabilities of occupancy for each point in space. Applying probabilities to the data collected by the sensors compensates for the uncertainty in the performance of sensors and in the environment itself.

Putting the Driverless Forklifts to Work

Figure 4. Seegrid's vision technology enables its vehicles to autonomously transport materials throughout facilities for customers like Whirlpool.

I asked Jeff Christensen, Seegrid's VP of Product, whether these forklifts are best suited for repetitive tasks. “Imagine for a simple case in manufacturing that a vehicle does a loop around the assembly line all day long bringing parts to the line and taking others away. That's the most repetitive you might get and obviously it would work for something like that,” he said. However, “something maybe less obvious would be a distribution warehouse where you have pallets at an inbound dock. Each pallet is going to a different location — that's the other extreme,” he said. “For that, we create a ‘route network,’ which is a fully-connected graph of all the places it's been. Then when you tell the vehicle, ‘move this pallet from here to there,’ it will search its entire route network and find the shortest path for that particular task. You can think of it as sort of a dot-to-dot connection, that goes all over your plant or warehouse. Then, when I'm sending a truck somewhere, it can pick the optimal path to go anywhere within that space. Note that you don't want it to move just anywhere in a plant — that's like a delivery truck driving over your lawn to avoid a stop sign — but it should it pick the most productive path that is safe and predictable.”

There are many different ways to input instructions for a specific task, depending on the customer's application. In the simplest case you can put a start and end location on the vehicle itself by punching in a couple of numbers. In a more sophisticated application, it can be fully directed by Seegrid Supervisor, a software system that controls entire fleets of vehicles. Supervisor sends instructions to a specific vehicle, telling it what to pick up and where it's supposed to go. In that case, no operator actually touches it at all. Alternatively, Supervisor has a tablet interface that acts as more of a pull system — there are lots of different operational modes.

Sensor Technology Magazine

This article first appeared in the March, 2019 issue of Sensor Technology Magazine.

Read more articles from the archives here.