Automatic image recognition is widely used today. There are computer programs that can reliably diagnose skin cancer, navigate self-driving cars, or control robots. Up to now, all this has been based on the evaluation of image data as delivered by normal cameras and that is time-consuming. Especially when the number of images recorded per second is high, a large volume of data is generated that can hardly be handled.
Using a special 2D material, an image sensor was developed that can be trained to recognize certain objects. The chip represents an artificial neural network capable of learning. The data does not have to be read out and processed by a computer; the chip itself provides information about what it is currently seeing within nanoseconds.
Artificial learning involves a network of neurons that is simulated digitally and the strength with which one node of this network influences the other is changed until the network shows the desired behavior. Typically, the image data is first read out pixel by pixel and then processed on the computer. With the new method, the neural network with its artificial intelligence is integrated directly into the hardware of the image sensor, making object recognition many orders of magnitude faster.
The chip is based on photodetectors made of tungsten diselenide — an ultrathin material consisting of only three atomic layers. The individual photodetectors — the pixels of the camera system — are all connected to a small number of output elements that provide the result of object recognition.
The sensitivity of each individual detector element can be adjusted, controlling the way the signal picked up by a particular detector affects the output signal. This requires simply adjusting a local electric field directly at the photodetector. The adaptation is done externally, with the help of a computer program.
One can, for example, use the sensor to record different letters and change the sensitivities of the individual pixels, step by step, until a certain letter always leads exactly to a corresponding output signal. This is how the neural network in the chip is configured — making some connections in the network stronger and others weaker.
Once this learning process is complete, the computer is no longer needed; the neural network can now work alone. If a certain letter is presented to the sensor, it generates the trained output signal within 50 nanoseconds; for example, a numerical code representing the letter that the chip has just recognized.
The technology can be usefully applied wherever extremely high speed is required, from fracture mechanics to particle detection.
For more information, contact Prof. Thomas Mueller at