With speeds of over 1-million frames per second, high-speed imaging is an invaluable research tool capable of capturing the most fleeting events in scientific and engineering applications. Yet, achieving this speed comes with strings attached. Most high-speed cameras are designed to achieve a maximum gigapixel/second (Gpx/sec) throughput, offered in a trade-off of frame rate to resolution. For example, a 25 Gpx/sec camera reaches 25,700 frames per second (fps) at a resolution of 1280 × 800 pixels and can achieve a higher frame rate of 28,500 at a smaller resolution of 1280 × 720. Both combinations have almost the same throughput. Very high frame rates such as 1-million fps are accompanied by very small resolutions, making it more challenging to see the subject matter.
As frame rates increase, the exposure time a pixel has to light decreases. At 25,700 fps, each frame has a maximum exposure of 39 microseconds (µs), and at 1 million fps, the maximum exposure time is only 733 nanoseconds (ns). The short exposure times require high levels of illumination to compensate for the short time the pixel receives light. In fact, many high-speed applications are light starved, meaning that, given the very short exposure times at high frame rates, the available illumination won’t deliver enough light to the camera’s imaging sensor to produce an ideal image and may even be impractical in certain applications.
High-speed camera operators have become adept at balancing their need for speed and resolution with their need for adequate illumination. They are able to capture spectacular images that advance the frontiers of scientific understanding and engineering analysis, but the trade-offs become more difficult to manage as users push the boundaries of high-speed imaging.
Recently, a technical breakthrough was made that eases the speed-resolution-sensitivity constraint. A new high-speed image sensor has been developed that employs backside illumination (BSI) to increase the pixel surface area that can capture photons. Because it’s more effective at capturing light, the BSI sensor is better suited for applications requiring high frame rates. In this camera, throughput — max frame rate times max frame resolution — increased by a factor of three-times compared to previous generations of high-speed CMOS imaging sensors. The new sensor debuted March 2021 in the new Phantom TMX cameras, the fastest of which can shoot 76,000 fps at a full resolution of 1280 × 800 pixels.
Bringing BSI To High-Speed Applications
Until now, the CMOS sensors used in high-speed cameras have been based on frontside illuminated (FSI) architectures, in which the sensor’s metal circuitry sitting above the pixels’ photodiodes is facing the light source. This metal circuitry prevents some incident light from reaching the pixels, which in turn affects the fill factor and reduces the sensor’s sensitivity.
BSI sensors are designed with a thick carrier wafer attached to the top of the metal stack. This arrangement allows the bulk silicon to be thinned and flipped to expose the diodes facing the light source and the metal surface behind them. There are two significant advantages to BSI sensors in high-speed: improved fill factor, by providing a direct route for light to reach the light-receiving surface (see Figure 1), and improved processing speed, by adding more metal to the sensor’s metal surface.
Improved fill factor: This effectiveness at capturing incident light is expressed in terms of the sensor’s fill factor — or the percentage of the pixel surface area that is able to capture photons. With its metal circuitry blocking or reflecting some of the light, a typical FSI sensor used in high-speed imaging will have a fill factor between 50 and 60%, partially compensated for by a microlens in typical current FSI sensors. By moving the circuitry out of the way, this new BSI sensor has a fill factor of close to 100%.
Increased processing speed: The basic speed of the pixel array is limited by resistor-capacitor (RC) time constants, and adding metal reduces the resistance and increases the speed. In FSI sensors, the amount of metal on the sensor front is limited to allow light to reach the photodiodes. This constraint leads to overhead in the processing speed. As frame rates increase and resolutions decrease, the camera cannot provide maximum Gpx/sec throughput because of losses to overhead. BSI sensors do not have this constraint and can have significantly increased metal circuitry, substantially reducing or even eliminating overhead. This capability allows a BSI sensor to maintain its maximum Gpx/sec throughput even at very high frame rate/low resolution combinations.
BSI sensors have been available for more than 10 years in a variety of cellphone and standard digital cameras. They’ve offered proven advantages when it comes to improving low light performance and dynamic range of these consumer-focused cameras. Why did it take so long to bring these sensors to high-speed imaging? In a word, size.
The sensors and pixels used in high-speed cameras are much larger than standard cameras to minimize speed-resolution-sensitivity trade-offs. For instance, while a cellphone camera may have a pixel that measures less than 2 µm per side, pixels on this new image sensor are typically more than 6 µm and as much as 28 µm per side.
The manufacturing process for BSI sensors is inherently more difficult than comparable FSI sensors and requires additional manufacturing steps. Among them is a wafer backthinning step to remove the bulk silicon, bringing the photodiodes closer to the light source. There are also additional processing steps on the back side of the wafer to anneal the surface and provide electrical contacts to the front side. The size of high-speed image sensors only exacerbates manufacturing difficulties. The realities of semiconductor economics also made it difficult to transfer the technology from the high production volumes of standard cameras to the comparatively low volumes of high-speed imaging sensors. It took time to perfect the manufacturing process and achieve practical yields.
The BSI image sensor has been worth the wait. It sets new standards for:
Speed. The first camera using the sensor captures images at 76,000 fps at full 1-megapixel (1280 × 800) resolution, and it can reach speeds more than an order of magnitude faster at reduced resolutions and with binning. For example, the camera tops out at 1.75 million fps with a resolution of 1280 × 32 and 640 × 64-pixel binned. Historically, the resolutions associated with frame rates above 1 million fps were too low for nearly all scientific uses, but 1280 × 32 represents a truly usable resolution in a wide range of applications.
Exposure times. The new sensor supports minimum exposure times as fast as 95 ns with Export Controlled FAST option. The fast exposure times make it possible to capture ever-faster events without motion blur, which can be a limiting factor in obtaining high-quality images in applications as wide ranging as cytometry and combustion analysis.
Pixel size. To work in light-starved conditions, high-speed cameras have historically used very large pixel sizes as a means to catch as many photons as possible. Our existing FSI ultra-high-speed sensor, for example, has a pixel size of 28 µm per side for an area of 784 µm2. The new BSI high-speed image sensor has an 18.5 µm per side pixel, but its proficiency at capturing light makes it about as sensitive at three times the speed as earlier FSI sensors with 28 µm pixels. Smaller pixels also improve sampling frequency (Nyquist), allowing the sensor to resolve higher lp/mm spatial frequencies before aliasing. This capability enhances the imaging system’s performance in flow cytometry, particle image velocimetry (PIV), digital image correlation (DIC) and other high-speed applications limited by the resolving power of the sensor.
The performance breakthroughs associated with the new image sensor design mainly rest on its BSI architecture, but there’s more to the design. The new sensor also has a number of design features that boost performance beyond what BSI could accomplish alone — particularly related to the ability to read out the massive amounts of imaging data at high speeds and improve throughput.
Solving analog-to-digital conversion challenges. Embedding analog-to-digital converters (ADC) on CMOS image sensors is standard practice, but the BSI sensor’s speed required a massive increase in the amount of ADC. While modern CMOS image sensors typically have between 1,000 and 10,000 embedded ADC, the new BSI high-speed sensor has 40,000 ADC, each converting every 523 ns and generating a large amount of data to off-load from the sensor. To accomplish this task, it incorporates 160 high-speed serial outputs operating at greater than 5 Gbps. This technology is common on CPUs and FPGAs but new on a high-speed imaging sensor.
The density of ADC on the new sensor did create power management and electrical crosstalk challenges, which were solved with the help of our design and integrated production partner, Forza Silicon. While simulations are often used in predicting sensor performance, this sensor required the simulation to calculate for weeks to provide a prediction.
Forza has significant experience in simplifying simulations and analyzing actual versus predicted results for fast design modifications. In the case of the BSI sensor, testing of early designs revealed a higher level of ADC crosstalk in both normal imaging and binning modes than our simulation tools had predicted, causing noticeable artifacts in the images. Forza engineers discovered that the crosstalk exhibited predictable patterns and developed modeling techniques that helped eliminate the crosstalk altogether, which in turn mitigated imaging artifacts.
Binning for maximum throughput. The sensor supports 2 × 2 binning to maximize throughput at faster speeds. Though not common in high-speed sensors, we’ve implemented binning in two previous cameras. It helps mitigate limitations of the sensor’s column ADC architecture, enabling faster speeds than simply decreasing the y-dimension. This approach is subtly different than binning as applied in CCD cameras, where it’s used to primarily boost sensitivity. In this case, it was used to boost speed.
BSI is not a new technology, and it has been used with great success in standard and cellphone cameras. By adapting it to high-speed imaging, a sensor has been created that pushes the boundaries on speed in light-starved conditions.