The “always-true” use case for 360º vision systems is to avoid blind spots. Blind spots are a byproduct of conventional cameras. They have a limited field of view and they only see in the direction that they were pointed at. Point them to a new direction and you miss what happened at the old direction. 360º vision systems do not have this drawback. They are always “on”, in any view direction. Among their disadvantages we have seen, depending on their configuration: a trade-off between temporal (frame rate) and spatial (range) resolution, and a limitation to single EO/IR bands.
With the 360in1 spherical vision system we were able to find a novel solution to give a panoramic system a multispectral vision, and to allow it to keep its character as a spherical vision system with large vertical field of view around the whole horizontal 360º field of view while extending its recognition and identification capabilities in the medium to long range.
Line Scanner vs Focal Plane
Panoramic line scanners have been the state-of-the-art technology when there is an operational need to look at a wider horizontal field of view in order to avoid blind spots. These scanners operate in the MWIR or LWIR bands and typically have a variable focus lens. They are designed for long range detection and are therefore systems that scan at low frequencies (refresh rate). They build an image that is regularly refreshed at a given interval. They are similar to radar in that they refresh the image with each “rotation”, with each scan. Their advantage is the high spatial resolution, but their disadvantage is a low temporal resolution, which essentially restricts its use as optical radar.
The requirements for ground tactical 360º vision systems are evolving, putting more focus on raising temporal resolution while maintaining realtime vision. Using EO or IR focal plane sensors with low f-number lenses breaks the temporal barrier and allows generating a much higher rate of panoramic frames per second at very high angular speeds.
The system from 360in1 reaches 24 panoramic frames per second for both EO and MWIR bands. It was designed as a multispectral shared-aperture system that combines SWIR and MWIR bands using the same focal plane and camera engine. This extends the mission capability and the operational envelope. It makes it a multispectral tactical vision system that provides fused imagery at high spatial resolution in the short to medium range, and it can detect multiple target phenomena depending on the application. For the non-visible bands it uses a standard definition imager at 640 × 512 pixels, but it can be upgraded to a high definition focal plane of 1280 × 720 or higher.
The 360in1 multispectral imaging can combine three to four spectral imaging bands into a single EO/IR optical system. The system can combine visible, near infrared, short-wave infrared, and mid-wave infrared bands into a single fused imaging system. Like many conventional non-panoramic electro-optical/infrared multispectral systems for intelligence, surveillance and reconnaissance (ISR) missions it combines one reflective-band (visual) sensor with one combined SWIR and MWIR sensor that have similarly matching pixel sizes. The simultaneous use of multiple wavelength bands across the EO and IR spectrum enables operators to see a higher level of detail, which is beyond of what the human eye is capable of.
Hacking the Spatial Resolution of 360º Vision
Panoramic and 360º vision systems based on focal plane sensors are typically configured in a way which limits the spatial resolution. Many systems, whether EO or IR imaging, would have rather low detection ranges of a few hundred meters. This is due to the fact that they are based on a technology that combines several video cameras that typically have standard resolution and requires them to use a wide-angle lens (over 100º FOV) in order to be able to create a panoramic picture with the given number of cameras. Therefore their spatial resolution is low. Ground tactical systems, however, require high spatial and temporal resolutions, which in principle is also true for 360º vision systems.
One method to increase spatial resolution with panoramic focal plane sensor based systems is to reduce the FOV of the lens. This comes at the expense of vertical field of view (VFOV). 360º vision systems, in contrast to their scanning sisters, should be configured with the highest possible VFOV at the highest possible spatial and temporal resolution required for a specific mission profile. 360in1 systems can be configured for a VFOV that ranges between 20º and 70º while covering the whole 360º HFOV. For example with a 14mm C-mount lens (approximately 38º FOV) customized for this specific vertical field of view, it is possible to achieve a spatial resolution of approximately 1m at a distance of 2.5km, without compromising too much VFOV.
On the detection level such a system could therefore cover an area of at least 5km in diameter at a reasonable 38º VFOV. But further recognition and identification of targets would be possible only as the targets would be moving closer to the camera. In many situations this is undesired, as efforts to get a better recognition and identification start from the time of the detection of the target. 360º vision systems could, for example, be designed with a variable focus lens that would allow reducing the VFOV in order to “zoom” in on the range in which the target is located. But it would come at the expense of vertical vision, which is important in order to not loose other potential targets and in order to not loose sensing capabilities (situational awareness) as a lower VFOV increases the blind areas. We found that the best practice is to maintain the VFOV for a given mission profile. In this case, we need to find a solution that enables recognition and identification on the 360º imaging without reducing the VFOV of the panoramic system.
This can be achieved by combining EO and IR zoom cameras with high variable focal length numbers (e.g., 100mm) that are seamlessly integrated into the 360º vision system. Rather than accessing them separately, the panoramic system itself slews and cues the zoom cameras to the point of target detection while the VFOV of the spherical vision remains unchanged. This way the operator can zoom in on the target for further recognition and possible identification without loosing VFOV.
Therefore, a combination of at least one integrated zoom camera with the 360º vision system is how it is possible to hack the inherently lower spatial resolution of a 360º system for the long range, while still maintaining the original vertical field of view for highest possible situational awareness.