Effectiveness of Multi-bispectral Passive Infrared Range Finding

The Goal:

Following in the footsteps of recent research, I investigated three methods of combining multiple bispectral estimations: Type A combinations (pixel-by-pixel mean), Type B combinations (pixel-by-pixel median), and Type C combinations (pixel-by-pixel error minimization). The goal of this project is to show the potential of using combinations of only bispectral estimations to give a reliable estimate.

Method:

Using data from a single scene, bispectral estimations were combined in three distinct ways and compared against LiDAR ground truth data. Each combination was assessed using Root Mean Square Error (RMSE).

Type A Combinations – Mean Operator
Bispectral Estimations are combined using an elementwise mean operator which take the mean of each pixel from every bispectral estimation being combined.

Type B Combinations – Median Operator
Bispectral Estimations are combined using an elementwise median operator which take the median of each pixel from every bispectral estimation being combined.

Type C Combinations – Error Minimization
Each Bispectral Estimation is compared pixel-by-pixel to the ground truth using absolute error, and then are combined via choosing the pixel value with lowest absolute error.
Important note: this method of combination requires knowing the ground truth in advance and is therefore not a proper technique for combining new values without a known ground truth.

What went right:

Type A and Type B combinations are shown to slightly improve RMSE of bispectral estimations.

Furthermore, using only a selection of the original estimations achieves a lower average RMSE than using all of the estimations available for the combination. This was true of both Type A and Type B combinations and shows the influence outliers on the data for both the mean and median operators.

What went wrong:

I had theorized that Type A and Type B combinations would improve as the combinations grew in length, but due to the sparsity of data at higher combination lengths, it lead to a higher RMSE at the largest lengths.

Outcome:

I showed in this project that bispectral estimations can be combined to give estimations
that are closer to the ground truth than by the single bispectral estimations alone. To do this
effectively, however, the process by finding the lowest estimator needs optimization. These calculations may be able to be optimized by deep learning networks but would require more dedicated research into optimizing the process for real-time deployment.

Initially, this project sought to find
differences between Type A and Type B combinations of bispectral combinations. The idea of minimization and introducing Type C combinations did not happen until late in the project’s development, and, as stated above, it requires the foreknowledge of the ground truth. Because of this and for the sake of time, several avenues of analysis were not pursued, namely computational time analysis for the three types of combinations and code optimization. Future directions for this project include optimizing the computational time for the combination operations, and finding the amount of estimations needed to have a high probability
of finding a good minimum RMSE for each combination length.