On Kalman Filters

Kalman filtering is a 50 year old technique to obtain the "optimal" state estimate of a system that may change with time. It does this by using a two-step recursion: a prediction phase and a measurement phase. There are are several well written resources to understand Kalman Filtering such as this and this. It would be silly and unnecessary for me to try and explain that same standard material here. However, I recently developed a slightly different intuition to understand the principles behind Kalman filtering. I am fairly certain that I am extremely unlikely to be the only one with this explanation. However, since I could not find similar material on the internet while searching for Kalman filters, it should be of some use to those who land on this page. 

In the physical world, every observation/prediction is error-prone. My old professor, Dr. P. Subbanna Bhat liked to say that "God and noise are omnipotent". I don't know about God, but noise definitely has a strong case to make for it. Even the most precise models and the best instruments are limited by noise and other error sources inherent to the system. So when we launch a rocket, it extremely likely that the on-board sensors and the computer ground station report different flight parameter values. 

We can never know how much in error a particular measurement is. However, we do have a preliminary knowledge of how much error a particular source is likely to have on the basis of our prior experiments. This knowledge is called the variance of the source. A smaller variance does not mean that the source is always highly accurate. It is a good source. but there is a small chance that it can be terribly wrong. Similarly, a higher variance does not necessarily mean that its information is always incorrect. It is a bad source, but there is a small chance that it is more precise than the data from a good source. It would be foolish to completely ignore a bad source. 

Therefore, we have multiple information sources with different variances.  So, the question is: who do we trust?

A wise man would listen to all of them, but give higher consideration to more trust-worthy sources.

Kalman filters have two sets of equations: the prediction and the measurement equations. The prediction equations are used to measure the variance or the "trust-worthiness" of the model when making the prediction. The variance in the measurement is obtained by calibrating the sensors and instruments beforehand (or by some other means. It does not really matter how). 

So, in the Kalman scheme of things, we have two information sources: prediction and measurement, both error-prone with different variances. The trick is in using the variance information to weigh the predictions. The higher the variance or the more error-prone an information source is, the less consideration it is given. This can be done by finding the combination of weights that result in the minimal variance in the combined prediction. That optimal choice of weight is called the Kalman gain.

Each prediction and measurement affects the future prediction and measurement. That is why the algorithm is iterative and the value of Kalman gain is continually adjusted to obtain the best results.

The same intuition can easily be extended to cases where there are several information sources. Under all the jargon and mathematics, is a fairly intuitive and powerful idea. 

P.S - 

1. I am always amazed at how several ideas in signal processing and statistics are simply minimization or maximization problems applied in a higher dimensional space. Stripping away dimensions to do away with the horrible matrices, Jacobians, and all other evils, make them so much more intuitive.
2. I have left out all equations in this post because:

• There are several texts to teach the equations. I am simply trying to convey the intuition here. The equations would probably have been counter-productive.
• I could not figure out how to get Latex or Mathjax working quickly ... And I do not want to spend time figuring it out in the middle of writing this.