Cov Matrix footnote

Footnote

For those who know linear algebra and are interested, here is a sketch of the proof.

Let feature vector x have mean m_x and covariance matrix C_x, and let feature vector y have mean m_y and covariance matrix C_y. If y = Ax, we know that m_y = Am_x and C_y = A C_x A'. From this last expression, it follows that C_y^-1 = A^-1 ' C_x^-1 A^-1. Now consider

r_y² = ( y - m_y )' C_y^-1 ( y - m_y )

= ( Ax - Am_x )' ( A^-1 ' C_x^-1 A^-1 ) ( Ax - A m_x )

= ( x - m_x )' A' A^-1 ' C_x^-1 A^-1 A ( x - m_x )

= ( x - m_x )' (A^-1 A)' C_x^-1 (A^-1 A) ( x - m_x )

= ( x - m_x )' C_x^-1 ( x - m_x )

= r_x² .

Back