If A and B are symmetric matrices, then AB – BA is:

Concept:

• For symmetric matrices, A = A' and B = B'
• For skew-symmetric matrices, A = - A'
• (A ± B)' = A' ± B'
• (AB)' = B'A'

Calculation:

Given: A and B are symmetric matrices

As we know that for symmetric matrices, we have A = A' and B = B'

(AB - BA)' = (AB)' - (BA)' --------(∵ (A ± B)' = A' ± B')

⇒ (AB - BA)' = B'A' - A'B' --------(∵ (AB)' = B'A')

⇒ (AB - BA)' = BA - AB ----------(∵ A = A' and B = B')

⇒ (AB - BA)' = - (AB - BA)

Hence Option 3 is correct.