Matrix Norm

All matrix is belong to ${\mathbb{C}}^{m\times n}$ if not specified.

1. Vector p norm

$$\Vert x{\Vert }_p={\left(\sum _{i=1}^n|x_i{|}^p\right)}^{1/p}$$

2. Matrix p norm

$$\Vert A{\Vert }_p=\underset{\Vert x{\Vert }_p=1}{\max }\Vert Ax{\Vert }_p$$

3. Frobenius norm

$$\Vert A{\Vert }_F=\sqrt{\sum _{i=1}^n\sum _{j=1}^n|a_{ij}{|}^2}=\sqrt{tr(A^{H}A)}=\sqrt{tr(A{A}^H)}$$

Frobenius norm and singular value

$$\Vert A{\Vert }_F=\sqrt{\sum _{i=1}^r{\sigma }_i^2}$$

proof

$$\Vert A{\Vert }_F=\sqrt{tr(A^{T}A)}=\sqrt{tr(V{\Sigma }^2{V}^T)}=\sqrt{tr({\Sigma }^2)}=\sqrt{\sum _{i=1}^r{\sigma }_i^2}$$

Q.E.D.

Orthogonal matrix do not change the Frobenius norm

$$\Vert A{\Vert }_F=\Vert JA{\Vert }_F=\Vert AK{\Vert }_F,\forall J,K\in {\mathbb{R}}^{m\times m}\text{ is orthogonal matrix}$$

4. Spectral norm(Matrix 2 norm)

$$\Vert A{\Vert }_2=\underset{\Vert x{\Vert }_2=1}{\max }\Vert Ax{\Vert }_2=\underset{x}{\max }\frac{\Vert Ax{\Vert }_F}{\Vert x{\Vert }_F}$$

Orthogonal matrix do not change the spectral norm

$$\Vert A{\Vert }_2=\Vert JA{\Vert }_2=\Vert AK{\Vert }_2,\forall J,K\in {\mathbb{R}}^{m\times m}\text{ is orthogonal matrix}$$

proof

$$\Vert A{\Vert }_2=\underset{\Vert x{\Vert }_2=1}{\max }\Vert Ax{\Vert }_2=\underset{x}{\max }\frac{\Vert Ax{\Vert }_F}{\Vert x{\Vert }_F}=\underset{x}{\max }\frac{\Vert JAx{\Vert }_F}{\Vert x{\Vert }_F}=\Vert JA{\Vert }_2$$

similar, $\Vert A{\Vert }_2=\Vert AK{\Vert }_2$
Q.E.D.