Singular Value Decomposition | Sahithyan's Notes

Factorization that works for all matrices.

A = U \Sigma V^H

Here:

$U, V$: unitary
$\Sigma$: singular values on the diagonal

Every matrix $A \in \mathbb{C}_{m\times n}$ has a SVD.

$\text{rank}(A^HA) = \text{rank}(AA^H) = \text{rank}(A) = \text{rank}(AH)$ is equal to the no. of non zero singular values of A.

Singular Values and Singular Vectors

Let $A\in \mathbb{C}_{m\times n}$ . Singular values $\sigma \ge 0$ and the singular vectors $u \in \mathbb{C}_{m\times 1}$ and $v \in \mathbb{C}_{n\times 1}$ are defined by:

Av= \sigma u\text{ and }A^Hu= \sigma v

Calculation

Let $A\in \mathbb{C}_{m\times n}$ .

Singular values $\sigma \ge 0$ and singular vectors $u,v$ can be obtained by solving the below eigenvalue problems:

$AA^Hu=\sigma^2 u$ and
$A^HAv = \sigma^2v$

The common eigenvalues for the above 2 equations $\sigma^2$ are the singular values. We choose the positive solutions when $\sigma^2$ is positive.

Now $A$ can be written as:

AV= UΣ

Here:

$V \in \mathbb{C}_{n\times n}$: consists of $v$ ’s
$U \in \mathbb{C}_{m\times m}$: consists of $u$ ’s
$\Sigma \in \mathbb{R}_{n\times m}$: consists of eigenvalues on the diagonal and 0s elsewhere

Both $V$ and $U$ are unitary.

Application

Linear Least Solution

For $A_{m\times n}X_{n\times 1}=B_{m\times 1}$ . If $A=U\Sigma V^H$ the equation can be converted into $\Sigma V^H X = \Sigma Y = U^{H}B = C$ which is easy to solve. To solve the system, ignore the zero-only rows in $\Sigma$ and solve other equations.