Singular Value Decomposition

Factorization that works for all matrices.

Here:

• $U$, $V$ are unitary
• $\Sigma$ has singular values on diagonal

Every matrix $A \in \mathbb{C}_{m\times n}$ has a SVD.

$\text{rank}(A^HA) = \text{rank}(AA^H) = \text{rank}(A) = \text{rank}(AH)$ is equal to the no. of non zero singular values of A.

Singular Values and Singular Vectors

Let $A\in \mathbb{C}_{m\times n}$. Singular values $\sigma \ge 0$ and the singular vectors $u \in \mathbb{C}_{m\times 1}$ and $v \in \mathbb{C}_{n\times 1}$ are defined by:

Calculation

Let $A\in \mathbb{C}_{m\times n}$.

Singular values $\sigma \ge 0$ and singular vectors $u,v$ can be obtained by solving the below eigenvalue problems:

• $AA^Hu=\sigma^2 u$ and
• $A^HAv = \sigma^2v$

The common eigenvalues for the above 2 equations $\sigma^2$ are the singular values. We choose the positive solutions when $\sigma^2$ is positive.

Now $A$ can be written as:

Where:

• $V \in \mathbb{C}_{n\times n}$ consists of $v$’s
• $U \in \mathbb{C}_{m\times m}$ consists of $u$’s
• $\Sigma \in \mathbb{R}_{n\times m}$ consists of eigenvalues on the diagonal and 0s elsewhere.

Both $V$ and $U$ are unitary.

Application

Linear Least Solution

For $A_{m\times n}X_{n\times 1}=B_{m\times 1}$. If $A=U\Sigma V^H$ the equation can be converted into $\Sigma V^H X = \Sigma Y = U^{H}B = C$ which is easy to solve. To solve the system, ignore the zero-only rows in $\Sigma$ and solve other equations.