SVD

Singular Value Decomposition (SVD) is a matrix factorization technique that breaks down any matrix into three simpler matrices. It's one of the most fundamental tools in linear algebra and forms the mathematical foundation behind PCA, recommendation systems and more.

The SVD Formula

Any matrix A of shape (m x n) can be decomposed as:

A = U . Sigma . V^T

• U - an orthogonal matrix of left singular vectors (m x m)
• Sigma - a diagonal matrix of singular values (m x n)
• V^T - the transpose of an orthogonal matrix of right singular vectors (n x n)

Why SVD Matters

SVD works on any matrix, not just square ones, which makes it more general-purpose than eigendecomposition. It's the technique used under the hood by many PCA implementations, and it's also central to recommendation systems, where it helps uncover latent relationships between users and items.

Implementing SVD in Python

import numpy as np

A = np.array([[4, 0], [3, -5]])
U, S, VT = np.linalg.svd(A)

print("U:\n", U)
print("Singular Values:", S)
print("V^T:\n", VT)

Coming Up Next

Next, let's break down exactly how SVD arrives at these three matrices with a step-by-step working example.