PCA

Principal Component Analysis (PCA) is a statistical technique that transforms a set of possibly correlated features into a smaller set of uncorrelated features called principal components, while preserving as much variance (information) as possible.

How PCA Works

1. Standardize the data
2. Compute the covariance matrix
3. Calculate eigenvectors and eigenvalues of the covariance matrix
4. Sort eigenvectors by their eigenvalues in descending order
5. Select the top k eigenvectors as the new feature axes (principal components)
6. Project the original data onto these new axes

What are Principal Components?

Each principal component is a linear combination of the original features. The first principal component captures the maximum possible variance in the data, the second captures the next highest variance while being orthogonal (uncorrelated) to the first, and so on.

Implementing PCA in Python

from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler

X_scaled = StandardScaler().fit_transform(X)

pca = PCA(n_components=2)
X_pca = pca.fit_transform(X_scaled)

print("Explained Variance Ratio:", pca.explained_variance_ratio_)

Explained Variance

The explained variance ratio tells you how much of the original data's variance is captured by each principal component - this helps decide how many components to keep while still retaining most of the information.

Coming Up Next

Now that you understand how PCA works mathematically, let's look at where it's actually applied in real-world Data Science projects.