Q22Machine Learning
Question
Explain Principal Component Analysis (PCA) algorithm step-by-step. How does it compute the principal components?
Answer
PCA performs eigenvalue decomposition on the covariance matrix to find the directions of maximum variance.
### Introduction to PCA PCA is an unsupervised linear dimensionality reduction technique. It transforms the original variables into a new set of orthogonal variables called Principal Components, ordered by the amount of variance they explain.
### Algorithm Steps 1. Standardization: Scale the data to have a mean of 0 and variance of 1. 2. Covariance Matrix: Compute the covariance matrix of the standardized data to understand feature relationships. 3. Eigen Decomposition: Calculate the eigenvalues and eigenvectors of the covariance matrix. The eigenvectors represent the directions (principal components), and eigenvalues represent their magnitude (variance explained). 4. Sort and Select: Sort the eigenvectors by decreasing eigenvalues. Choose the top k eigenvectors to form a projection matrix W. 5. Transform Data: Multiply the original standardized dataset by W to obtain the new k-dimensional feature subspace.
