Q21Machine Learning
Question
Elaborate on Support Vector Machines (SVM). Explain margins, hyperplanes, and the optimization problem involved.
Answer
SVM finds the optimal hyperplane that maximizes the margin between classes using support vectors.
### Hyperplanes and Margins In an N-dimensional space, a hyperplane is a flat affine subspace of dimension N-1. SVM seeks the hyperplane that separates the classes with the maximum margin, which is the distance between the hyperplane and the nearest data points from either class. These nearest points are called Support Vectors.
### The Optimization Problem For linearly separable data, the goal is to maximize the margin, which mathematically converts to a constrained convex optimization problem. We use Lagrange multipliers to solve it. The dual formulation allows the use of kernels for non-linear decision boundaries.
### Soft Margin When data is not perfectly separable, SVM introduces slack variables to allow for some misclassifications, balancing the trade-off between maximizing the margin and minimizing the classification error using a regularization parameter C.
