Q18Machine Learning

A standard Multilayer Perceptron (MLP) fails catastrophically at Image Classification. If you input a $1000\times 1000$ pixel RGB image into an MLP, the first hidden layer requires 3,000,000 weights PER NODE. The network will instantly run out of RAM and violently overfit. Furthermore, MLPs completely destroy spatial structure (flattening the 2D image into a 1D line). Convolutional Neural Networks (CNNs) are a highly specialized architecture mathematically engineered to preserve 2D spatial relationships and drastically slash the number of parameters using weight sharing.

A CNN executes a rigid architectural pipeline: Convolution $\to$ Activation (ReLU) $\to$ Pooling $\to$ Fully Connected.

1. The Convolutional Layer (Feature Extraction)

This is the absolute core mathematical engine. Instead of dense weights, the CNN utilizes small $3\times 3$ or $5\times 5$ mathematical matrices called Filters (Kernels).

• The Sliding Window: The $3\times 3$ Filter physically slides (convolves) across the massive image matrix, moving by a set Stride (e.g., 1 pixel at a time).
• The Dot Product: At every stop, it mathematically calculates the dot product between the 9 filter weights and the 9 underlying image pixels.
• The Feature Map: The output of this sliding dot product is a brand new 2D matrix called a Feature Map. If the network uses 64 filters, it generates 64 distinct Feature Maps.
• Architectural Genius (Weight Sharing): A single $3\times 3$ filter only has 9 mathematical parameters, yet it processes the entire $1000\times 1000$ image. The CNN looks for the exact same feature (e.g., a vertical edge) everywhere in the image using the same 9 weights, providing massive translation invariance.

2. The Activation Layer (ReLU)

Every single number in the resulting Feature Map is violently forced through a Rectified Linear Unit (ReLU) function: $f(x)=\max (0,x)$. It mathematically annihilates all negative numbers to 0, injecting critical non-linearity into the network without triggering the vanishing gradient catastrophe.

3. The Pooling Layer (Downsampling)

After convolution, the 64 Feature Maps contain too much redundant data. The network executes Max Pooling.

• A $2\times 2$ window with a Stride of 2 slides across the Feature Map.
• It looks at 4 pixels and violently extracts ONLY the absolute maximum value, discarding the other 3.
• This mathematically shrinks the spatial dimensions of the image by exactly 75%, slashing computational load and aggressively controlling overfitting, while preserving the dominant features.

4. The Fully Connected (Dense) Layer (Classification)

After passing through multiple blocks of (Conv $\to$ ReLU $\to$ Pool), the image has been mathematically reduced from a $1000\times 1000$ pixel array into a deep, tiny $7\times 7\times 512$ block of highly abstract semantic features (e.g., "eyes", "wheels").

This final 3D block is violently Flattened into a single 1D vector and fed into a standard MLP (Dense Layer). The final output layer utilizes a Softmax mathematical function to output a strict probability distribution (e.g., 98% Cat, 2% Dog).