Q16Principles of Artificial Intelligence

A Decision Tree is a highly aggressive, non-parametric Supervised Machine Learning algorithm used for both Classification and Regression. Unlike Neural Networks, which are mathematical black boxes, a Decision Tree creates a massive, visually interpretable flowchart. The internal nodes represent strict mathematical tests on specific features (e.g., Age > 30?), the branches represent the outcome (True/False), and the leaf nodes represent the final predicted class.

The Mathematical Role in Learning

The learning process is an algorithm (like ID3 or CART) that violently attempts to construct the absolute smallest possible tree that perfectly separates the training data.

• Entropy (Chaos): The algorithm evaluates a node containing 50 Yes and 50 No labels. This node has absolute maximum Chaos (Entropy = 1.0). The algorithm's mathematical goal is to completely annihilate this entropy.
• Information Gain: The AI tests every single feature in the dataset (e.g., Gender, Income). It mathematically calculates how much splitting the data by Gender reduces the Entropy in the resulting child nodes. The feature that provides the absolute highest Information Gain (the biggest drop in chaos) is violently selected as the Root Node.
• Recursive Partitioning: The algorithm aggressively repeats this mathematical splitting process on every child node, continuing deeper and deeper until every single leaf node is absolutely "Pure" (contains strictly 100% Yes or 100% No).

Architectural Merits and Demerits

The model is mathematically beautiful and requires zero feature scaling. However, if unconstrained, the algorithm will violently Overfit, growing a massively deep tree that perfectly memorizes the training data but fails catastrophically on new data. This is countered by mathematically "Pruning" the deepest branches.