Concepts from classical statistics and data-driven learning that underpin modern AI.

Core ML paradigms

Three Core Task Types

Training vs Testing
Training data is the subset used to fit the model parameters. Testing data is a separate holdout set used to evaluate how well the model generalizes to unseen data.
Overfitting vs Underfitting
Overfitting occurs when a model learns the training data's noise and details too well, failing to generalize to new data. Underfitting occurs when the model is too simple to capture the underlying pattern.
Bias vs Variance
Bias is error from erroneous assumptions in the algorithm (leads to underfitting). Variance is error from sensitivity to small fluctuations in the training set (leads to overfitting).
Features vs Labels
Features are individual inputs or variables fed into the model (e.g., house size, age). Labels are the ground-truth targets or outputs the model is trying to predict (e.g., house price).
Loss Function
A mathematical function that measures how far the model's predictions are from the true labels. The goal of training is to minimize this loss. Examples: Mean Squared Error (MSE), Cross-Entropy.
Epoch vs Batch
A batch is the subset of training samples passed through the network in one forward/backward pass. An epoch represents one complete pass of the entire training dataset.
Gradient Descent
An optimization algorithm used to minimize the loss function by iteratively updating the model's weights in the opposite direction of the gradient (steepest ascent) of the loss curve.