Machine learning builds programs that improve at a task from data instead of from hand-written rules. You supply examples; a learning algorithm finds the parameters of a model that generalises to examples it has never seen.
Rules vs learning
In traditional programming a person writes the rules. In ML the rules — the model's parameters — are fitted from examples of inputs paired with the right answers.
Data + hand-written rules → Program → Answers
Data + known answers → Learning algorithm → Model (learned rules) → Answers for new data
A tiny example: the algorithm is never told the rule y = 3x + 2; it recovers it from 50 noisy examples.
import numpy as np
from sklearn.linear_model import LinearRegression
rng = np.random.default_rng(0)
X = rng.uniform(0, 10, size=(50, 1)) # feature: hours studied
y = 3 * X[:, 0] + 2 + rng.normal(0, 1, size=50) # target: score, with noise
model = LinearRegression().fit(X, y) # training = fitting parameters
print(round(model.coef_[0], 2), round(model.intercept_, 2)) # 3.05 1.76 ← close to the true 3 and 2
print(round(model.predict([[4.0]])[0], 1)) # 14.0 ← inference on a new input (true value 14)The supervised setting
y = f(x) + ε learn f̂ ≈ f from pairs (x₁, y₁), …, (xₙ, yₙ)
f is the unknown true relationship and ε is noise no model can predict (measurement error, missing information). Even a perfect f̂ keeps the error caused by ε — the irreducible error. This picture covers supervised learning; unsupervised learning has no y at all (see Types of Learning).
Vocabulary
| Term | Meaning | House-price example |
|---|---|---|
| Example / sample / row | One observation | One house sale |
Feature (input, predictor) x | A measurable input variable | Area, bedrooms, distance to metro |
Label / target y | The value to predict (supervised learning only) | Sale price |
| Model | A function f̂ from features to predictions | price ≈ w₁·area + w₂·bedrooms + b |
| Parameters | Learned from data during training | The weights w₁, w₂, b |
| Hyperparameters | Chosen before training, tuned on validation data | Regularisation strength, tree depth |
| Training / fitting | Adjusting parameters to reduce a loss on training data | model.fit(X, y) |
| Inference / prediction | Applying the trained model to new inputs | model.predict(X_new) |
| Generalisation | Performing well on unseen data from the same distribution | Accurate prices for next month's listings |
AI, ML and deep learning
Artificial Intelligence → Machine Learning → Deep Learning
| Field | What it is | Example |
|---|---|---|
| Artificial intelligence | Any technique that makes machines act intelligently — including search, logic and planning that learn nothing from data | A route planner using A* search |
| Machine learning | The part of AI that learns behaviour from data | Gradient-boosted trees predicting customer churn |
| Deep learning | ML with many-layered neural networks that learn their own features from raw data | A transformer transcribing speech |
When ML is — and isn't — the right tool
The workflow at a glance
Frame the problem → Collect & explore data → Split train / validation / test → Prepare features → Train a baseline, then better models → Tune & validate → Test once → Deploy & monitor
Each box has its own topic in these notes. The split comes before any preprocessing is fitted, and the test set is used once at the end — breaking either rule causes data leakage.
What is the difference between a parameter and a hyperparameter?
What does it mean for a model to generalise?
How is ML different from traditional programming?