Machine Learning

Supervised and unsupervised learning

Two families. The difference is whether your data already carries the answer.

• Supervised: every row has a label – this claim was fraud, this customer left, this defect was severity 1. The model learns to reproduce the label.
• Unsupervised: no labels. The model finds structure – groups of similar customers, unusual transactions.
• Supervised splits again into classification (which category) and regression (how much).
• Getting labels is usually the hard part. Months of manual tagging sits behind most working models.

Linear and logistic regression

The two models to learn first. They are simple, fast and – importantly – explainable.

• Linear regression predicts a number: price, delivery time, premium.
• Logistic regression predicts a probability of a class: will churn / will not.
• Both give you coefficients – how much each input pushed the answer. Regulators like that.
• They are the baseline. If a complicated model cannot beat logistic regression, it is not worth the extra risk.

from sklearn.linear_model import LinearRegression, LogisticRegression

# how much? -> linear regression
price = LinearRegression().fit(X_train[["sqft", "rooms"]], y_price_train)
print(price.coef_, price.intercept_)        # the formula, readable

# yes or no? -> logistic regression
churn = LogisticRegression(max_iter=1000).fit(X_train, y_churn_train)
print(churn.predict_proba(X_test)[:3])      # probability per class

Decision trees, random forests and support vector machines

The workhorses for ordinary table data – and usually the strongest choice.

• A decision tree is a flowchart of if-then splits learned from data. Readable, but it memorises the training set if you let it grow.
• A random forest trains many trees on random subsets and takes the majority vote. Much more stable; the standard first serious attempt.
• Gradient boosting (XGBoost, LightGBM) builds trees that correct the previous one’s mistakes – usually the winner on tabular data.
• Support vector machines find the widest boundary between classes. Good on small, clean, high-dimensional data such as text classification.
• For rows and columns, trees beat neural networks far more often than beginners expect.

K-means clustering

The unsupervised one you will meet most often: split the data into k groups of similar rows.

• You choose k. There is no correct answer – try several and look at whether the groups mean anything to the business.
• Scale your features first. Without it, a column measured in rupees dominates one measured in days.
• Clusters have no names. Someone has to look at each group and decide it means “lapsed high spenders”.
• Re-running on new data can produce different groups – cluster numbers are not stable identities.

from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

X = StandardScaler().fit_transform(df[["orders", "avg_value", "days_since_last"]])
km = KMeans(n_clusters=4, n_init=10, random_state=42).fit(X)

df["segment"] = km.labels_
print(df.groupby("segment")[["orders", "avg_value", "days_since_last"]].mean())

Classification and regression in practice

Choosing the right shape of problem – and noticing when someone has chosen wrongly.

• Ask what the business will do with the answer. A number to plan with means regression; a decision to take means classification.
• A probability is often more useful than a hard class – it lets the business set the threshold.
• Imbalanced classes (1% fraud) need care: stratified splits, class weights, and never accuracy as the headline metric.
• Turning a number into buckets throws information away. Do it only if the business genuinely works in buckets.

Feature engineering and data preprocessing

Where most of the real improvement comes from – more than from switching algorithms.

• Encoding: models need numbers. One-hot encoding for categories without an order, ordinal only when the order is real.
• Scaling: required for distance-based models (k-means, SVM, neural networks), irrelevant for trees.
• Derived features carry the domain knowledge: value per item, days since last order, hour of day.
• Fit your encoder and scaler on the training set only, then apply to validation and test – otherwise you have leaked.

df["order_hour"] = pd.to_datetime(df["created_at"]).dt.hour
df["is_weekend"] = pd.to_datetime(df["created_at"]).dt.dayofweek >= 5
df["value_per_item"] = df["order_value"] / df["item_count"].clip(lower=1)
df["days_since_last"] = (pd.Timestamp.now() - pd.to_datetime(df["last_order"])).dt.days

X = pd.get_dummies(df[["city", "order_hour", "is_weekend", "value_per_item"]], columns=["city"])

Overfitting, underfitting and cross-validation

The central failure mode: a model that has memorised instead of learned.

• Overfitting: excellent on training data, poor on new data. It learned the noise.
• Underfitting: poor on both. The model is too simple, or the features carry no signal.
• The tell-tale sign is a wide gap between training and validation scores.
• Cross-validation splits the data k ways and trains k times, so your score does not depend on one lucky split.
• Fixes: more data, fewer features, simpler model, regularisation, early stopping.

from sklearn.model_selection import cross_val_score

scores = cross_val_score(model, X_train, y_train, cv=5, scoring="f1")
print(scores, "mean", round(scores.mean(), 3), "spread", round(scores.std(), 3))
# a big spread means the result depends on which rows you happened to train on

Model evaluation: accuracy, precision, recall, F1 and ROC-AUC

The vocabulary you need to challenge a number. Learn this section properly – it is the one you will use in meetings.

• Accuracy = how many were right overall. Useless when classes are imbalanced.
• Precision = of the ones we flagged, how many were real. Low precision annoys honest customers.
• Recall = of the real ones, how many we caught. Low recall lets fraud through.
• F1 balances precision and recall in one number.
• ROC-AUC measures ranking quality across all thresholds – useful for comparing models, not for choosing a cut-off.
• Always ask for the confusion matrix: true positives, false positives, true negatives, false negatives. Every argument about a model ends there.

from sklearn.metrics import classification_report, confusion_matrix

pred = model.predict(X_test)
print(confusion_matrix(y_test, pred))
print(classification_report(y_test, pred, digits=3))

Introduction to scikit-learn

One library, one shape: fit, predict, score. Learn the pattern once and every model follows it.

• Every estimator has .fit(), .predict() and usually .predict_proba().
• A Pipeline bundles preprocessing with the model so the same steps run in training and in production. This prevents a whole class of bugs.
• joblib.dump saves the pipeline; the serving code loads it and calls predict.
• Pin your library versions. A model saved with one scikit-learn version may refuse to load in another.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report

pipe = Pipeline([
    ("scale", StandardScaler()),
    ("model", RandomForestClassifier(n_estimators=200, random_state=42)),
])

pipe.fit(X_train, y_train)                      # learn
print(classification_report(y_test, pipe.predict(X_test)))

import joblib
joblib.dump(pipe, "churn_model.joblib")         # ship the whole pipeline, not just the model

Project: predict customer churn or classify transactions

1. Use the cleaned dataset from Module 3. Pick a label that already exists in the data.
2. Split 70/15/15 with stratify. Train logistic regression first – that is your baseline.
3. Train a random forest. Print both classification reports side by side.
4. Print the confusion matrix and decide, in business terms, whether a false positive or a false negative costs more. Move the threshold accordingly and show the new matrix.
5. Write five sentences a manager could read, ending with the one risk you would flag before go-live.