Platform Engineering · Lesson 210

Deep Learning with PyTorch and TensorFlow

Lesson 181 trained a forest on 2,000 past deploys and read its confusion matrix. The forest caught 8 of 21 incidents and never cried wolf. This lesson takes the same 2,000 deploys and trains a network on them instead. A network is not a smarter forest. It is a different machine: layers of weighted sums, a bend at each one, a loss that says how wrong it is, and a gradient that says which way to move. I build that machine from first principles, by hand, on a tiny network, before I let a framework near it. Then I train it twice, once in PyTorch and once in Keras, on the same data, and read the threshold the same way Lesson 181 taught me. Then I take a model someone else trained, freeze its base, retrain its head on 200 rows, and watch it work. And I break it once on purpose, by fine-tuning everything on those 200 rows, so I can see what forgetting looks like when it is measured.

Difficulty: Advanced · Know this first: Lesson 181, Lesson 179, Lesson 75

What you will build

By the end of this lesson you will have worked backpropagation by hand on a 2-2-1 network and then run the same idea in numpy. Second, you will have trained an 8-32-1 classifier in PyTorch on the same 2,000 deploys Lesson 181 used, and read its confusion matrix, precision, recall, F1, and threshold sweep beside the forest's numbers. Third, you will have built the same model in Keras and saved it in the SavedModel format, and you will know when TF Serving is the pick and when a plain PyTorch export is. Fourth, you will have done transfer learning on a small image task: frozen base, retrained head, and the failure mode where fine-tuning everything on tiny data forgets the base task.

What you need: Python 3.12, a virtual environment, pip install torch --index-url https://download.pytorch.org/whl/cpu for the PyTorch steps, pip install tensorflow-cpu for the Keras step, pip install scikit-learn matplotlib for the data and the plots. Nothing else. No cloud account, no live system, no GPU. The data is synthetic, the scripts run on your own machine, and the outputs shown are the outputs as they ran.

The story: the same deploys, a different machine

Every deploy leaves a trail, and Lesson 181 turned that trail into 2,000 rows: 8 numbers per deploy, and a label that is 1 when the deploy caused an incident. The forest read those rows by voting. A network reads them by stacking layers. Each layer is a weighted sum followed by a bend, and the bend is what lets the network draw a curved boundary where a straight line fails.

PieceThe question it answersThe ops shape of it
One neuronWhat does one unit computeA weighted sum of its inputs, plus a bias, then a bend
A layerWhat does a stack computeMany neurons at once, one matrix multiply, one bend per neuron
ActivationWhere does the curve come fromWithout the bend, the whole network is one straight line
LossHow wrong is it, in one numberBinary cross-entropy on the deploy label, same 0.5 default as Lesson 181
BackpropagationWhich weight moves, and which wayThe gradient, one derivative per weight, run backwards through the layers
The training loopHow does it learnForward, loss, backward, step, repeat; PyTorch writes it out, Keras hides it
Transfer learningWhat do I do with 200 rowsFreeze the base, retrain the head; never train from scratch on tiny data

The maths behind the gradient lives in Lesson 212. If the derivative feels like a new word, read that lesson first: the gradient is a slope, and backpropagation is the chain rule run backwards. This lesson points there. It does not re-derive the chain rule.

Do this now, before the first hop

Write down the Lesson 181 forest numbers from memory. Precision 1.000, recall 0.381, F1 0.552. The forest caught 8 of 21 incidents. Keep those numbers beside you. Every network number in this lesson is read against them.

Run it as:

When it breaks, I check

One thing to remember

Four paths, one family

The diagram above follows one deploy-risk question from one neuron to a shipped model. These four paths are the same idea drawn four ways. One clean run. One overfit run, stopped at the training loop. One tiny-data run, where the frozen base saves it. One numbers trail, hop by hop. Run each path. One run takes 7.2 seconds across the path, so you can watch each hop light up in turn.

The return rule: the green dot only starts moving after the orange dot reaches the data hop. A return with no finished work is a lie. The diagram never draws one. The failure replay at the end of Run all four shows the other side of the rule. Run it and watch the orange dot stop at the training loop and never come back.

Path 1: the clean run

Nothing skipped

One question, 2,000 deploys. The loss falls, the threshold holds, the transfer works, and the model ships. The green dot starts back from the serving pick, because that is where the finished work lives.

One neuron0 ms
A layer4 ms
The loss6 ms
Backprop10 ms
Train loop14 ms
Keras12 ms
Threshold8 ms
Transfer16 ms
Serving pick18 ms

Not run yet.

Path 2: the overfit run, stopped at the training loop

Everything after: skipped

Same run, trained far past the point where the validation loss stops falling. The training loss falls to nearly zero and the validation loss climbs. The run stops at the training loop, and no green dot ever starts back, because no finished work exists.

One neuron0 ms
A layer4 ms
The loss6 ms
Backprop10 ms
Train loop30 ms
Stopped hereno verdict

Not run yet.

Path 3: the tiny-data run

From-scratch training: skipped

Same network, only 200 training rows. The loss barely moves from scratch, the threshold read is noise, and the frozen base saves the run. The green dot starts back from the transfer hop, because the frozen base is the finished work here.

One neuron0 ms
A layer4 ms
The loss6 ms
Backprop10 ms
Train loop22 ms
Threshold14 ms
Transfer28 ms
Serving pick18 ms

Not run yet.

Path 4: the numbers trail

Nothing skipped

The same question, measured instead of timed. Train loss 0.69 to 0.08, precision 0.85, recall 0.52, 321 parameters. The trail adds up to 88 ms, and the numbers are the check that keeps the claim honest.

Train loss0.69 to 0.08
Precision0.85
Recall0.52
Params321
Verdict88 ms

Not run yet.

Where the time goes

Every hop charges rent in milliseconds. The bar below is a worked example for one training run on 2,000 past deploys, not a measurement from a live system, but the shape is honest. The serving pick is the big hop at 18 ms, and the total is 88 ms. Switch the scenario and watch the bar change. In the overfit story the training loop hop grows to 30 ms and the total to 116 ms, because the network trains long past the point where the validation loss stops falling. In the tiny-data story the transfer hop grows to 28 ms and the total to 114 ms, because 200 rows make the frozen base carry the lesson. Move a simulator slider and watch the hops, the verdict, and the total move with it.

The network, the step, and the data: move the sliders and watch the training bend

The budget bar above shows three fixed stories. This simulator lets you write your own. It answers the sizing question at the heart of this lesson: how wide must the network be, how big may each step be, how long must it train, and how much data does it need. You set the hidden units, the learning rate, the epochs, and the training rows. Move a slider and watch the parameter count, the training steps, the train loss, the validation loss, the precision, the recall, and the budget total change. The numbers are a teaching example, built from the formulas below, not live measurements. The shape is the lesson: tiny data owns the validation gap, a small step owns the recall, and the effect you want the network to learn decides whether the training is worth running.

32 units

How wide the hidden layer is. More units can draw a curvier boundary, and more units can memorise the training rows.

0.010

How big each downhill step is, in thousandths. 0.010 is the default. Too small and the loss crawls. Too big and the loss jumps over the minimum.

10 epochs

How many full passes over the training rows. More passes teach more, until the validation loss stops falling and the extra passes only memorise.

2,000 rows

How many deploys the network learns from. Fewer rows make every weight harder to pin down, and the validation gap opens up.

Try this: (1) At the defaults, 32 hidden units, learning rate 0.010, 10 epochs, 2,000 rows, the network has 321 parameters, trains in 630 steps, the train loss is 0.24, the validation loss is 0.32, precision is 0.90, recall is 0.80, and the budget total is 88 ms. (2) Set the training rows to 200. The steps fall to 70, the train loss stays 0.24, the validation loss climbs to 0.58, precision falls to 0.61, recall falls to 0.45, and the budget total is 101 ms. The status line turns red. Two hundred rows cannot teach 321 weights from scratch. (3) Back to 2,000 rows, set the learning rate to 0.001. The train loss is 0.46, the validation loss is 0.54, precision is 0.90, recall is 0.40, and the budget total is 94 ms. The steps are too small, and ten epochs are not enough. (4) Learning rate back to 0.010, epochs to 30, hidden units to 128. The network has 1,281 parameters, trains in 1,890 steps, the train loss is 0.24, the validation loss is 0.42, precision is 0.90, recall is 0.80, and the budget total is 109 ms. The gap opened, and the extra epochs only memorised. (5) Hidden units to 64, learning rate to 0.050, epochs to 20. The network has 641 parameters, trains in 1,260 steps, the train loss is 0.40, the validation loss is 0.48, precision is 0.90, recall is 0.52, and the budget total is 99 ms. Bigger steps and more epochs, and the recall is still only 0.52. The learning rate is doing the damage, not the data.

Formulas, in plain terms. Parameters = 10 times hidden plus 1. Steps = epochs times training rows over 32, rounded up per epoch. Capacity = hidden over 32, capped at 1. Learning-rate fit = 1 minus the absolute log10 of the learning rate over 0.01, times 0.5, clamped between 0.1 and 1. Data factor = training rows over 1,600, capped at 1. Train loss = 0.69 minus 0.45 times capacity times learning-rate fit times epochs over 10, capped at 1. Validation gap = 0.08 plus 0.30 times one minus data factor times epochs over 10, capped at 1, plus 0.10 when hidden is above 64. Validation loss = train loss plus the gap. Precision = 0.90 minus 0.30 times one minus data factor, minus 0.15 times the gap above 0.15. Recall = 0.80 times capacity times learning-rate fit times 0.5 plus 0.5 times data factor. Train hop = 4 plus steps over 63, rounded. Keras hop = 12 plus parameters over 1,000, rounded. Threshold hop = 8, plus 6 when recall is below 0.5, plus 4 when precision is below 0.7. Transfer hop = 16, plus 12 when training rows are below 500. Neuron 0, layer 4, loss 6, backprop 10, serving 18. Budget total = the nine hop times added up. The base loss of 0.69 and the starting precision of 0.90 are teaching figures. Your system will differ. The shape will not.

Lab: run the network yourself, twice, then transfer it

Four scripts, all run on your own machine. The outputs are shown exactly as they ran. No cloud account, no live system, no GPU. Your numbers will differ when you use your own data. The maths will not.

Step 0: one network, by hand, in numpy

Save this file as fcn_numpy.py. It builds a 2-2-1 network with no framework, just numpy, and writes backpropagation out by hand: the output error, carried back through the weights, times the hidden bend. The task is small on purpose: label 1 when x0 plus x1 is above zero, label 0 when it is below. It trains for 500 epochs and prints the loss every 100 epochs.

"""Lesson 210 lab step 0: a fully connected network, from first principles.

No framework. Just numpy, two inputs, two hidden neurons, one output,
and backpropagation written out by hand. The task is small on purpose:
label 1 when x0 + x1 is above zero, label 0 when it is below. The network
has to learn the diagonal line, and the bend in each neuron is what lets it.

The gradient is a slope. Lesson 212 proves the chain rule this code uses.
Here the chain rule is four lines: the output error, times the output
bend, carried back through the weights, times the hidden bend.

Run it: python3 fcn_numpy.py
"""

import numpy as np

rng = np.random.default_rng(7)

# 200 rows, two features. The label is 1 when the sum is above zero.
X = rng.standard_normal((200, 2))
y = (X[:, 0] + X[:, 1] > 0).astype(float).reshape(-1, 1)

# 2 inputs -> 2 hidden -> 1 output. Small weights, so the first loss is honest.
W1 = rng.standard_normal((2, 2)) * 0.5
b1 = np.zeros((1, 2))
W2 = rng.standard_normal((2, 1)) * 0.5
b2 = np.zeros((1, 1))


def sigmoid(z):
    return 1.0 / (1.0 + np.exp(-z))


def forward(X):
    h = sigmoid(X @ W1 + b1)
    out = sigmoid(h @ W2 + b2)
    return h, out


def loss_and_grads(X, y):
    h, out = forward(X)
    n = len(y)
    eps = 1e-12
    out_c = np.clip(out, eps, 1 - eps)
    loss = -np.mean(y * np.log(out_c) + (1 - y) * np.log(1 - out_c))
    # Backpropagation, by hand. The output error is out minus y.
    d_out = (out - y) / n
    dW2 = h.T @ d_out
    db2 = np.sum(d_out, axis=0, keepdims=True)
    # Carried back through W2, times the hidden bend: h * (1 - h).
    d_h = (d_out @ W2.T) * h * (1 - h)
    dW1 = X.T @ d_h
    db1 = np.sum(d_h, axis=0, keepdims=True)
    return loss, dW1, db1, dW2, db2


lr = 0.5
for epoch in range(501):
    loss, dW1, db1, dW2, db2 = loss_and_grads(X, y)
    if epoch % 100 == 0:
        print(f"Epoch {epoch}: loss {loss:.4f}")
    W1 -= lr * dW1
    b1 -= lr * db1
    W2 -= lr * dW2
    b2 -= lr * db2

_, out = forward(X)
pred = (out > 0.5).astype(float)
acc = float(np.mean(pred == y))
print(f"Final loss: {loss:.4f}")
print(f"Accuracy on 200 rows: {acc:.3f}")
print("W1 (2 inputs x 2 hidden):")
print(np.round(W1, 3))
print("W2 (2 hidden x 1 output):")
print(np.round(W2, 3))
python3 fcn_numpy.py

Expected output, exactly as it ran on this machine:

Epoch 0: loss 0.7138
Epoch 100: loss 0.4022
Epoch 200: loss 0.1418
Epoch 300: loss 0.0904
Epoch 400: loss 0.0701
Epoch 500: loss 0.0592
Final loss: 0.0592
Accuracy on 200 rows: 0.995
W1 (2 inputs x 2 hidden):
[[ 2.368 -3.67 ]
 [ 2.309 -3.721]]
W2 (2 hidden x 1 output):
[[ 4.046]
 [-6.565]]

What to look for: the loss falls from 0.7138 to 0.0592, and the accuracy is 0.995 on 200 rows. The two hidden weights in the first row, 2.368 and 2.309, both point the same way, because the task is a sum. The second row points the other way, -3.67 and -3.721. Two neurons, one line, learned from scratch. That is the whole machine, with no framework hiding any of it.

Step 1: train the deploy classifier in PyTorch

Save this file as train_torch.py. It trains an 8-32-1 network on the same 2,000 deploys Lesson 181 used, with the same split and the same seed, so the two models are compared on the same test set. Adam at 0.01, ten epochs, batch 32. It prints the loss per epoch, the confusion matrix, the four numbers, and the threshold sweep, and it saves a plot, loss_torch.png.

"""Lesson 210 lab step 1: the same 2,000 deploys, trained in PyTorch.

The data is the same data Lesson 181 used: make_classification with
2,000 rows, 8 features, 5 percent incidents, random_state 42, split 80/20
stratified. The forest on this data scored precision 1.000, recall 0.381,
F1 0.552, accuracy 0.968, ROC AUC 0.930. This script trains an 8-32-1
network on the same rows and reads the same four numbers, so the two
models are compared on the same test set, not on two different stories.

The loop is four lines, repeated: forward, loss, backward, step.
A tensor is a number with a shape and a history. Autograd records every
operation, so the backward call knows how the loss was built.

Run it: python3 train_torch.py
Needs: pip install torch --index-url https://download.pytorch.org/whl/cpu
        pip install scikit-learn matplotlib
"""

import numpy as np
import torch
import torch.nn as nn
from sklearn.datasets import make_classification
from sklearn.metrics import (accuracy_score, confusion_matrix, f1_score,
                             precision_score, recall_score, roc_auc_score)
from sklearn.model_selection import train_test_split

torch.manual_seed(42)
np.random.seed(42)

X, y = make_classification(n_samples=2000, n_features=8, n_informative=5,
                           n_redundant=1, weights=[0.95, 0.05],
                           random_state=42)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y)

print(f"Train rows: {len(y_train)}, test rows: {len(y_test)}")
print(f"Incidents in train: {int(y_train.sum())} of {len(y_train)}")
print(f"Incidents in test: {int(y_test.sum())} of {len(y_test)}")

Xtr = torch.tensor(X_train, dtype=torch.float32)
ytr = torch.tensor(y_train, dtype=torch.float32).reshape(-1, 1)
Xte = torch.tensor(X_test, dtype=torch.float32)
yte = torch.tensor(y_test, dtype=torch.float32).reshape(-1, 1)


class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.hidden = nn.Linear(8, 32)
        self.out = nn.Linear(32, 1)

    def forward(self, x):
        x = torch.relu(self.hidden(x))
        return torch.sigmoid(self.out(x))


model = Net()
opt = torch.optim.Adam(model.parameters(), lr=0.01)
loss_fn = nn.BCELoss()

n_params = sum(p.numel() for p in model.parameters())
print(f"Parameters: {n_params} (8x32 + 32 + 32x1 + 1)")

batch = 32
losses = []
for epoch in range(10):
    model.train()
    perm = torch.randperm(len(Xtr))
    epoch_loss = 0.0
    for i in range(0, len(Xtr), batch):
        idx = perm[i:i + batch]
        xb, yb = Xtr[idx], ytr[idx]
        pred = model(xb)              # forward
        loss = loss_fn(pred, yb)      # loss
        opt.zero_grad()
        loss.backward()               # backward
        opt.step()                    # step
        epoch_loss += loss.item() * len(idx)
    epoch_loss /= len(Xtr)
    losses.append(epoch_loss)
    print(f"Epoch {epoch + 1}: loss {epoch_loss:.4f}")

model.eval()
with torch.no_grad():
    proba = model(Xte).numpy().flatten()
pred = (proba > 0.5).astype(int)

print("Confusion matrix, rows are the truth, columns are the prediction:")
print(confusion_matrix(y_test, pred))
print(f"Precision: {precision_score(y_test, pred):.3f}")
print(f"Recall: {recall_score(y_test, pred):.3f}")
print(f"F1 score: {f1_score(y_test, pred):.3f}")
print(f"Accuracy: {accuracy_score(y_test, pred):.3f}")
print(f"ROC AUC: {roc_auc_score(y_test, proba):.3f}")

for t in [0.3, 0.5, 0.7, 0.9]:
    p = (proba > t).astype(int)
    print(f"Threshold {t}: flagged {int(p.sum())} of {len(y_test)}, "
          f"precision {precision_score(y_test, p):.3f}, "
          f"recall {recall_score(y_test, p):.3f}, "
          f"F1 {f1_score(y_test, p):.3f}")

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

plt.figure()
plt.plot(range(1, 11), losses, marker="o")
plt.xlabel("Epoch")
plt.ylabel("Loss")
plt.title("PyTorch training loss, 8-32-1 on 2,000 deploys")
plt.savefig("loss_torch.png")
print("Saved loss_torch.png")
python3 train_torch.py

Expected output, exactly as it ran on this machine:

Train rows: 1600, test rows: 400
Incidents in train: 83 of 1600
Incidents in test: 21 of 400
Parameters: 321 (8x32 + 32 + 32x1 + 1)
Epoch 1: loss 0.2212
Epoch 2: loss 0.1322
Epoch 3: loss 0.1178
Epoch 4: loss 0.1060
Epoch 5: loss 0.1009
Epoch 6: loss 0.0962
Epoch 7: loss 0.0889
Epoch 8: loss 0.0835
Epoch 9: loss 0.0797
Epoch 10: loss 0.0766
Confusion matrix, rows are the truth, columns are the prediction:
[[377   2]
 [ 10  11]]
Precision: 0.846
Recall: 0.524
F1 score: 0.647
Accuracy: 0.970
ROC AUC: 0.959
Threshold 0.3: flagged 20 of 400, precision 0.700, recall 0.667, F1 0.683
Threshold 0.5: flagged 13 of 400, precision 0.846, recall 0.524, F1 0.647
Threshold 0.7: flagged 8 of 400, precision 1.000, recall 0.381, F1 0.552
Threshold 0.9: flagged 4 of 400, precision 1.000, recall 0.190, F1 0.320
Saved loss_torch.png

What to look for: the loss falls from 0.2212 in epoch 1 to 0.0766 in epoch 10. The confusion matrix reads 377 true negatives, 2 false positives, 10 false negatives, 11 true positives. At 0.5 the network catches 11 of 21 incidents with precision 0.846. The forest caught 8 of 21 with precision 1.000. The network beats the forest on recall and loses on precision, and that is a different price, not a better model. At 0.3 the network catches 14 of 21 and cries wolf 6 times. At 0.9 it catches 4 and never cries wolf.

Step 2: the same model in Keras, and the SavedModel

Save this file as train_keras.py. It builds the same 8-32-1 model the Keras way, trains it on the same data and the same split, prints the same metrics and the same sweep, saves the model as a SavedModel directory, and loads it back in the same run to prove the saved model works. It saves a plot, loss_keras.png.

"""Lesson 210 lab step 2: the same model, the Keras way.

The same 8-32-1 shape, the same 2,000 deploys, the same split and seed
as step 1. Keras builds it in three lines, compiles it, and fits it.
The loop is still forward, loss, backward, step, but Keras writes it
for you. That is the trade: PyTorch shows you the loop, Keras hides it.

The difference that matters is the saving. Keras saves a SavedModel
directory, and TF Serving loads that directory and answers requests,
with versioning built in. This script saves the model and then loads it
back in the same run, to prove the saved model works.

Run it: python3 train_keras.py
Needs: pip install tensorflow-cpu scikit-learn matplotlib
"""

import numpy as np
import tensorflow as tf
from sklearn.datasets import make_classification
from sklearn.metrics import (accuracy_score, confusion_matrix, f1_score,
                             precision_score, recall_score, roc_auc_score)
from sklearn.model_selection import train_test_split

tf.random.set_seed(42)
np.random.seed(42)

X, y = make_classification(n_samples=2000, n_features=8, n_informative=5,
                           n_redundant=1, weights=[0.95, 0.05],
                           random_state=42)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y)

print(f"Train rows: {len(y_train)}, test rows: {len(y_test)}")
print(f"Incidents in train: {int(y_train.sum())} of {len(y_train)}")
print(f"Incidents in test: {int(y_test.sum())} of {len(y_test)}")

model = tf.keras.Sequential([
    tf.keras.Input(shape=(8,)),
    tf.keras.layers.Dense(32, activation="relu"),
    tf.keras.layers.Dense(1, activation="sigmoid"),
])
model.compile(optimizer=tf.keras.optimizers.Adam(0.01),
              loss="binary_crossentropy")
model.summary(print_fn=lambda line: print(line))
print(f"Parameters: {model.count_params()} (8x32 + 32 + 32x1 + 1)")

history = model.fit(X_train, y_train, epochs=10, batch_size=32, verbose=0)
for i, loss in enumerate(history.history["loss"], start=1):
    print(f"Epoch {i}: loss {loss:.4f}")

proba = model.predict(X_test, verbose=0).flatten()
pred = (proba > 0.5).astype(int)

print("Confusion matrix, rows are the truth, columns are the prediction:")
print(confusion_matrix(y_test, pred))
print(f"Precision: {precision_score(y_test, pred):.3f}")
print(f"Recall: {recall_score(y_test, pred):.3f}")
print(f"F1 score: {f1_score(y_test, pred):.3f}")
print(f"Accuracy: {accuracy_score(y_test, pred):.3f}")
print(f"ROC AUC: {roc_auc_score(y_test, proba):.3f}")

for t in [0.3, 0.5, 0.7, 0.9]:
    p = (proba > t).astype(int)
    print(f"Threshold {t}: flagged {int(p.sum())} of {len(y_test)}, "
          f"precision {precision_score(y_test, p):.3f}, "
          f"recall {recall_score(y_test, p):.3f}, "
          f"F1 {f1_score(y_test, p):.3f}")

model.export("deploy_risk_savedmodel")
print("Saved SavedModel to deploy_risk_savedmodel/")
loaded = tf.saved_model.load("deploy_risk_savedmodel/")
print(f"Reloaded SavedModel, signatures: {list(loaded.signatures.keys())}")

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

plt.figure()
plt.plot(range(1, 11), history.history["loss"], marker="o")
plt.xlabel("Epoch")
plt.ylabel("Loss")
plt.title("Keras training loss, 8-32-1 on 2,000 deploys")
plt.savefig("loss_keras.png")
print("Saved loss_keras.png")
python3 train_keras.py

Expected output, exactly as it ran on this machine:

Train rows: 1600, test rows: 400
Incidents in train: 83 of 1600
Incidents in test: 21 of 400
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ dense (Dense)                   │ (None, 32)             │           288 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 1)              │            33 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 321 (1.25 KB)
 Trainable params: 321 (1.25 KB)
 Non-trainable params: 0 (0.00 B)

Parameters: 321 (8x32 + 32 + 32x1 + 1)
Epoch 1: loss 0.2367
Epoch 2: loss 0.1234
Epoch 3: loss 0.1008
Epoch 4: loss 0.0902
Epoch 5: loss 0.0844
Epoch 6: loss 0.0814
Epoch 7: loss 0.0785
Epoch 8: loss 0.0769
Epoch 9: loss 0.0753
Epoch 10: loss 0.0743
Confusion matrix, rows are the truth, columns are the prediction:
[[377   2]
 [ 11  10]]
Precision: 0.833
Recall: 0.476
F1 score: 0.606
Accuracy: 0.968
ROC AUC: 0.959
Threshold 0.3: flagged 16 of 400, precision 0.812, recall 0.619, F1 0.703
Threshold 0.5: flagged 12 of 400, precision 0.833, recall 0.476, F1 0.606
Threshold 0.7: flagged 10 of 400, precision 1.000, recall 0.476, F1 0.645
Threshold 0.9: flagged 4 of 400, precision 1.000, recall 0.190, F1 0.320
Saved artifact at 'deploy_risk_savedmodel'. The following endpoints are available:

* Endpoint 'serve'
  args_0 (POSITIONAL_ONLY): TensorSpec(shape=(None, 8), dtype=tf.float32, name='keras_tensor')
Output Type:
  TensorSpec(shape=(None, 1), dtype=tf.float32, name=None)
Captures:
  129206238173264: TensorSpec(shape=(), dtype=tf.resource, name=None)
  129206238174608: TensorSpec(shape=(), dtype=tf.resource, name=None)
  129206238174800: TensorSpec(shape=(), dtype=tf.resource, name=None)
  129206238173840: TensorSpec(shape=(), dtype=tf.resource, name=None)
Saved SavedModel to deploy_risk_savedmodel/
Reloaded SavedModel, signatures: ['serve', 'serving_default']
Saved loss_keras.png

What to look for: the loss falls from 0.2367 in epoch 1 to 0.0743 in epoch 10. The confusion matrix reads 377 true negatives, 2 false positives, 11 false negatives, 10 true positives. At 0.5 the Keras model catches 10 of 21 with precision 0.833, against the PyTorch 11 of 21 with precision 0.846. Same shape, same data, same seed, and the two runs agree within a small margin. The SavedModel saved, and it reloaded with the serve and serving_default signatures. That directory is what TF Serving loads.

Step 3: transfer learning, and the forgetting it stops

Save this file as transfer.py. It trains a base model to tell all ten digits apart on 8 by 8 images, 0.971 accuracy, then gives it a new task on only 200 rows: tell 3 from 8. Three runs on the same 200 rows, 30 epochs each: frozen base with a retrained head, fine-tune everything, and from scratch. Each run prints the new task accuracy and the base task accuracy, so the forgetting is measured, not asserted.

"""Lesson 210 lab step 3: transfer learning, and the forgetting it stops.

The task is a small image task: 8 by 8 digit images, 64 numbers per image.
The base task is to tell all ten digits apart. The new task, on only
200 rows, is to tell 3 from 8, two digits that look alike. Three runs on
the same 200 rows:

1. Frozen base, retrained head. The base layers never move. Only the
   head learns the new question.
2. Fine-tune everything. Every weight moves on the same 200 rows.
3. From scratch. A fresh network, trained only on the 200 rows.

Each run prints two accuracies: the new task, and the base task. The base
task accuracy is the forgetting measurement. The base head is kept beside
the moved features, so the base task is read the same way in every run.
If it falls, the model forgot what it knew. The from-scratch run has no
base head at all, because it never learned the base task.

Run it: python3 transfer.py
Needs: pip install torch --index-url https://download.pytorch.org/whl/cpu
        pip install scikit-learn
"""

import copy

import numpy as np
import torch
import torch.nn as nn
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

torch.manual_seed(42)
np.random.seed(42)

digits = load_digits()
X = digits.data / 16.0  # pixel values 0..16, scaled to 0..1
y10 = digits.target

# The new task uses only the 3s and the 8s. Label 1 is an 8, label 0 is a 3.
mask = np.isin(digits.target, [3, 8])
X38, y38 = X[mask], (digits.target[mask] == 8).astype(int)

X_train, X_test, y10_train, y10_test = train_test_split(
    X, y10, test_size=0.25, random_state=42, stratify=y10)
X38_train, X38_test, y38_train, y38_test = train_test_split(
    X38, y38, test_size=0.25, random_state=42, stratify=y38)
print(f"Base train rows: {len(y10_train)}, base test rows: {len(y10_test)}")
print(f"New-task rows: {len(y38_train)} train, {len(y38_test)} test")
print(f"New-task rows used: 200 of {len(y38_train)}, "
      f"8s: {int(y38_train[:200].sum())} of 200")


class Base(nn.Module):
    """The base model: 64 inputs, 32 hidden units, 10 digit outputs."""

    def __init__(self):
        super().__init__()
        self.features = nn.Linear(64, 32)
        self.clf = nn.Linear(32, 10)

    def forward(self, x):
        return self.clf(torch.relu(self.features(x)))


class Head(nn.Module):
    """A new head on a base model's features: one binary output."""

    def __init__(self, base):
        super().__init__()
        self.base = base.features
        self.head = nn.Linear(32, 1)

    def forward(self, x):
        return torch.sigmoid(self.head(torch.relu(self.base(x))))


def train_clf(model, X, y, epochs, lr=0.01):
    opt = torch.optim.Adam(model.parameters(), lr=lr)
    loss_fn = nn.CrossEntropyLoss()
    Xt = torch.tensor(X, dtype=torch.float32)
    yt = torch.tensor(y, dtype=torch.long)
    for _ in range(epochs):
        model.train()
        perm = torch.randperm(len(Xt))
        for i in range(0, len(Xt), 32):
            idx = perm[i:i + 32]
            loss = loss_fn(model(Xt[idx]), yt[idx])
            opt.zero_grad()
            loss.backward()
            opt.step()
    return model


def train_bin(model, X, y, epochs, lr=0.01, freeze_base=False):
    if freeze_base:
        for p in model.base.parameters():
            p.requires_grad = False
    opt = torch.optim.Adam(
        [p for p in model.parameters() if p.requires_grad], lr=lr)
    loss_fn = nn.BCELoss()
    Xt = torch.tensor(X, dtype=torch.float32)
    yt = torch.tensor(y, dtype=torch.float32).reshape(-1, 1)
    for _ in range(epochs):
        model.train()
        perm = torch.randperm(len(Xt))
        for i in range(0, len(Xt), 32):
            idx = perm[i:i + 32]
            loss = loss_fn(model(Xt[idx]), yt[idx])
            opt.zero_grad()
            loss.backward()
            opt.step()
    for p in model.parameters():
        p.requires_grad = True
    return model


def acc10(model, X, y):
    model.eval()
    with torch.no_grad():
        pred = model(torch.tensor(X, dtype=torch.float32)).argmax(1).numpy()
    return float(np.mean(pred == y))


def acc_bin(model, X, y):
    model.eval()
    with torch.no_grad():
        proba = model(torch.tensor(X, dtype=torch.float32)).numpy().flatten()
    return float(np.mean((proba > 0.5).astype(int) == y))


def base_task_accuracy(head_model, clf_head, X, y):
    """Read the base task through the original clf head on moved features."""
    probe = Base()
    probe.features = head_model.base
    probe.clf = clf_head
    return acc10(probe, X, y)


# Step 1: train the base model on the base task, all base training rows.
base = Base()
train_clf(base, X_train, y10_train, epochs=30)
print(f"Base model, base task accuracy: {acc10(base, X_test, y10_test):.3f}")

# The new task gets only 200 rows.
X_new, y_new = X38_train[:200], y38_train[:200]

# Step 2a: frozen base, retrained head, 200 rows.
frozen = Head(copy.deepcopy(base))
train_bin(frozen, X_new, y_new, epochs=30, freeze_base=True)
print(f"Frozen base, new task accuracy: {acc_bin(frozen, X38_test, y38_test):.3f}")
print(f"Frozen base, base task accuracy: "
      f"{base_task_accuracy(frozen, base.clf, X_test, y10_test):.3f}")

# Step 2b: fine-tune everything, same 200 rows.
tuned = Head(copy.deepcopy(base))
train_bin(tuned, X_new, y_new, epochs=30, freeze_base=False)
print(f"Fine-tuned all, new task accuracy: {acc_bin(tuned, X38_test, y38_test):.3f}")
print(f"Fine-tuned all, base task accuracy: "
      f"{base_task_accuracy(tuned, base.clf, X_test, y10_test):.3f}")

# Step 2c: from scratch, same 200 rows. No base head was ever trained.
scratch = nn.Sequential(nn.Linear(64, 32), nn.ReLU(), nn.Linear(32, 1))


class Wrap(nn.Module):
    def __init__(self, m):
        super().__init__()
        self.m = m

    def forward(self, x):
        return torch.sigmoid(self.m(x))


scratch = Wrap(scratch)
opt = torch.optim.Adam(scratch.parameters(), lr=0.01)
loss_fn = nn.BCELoss()
Xt = torch.tensor(X_new, dtype=torch.float32)
yt = torch.tensor(y_new, dtype=torch.float32).reshape(-1, 1)
for _ in range(30):
    scratch.train()
    perm = torch.randperm(len(Xt))
    for i in range(0, len(Xt), 32):
        idx = perm[i:i + 32]
        loss = loss_fn(scratch(Xt[idx]), yt[idx])
        opt.zero_grad()
        loss.backward()
        opt.step()
print(f"From scratch, new task accuracy: {acc_bin(scratch, X38_test, y38_test):.3f}")
print("From scratch, base task accuracy: no base head was ever trained")
python3 transfer.py

Expected output, exactly as it ran on this machine:

Base train rows: 1347, base test rows: 450
New-task rows: 267 train, 90 test
New-task rows used: 200 of 267, 8s: 102 of 200
Base model, base task accuracy: 0.971
Frozen base, new task accuracy: 1.000
Frozen base, base task accuracy: 0.971
Fine-tuned all, new task accuracy: 0.989
Fine-tuned all, base task accuracy: 0.800
From scratch, new task accuracy: 0.989
From scratch, base task accuracy: no base head was ever trained

What to look for: the frozen base scores 1.000 on the new task and keeps 0.971 on the base task, because its weights never moved. The fine-tuned run scores 0.989 on the new task and falls to 0.800 on the base task. Thirty epochs on 200 rows rewrote the base, and the base task is the measurement that proves it. The from-scratch run scores 0.989 on the new task and never learned the base task at all. The frozen run is the ship candidate. The fine-tuned run is the proof that the freeze matters.

Check yourself

Change the hidden size in step 1 from 32 to 8 and run it again. The network has 81 parameters now instead of 321. Watch the train loss stall. Write down the final loss for both sizes. Then change it back to 32.

Clean up

Nothing to destroy. The scripts, the SavedModel directory, and the plots live on your machine, and you can run them again on the next model.

Check yourself

Three questions. No score is saved anywhere. This is just you against the material.

Basic to master

Deep learning is not one skill. It is four levels of the same job, and every team I have worked with sits on one of them. The levels are not about how many frameworks you can name. They are about whether the model you ship survives one hard question. Read them in order. Each level fixes the failure that hurt the most at the level below it.

Basic · One neuron, one layer, one loss

At the basic level you work one neuron by hand, you check the shapes, and you read the first loss against 0.69. The neuron is a weighted sum plus a bend. The layer is many neurons at once, one matrix multiply. The loss is binary cross-entropy, and 0.69 is the coin-flip line. A first loss near 0.69 means the network started honest. A first loss of 0.0 means the labels leaked. The gradient is a slope, and Lesson 212 proves the chain rule it rests on. You use the result here. You do not re-derive it.

Do this: run step 0 of the lab and write down the loss at epoch 0 and epoch 500. One page, two numbers, and the bend doing its work.

Intermediate · The loop, the threshold, and the comparison

At the intermediate level you run the PyTorch loop, you sweep the threshold from 0.3 to 0.9, and you read the four numbers beside the forest's. The loop is four lines: forward, loss, backward, step. The threshold is a price. At 0.5 the network catches 11 of 21 with precision 0.846, against the forest's 8 of 21 with precision 1.000. The network wins on recall and loses on precision. That is a different price, not a better model, and the team picks the price.

Do this: run step 1 and write one sentence per threshold: what it costs and what it buys. Then pick one threshold and say why.

Advanced · Keras, the serving pick, and the overfit audit

At the advanced level you build the same model in Keras, you save the SavedModel, you load it back in a fresh process, and you name the serving pick for your own stack. Keras and TF Serving win when the serving stack decides. PyTorch and an export win when the research code decides. Then you overfit on purpose: train past the point where the validation loss stops falling, and watch the gap open. The training loss falls to nearly zero and the validation loss climbs, and the threshold cannot fix it, because the threshold reads a model that memorised.

Do this: run step 2, then retrain step 1 for 50 epochs and write down the train loss at epoch 10 and epoch 50. The loss keeps falling. The model keeps getting worse. That is the overfit, measured.

Master · Transfer learning, and the model as a service

At the master level the model is watched like a service, and the transfer is measured on both tasks. Freeze the base, retrain the head, and read the base task accuracy beside the new task accuracy. The frozen run keeps 0.971 on the base task and scores 1.000 on the new one. The fine-tuned run falls to 0.800 on the base task. The retrain rule is set in advance: when the live precision falls below the shipped number, the model retrains on fresh data, and the old model stays versioned in the registry. A wrong prediction is priced: a missed incident costs a page and a rollback, a false alarm costs an engineer twenty minutes. The model ships when the price is written down, not before.

Do this: run step 3, write down all six accuracies, and write the retrain rule for your own system in one paragraph, with the number that triggers it. If you cannot name the number, you do not have a rule yet.

Go deeper

The gradient is the thing Lesson 212 computes

Backpropagation is the chain rule, run backwards through the network. Each weight gets its own derivative, each derivative is a limit, and the update is the learning rate times the gradient. Lesson 212 works the derivative from the limit and proves the chain rule, with the gradient as the thing this lesson's backpropagation computes. Read that lesson first, and this lesson's backprop section is a machine standing on ground you have already walked. The lesson points there. It does not reteach it.

The forest is still the right first model on a table

A network wins on images, text, and sound, and on signals too tangled for trees. On 2,000 rows and 8 numbers, the forest is simpler, faster to train, and easier to explain. The network earns its keep when the table stops being enough. This lesson trains the network on the forest's table so the comparison is honest: recall 0.524 against 0.381, precision 0.846 against 1.000. The network catches more and cries wolf more. Lesson 211 builds the CNN, the RNN, and the LSTM on top of this lesson's loop, and those are the architectures where the network stops competing with the forest and starts doing work the forest cannot do.

ReLU against sigmoid, in one paragraph

The hidden layer in this lesson uses ReLU: zero below zero, the input above it. Its gradient is 1 when the unit is on and 0 when it is off, so it never saturates the way a sigmoid does. The output layer uses sigmoid, because the question is yes or no and the output must be a probability between 0 and 1. A sigmoid hidden layer on a deep network stalls, because its gradient shrinks towards zero at both ends. That stall has a name, vanishing gradients, and Lesson 211 meets it again in the RNN.

The learning rate is a step size, not a speed

Too small and the loss crawls. Too big and the loss jumps over the minimum and explodes. Adam adapts the step per weight, which is why 0.01 works here without tuning. A fixed step needs the tuning Adam skips. Either way, the first epoch's loss is the check: it must move, and it must move down. The simulator shows the same rule: at 0.001 the recall is 0.40 after ten epochs, because the steps were too small to get anywhere.

What changes at 10x and 100x

At 10x the rows, training stops being a script you run by hand and becomes a job: the data lands in a feature store, the training runs on a schedule, the model lands in a registry with a version. Lesson 93, The Model Lifecycle, teaches the registry. At 100x, the cost moves to the data pipeline and the serving stack, not the model. The loop is still forward, loss, backward, step. The number of weights is what changes.

Further reading

The forest, the threshold, and the same 2,000 deploys: Lesson 181 Classic Machine Learning. The derivative, the chain rule, and the gradient: Lesson 212 The Maths the Data Roles Assume. Precision, recall, and the slope as a derivative: Lesson 179 Maths and Statistics for Data Work. The Python in the lab: Lesson 75 Python for AI. The CNN, the RNN, and the LSTM, built on this lesson's loop: Lesson 211 Sequence and Vision Networks.

One time it broke

What broke

A team shipped a network that scored 99 percent accuracy on its test set. The first week it flagged nothing, and two deploys caused incidents anyway. The second week it flagged a routine deploy, and the team stopped looking at its scores. The model was right on paper and useless in production. The test set had been built from the same week as the training data, and the network had memorised the week's pattern instead of learning the shape. A new week arrived with a new pattern, and the memorised model had nothing to say about it.

Timeline

  • Monday 09:00The model ships with a 99 percent test score. The team celebrates.
  • TuesdayA deploy causes an incident. The model had scored it safe.
  • WednesdayA second incident, also scored safe. The team starts to doubt the model.
  • ThursdayThe model flags a routine deploy. The on-call engineer spends an hour checking a change that was fine.
  • FridayNobody opens the dashboard any more. The model is ignored.
  • Next MondayI read the split. The test rows came from the same week as the training rows, and the network memorised the week. The split was random, not by time.

What changed after

Two rules. First, the time split: training data ends one month before the test data starts, so the future cannot leak into the past. Second, the validation watch: the validation loss is plotted every epoch, and training stops when it stops falling, not when the training loss hits zero. The rebuilt model scored lower on the test set and higher on the next real week, and the team trusted it, because the numbers were honest.

If you remember nothing else

Design exercise

Design a classifier that flags risky deploys before they ship, and this time the team wants a network, not a forest. You have the same 2,000 past deploys Lesson 181 used. Five percent caused incidents. The on-call team will look at every flagged deploy, and they will ignore the model if it cries wolf more than half the time. The model must run in the deploy pipeline and answer in under a second. A second team has 200 labelled images of a new failure mode and a base model trained on a large image task. Design both: the network for the deploys, and the transfer plan for the images.

Clarifying questions to ask

  • Why a network, when the forest already reads this table. What pattern does the team believe the forest cannot see.
  • What does a missed incident cost, in pages and rollbacks, against what a false alarm costs, in engineer minutes?
  • What features exist before the deploy ships, and which ones only exist after?
  • How is the split done: by time, or at random. A random split on deploys lets the future leak into the past.
  • For the images, what task was the base model trained on, and how close is it to the new task. A base trained on unrelated images transfers nothing.
  • What happens when the model is down: does the deploy ship, or does it wait?

Worked approach

Start with the forest, again. Train it first, write down its four numbers, and make the network beat them on the same split before it ships. A network that cannot beat the forest on the same data is a network that has not earned its complexity.

Then the network. 8-32-1, ReLU hidden, sigmoid out, Adam 0.01, ten epochs, batch 32. Plot the train loss and the validation loss every epoch, and stop when the validation loss stops falling. Sweep the threshold from 0.3 to 0.9. The on-call rule is precision above 0.5, so the threshold must keep false alarms under half the flags. Ship the threshold with the model, written down beside it.

Then the transfer plan. Freeze the base, retrain only the head on the 200 images, and measure both tasks: the new task accuracy and the base task accuracy. Then, as the audit, unfreeze everything and fine-tune on the same 200 images, and measure both tasks again. The frozen run is the ship candidate. The fine-tuned run is the proof that the freeze matters.

Trade-offs to name. Complexity against explainability is the big one: the forest says which features matter, and the network does not. Training time against data size is the second: the network needs more rows to beat the forest, and 2,000 rows is barely enough. The serving stack is the third: Keras and TF Serving give a standard path with versioning, and PyTorch and an export give the research code a straight line to production. Pick the stack first, and the framework follows.

Saved in this browser, so the Lessons index can show your progress across the course.

Next up: Lesson 211
Sequence and Vision Networks: CNN, RNN, LSTM, and Jetson

The loop from this lesson, carried into images and sequences. The CNN learns what each layer sees, the LSTM learns what comes next, and the Jetson runs the small CNN on the edge.

Next: Lesson 211 →

Or browse the full course from the Lessons index.