~/teaching · machine learning · session 8, friday 12 february 2027

From the neural network to the model in service: backpropagation, drift, conformal prediction and an API

A network computed by hand, then the questions a model faces once it leaves the notebook.

Sitraka Forler, Centrale Méditerranée

Revision 16 October 2026 · about 16 minutes

Abstract

This deck closes the modelling part of the Machine Learning course at Centrale Méditerranée (option DDEFI, 2026-2027). The morning opens the last family of models of the course, the neural network: a forward pass and a backward pass computed number by number on networks small enough to fit on a board, the practical habits of training (mini-batches, Adam, initialisation, early stopping), then the honest question of whether a network is worth anything on a credit table where boosting is already strong.

The afternoon leaves the notebook. A model that goes live ages, so we measure drift with the population stability index and the Kolmogorov-Smirnov statistic and write a monitoring plan; we let the model say "I don't know" with split conformal prediction; we convert a decision threshold into euros per year for a reader who will never open the report; we read the EU AI Act as an engineer; and we put the model behind a small FastAPI service with tests and a measured latency.

The page is for engineering and finance students, and for practitioners who want the mechanics without the hype. After it you can redo a backpropagation step on paper, compute a PSI and a conformal margin, size an A/B test for a credit policy, and list what a regulator will ask of a credit score.

The slides

Neural networks, drift, conformal prediction, going live

Session 8, Friday 12 February 2027. Machine Learning, Centrale Méditerranée, option DDEFI, 2026-2027.

Open the slides (password from your teacher)

The deck itself is encrypted for the students of the course; this page summarises it for everyone.

The course page · Attention and transformers, step by step

Plan of the deck

The neural network. From Rosenblatt's perceptron to a 2-2-1 network, activations and their derivatives, forward pass, backpropagation and the training habits that make it work.
Networks on tabular data. Why trees still win on medium-sized tables, when a network deserves a place, embeddings, scaling, and how the network joins the existing comparison.
Drift and monitoring. Three kinds of drift, the Bike Sharing example, PSI and Kolmogorov-Smirnov, late labels, alerts with an owner and retraining policies.
Conformal prediction. Split conformal in four steps, a nine-score example, where the coverage guarantee comes from, its limits, and prediction sets for credit.
The client memo in euros. Expected cost of a threshold, gain per application and per year, sensitivity to the cost assumptions, and the size of the A/B test that checks the promise.
AI Act workshop. Why a credit score is a high-risk system, the other texts that already apply, and a gap analysis of Articles 9 to 15 against the project repository.
The FastAPI service. Three routes, an input contract, a property-based test, a test that the served model equals the evaluated one, and latency percentiles.

A network is a stack of logistic regressions

Each neuron weighs its inputs, adds them up and bends the result; the last neuron turns everything into a probability of default.

The deck starts from Rosenblatt's perceptron (1958): a weighted sum of the inputs, a bias, a hard threshold, and a correction rule that moves the weights only when the prediction is wrong. Its boundary is always a straight line (a hyperplane), which is why it cannot learn the "exclusive or", the point Minsky and Papert made in 1969. Adding a hidden layer fixes that: each hidden neuron builds a new variable, and the output neuron reads those variables instead of the raw columns.

h = \mathrm{ReLU}(W_1 x + b_1), \qquad p = \sigma(w_2^{\T} h + b_2)

(1)

The output layer is nothing new: it is the logistic regression of maximum likelihood, with the same log-loss. Without a hidden layer, a sigmoid output trained on the log-loss is logistic regression. The deck goes through the usual activation functions (sigmoid, hyperbolic tangent, ReLU and its leaky variants, softmax for several classes) with their derivatives, because the derivative decides whether the gradient gets through. ReLU became the default for hidden layers (Nair and Hinton, 2010) because its slope is exactly 1 wherever the neuron is active.

L = -\big[y \ln p + (1-y)\ln(1-p)\big], \qquad \frac{\partial L}{\partial z_2} = p - y

(2)

Worked example: a 1-1-1 network

One input, one hidden ReLU neuron, one sigmoid output, four parameters: $w_1 = 0.5$ , $b_1 = 0$ , $w_2 = 1$ , $b_2 = -1$ . The client has $x = 2$ and defaulted, $y = 1$ . The forward pass keeps every intermediate value, because the backward pass will need them.

**Table 1:** Forward pass of the 1-1-1 network for one client who defaulted.
Quantity	Computation	Value
$z_1$	$0.5 \times 2 + 0$	1
$h$	$\max(0, 1)$	1
$z_2$	$1 \times 1 - 1$	0
$p$	$\sigma(0)$	0.5
$L$	$-\ln 0.5$	0.693

The network is exactly undecided about a client who did default: its loss is that of a coin toss. The deck repeats the same exercise on a 2-2-1 network with nine parameters, where one hidden neuron fires and the other is switched off by the ReLU, and checks every number against PyTorch.

Back to contents.

Backpropagation is the chain rule, done in the right order

We carry the error from the output back to the inputs, multiplying by the slope of each step, and each weight learns which way to move.

Backpropagation (popularised for networks by Rumelhart, Hinton and Williams in 1986) starts from the error at the output, $p - y$ , and walks back through the network. At each box it multiplies by the local derivative: the output weight for the way down into the hidden layer, then the slope of the activation, which for ReLU is 1 if the neuron was active and 0 if not. The gradient of a weight is the error that reaches its box times what entered the box.

\delta_j = (p - y)\, w_{2,j}\, \mathbf{1}\{z_{1,j} > 0\}, \qquad \frac{\partial L}{\partial W_{1,jk}} = \delta_j\, x_k, \qquad \frac{\partial L}{\partial w_{2,j}} = (p - y)\, h_j

(3)

The 1-1-1 example, backwards, then one step with $\eta = 0.1$

Output. $p - y = 0.5 - 1 = -0.5$ , so $\partial L/\partial b_2 = -0.5$ and $\partial L/\partial w_2 = -0.5 \times h = -0.5$ .
Hidden layer. Multiply by $w_2 = 1$ and by the ReLU slope, 1 since $z_1 = 1 > 0$ , which gives $\partial L/\partial z_1 = -0.5$ , hence $\partial L/\partial b_1 = -0.5$ and $\partial L/\partial w_1 = -0.5 \times 2 = -1$ .
Step. Subtract $0.1$ times each gradient: $w_1 = 0.6$ , $b_1 = 0.05$ , $w_2 = 1.05$ , $b_2 = -0.95$ .
Check. New forward pass: $z_1 = 1.25$ , $z_2 = 0.3625$ , $p$ rises from 0.5 to 0.590 and the loss falls from 0.693 to 0.528.

Why not simply perturb each weight and measure the change? For the 23-64-1 credit network of the project (1,601 parameters), finite differences need 1,602 forward passes per gradient; backpropagation needs one forward and one backward pass, whatever the number of parameters. That ratio is what makes training networks possible. Finite differences keep one job: checking a hand-written gradient on a few weights.

Remark (Training habits). Mini-batches (200 rows by default in scikit-learn) and epochs; Adam as the default solver; random initialisation at the Glorot or He scale, never all zeros, or every hidden neuron stays a twin of the others; early stopping on a validation slice taken from the training data. The loss of a network is not convex, so results depend on the seed: report the mean and spread over several seeds, never the best one.

Back to contents.

On a credit table, the network is a challenger, not a favourite

On ordinary tables of a few thousand rows, tree ensembles usually beat neural networks, so the network must earn its place through the same comparison as every other model.

The deck relies on Grinsztajn, Oyallon and Varoquaux (NeurIPS 2022): on 45 medium-sized tabular datasets, random forests and gradient boosting stayed ahead of networks even after generous tuning. Their three reasons are worth remembering. Networks prefer smooth functions while tabular targets have steps; they are disturbed by useless columns that a tree simply ignores; and they mix all columns together, while in a table each column has its own meaning that a tree respects by cutting one axis at a time. The credit dataset of the course, 30,000 rows and 23 columns, sits exactly in that regime.

A network has a real chance with categorical variables of hundreds or thousands of levels, through learnt embeddings (Guo and Berkhahn, 2016): one small vector per level, tuned by backpropagation like any other weight.
It also helps when a table comes with text, images or sequences, when there are far more rows, or as a member of an ensemble with boosting, because its errors are weakly correlated with those of trees.
Scaling is mandatory: a tree does not care about units, a network does. Standardise, tame heavy-tailed amounts with quantiles or a signed log, and fit every transformer inside the Pipeline on the training part of each fold.

The method does not change because the model is new. The multilayer perceptron enters the existing comparison as one more row, on the same folds and the same seeds as boosting and random forests, with its hyperparameters tuned in the inner loop and the champion rule fixed in advance. If the network does not win, that is a result, and the one the literature predicts.

Back to contents.

Drift: the world moves, the model stays frozen

A deployed model learnt a snapshot of the past; monitoring compares today's data with that snapshot and says who acts when they differ too much.

Writing the joint law as $P(X, Y) = P(Y \mid X)\,P(X)$ separates three cases. Covariate drift: the incoming clients change (younger after a campaign) but the risk for a given profile does not. Target drift: the default rate itself moves, in a recession for example, which shifts calibration and the cost-based threshold. Concept drift: the relation between profile and risk changes, and no statistic on the inputs alone can see it; you need the labels. The deck shows the mechanics on the UCI Bike Sharing data (Washington, 2011 and 2012), where seasons move the temperature and the growth of the service moves the target, then transposes them to credit.

\mathrm{PSI} = \sum_{i=1}^{K} (a_i - e_i)\,\ln\frac{a_i}{e_i} = \mathrm{KL}(a \,\|\, e) + \mathrm{KL}(e \,\|\, a)

(4)

The population stability index cuts a variable into bins on the reference period, compares the share $e_i$ of each bin in the reference with its share $a_i$ in the current month, and sums. Each term is positive or zero, so the index is zero only for identical histograms. The usual reading in credit scoring (Siddiqi, 2006) is below 0.1 stable, 0.1 to 0.25 watch, above 0.25 act; these are habits of the trade, not statistical results, and a monitoring plan should justify its own.

**Table 2:** PSI with four bins and a uniform reference, $e_i = 0.25$ in every bin: the mass has slid towards the upper bins.
Bin	Month $a_i$	$a_i - e_i$	$\ln(a_i/e_i)$	Term
1	0.10	−0.15	−0.916	0.137
2	0.20	−0.05	−0.223	0.011
3	0.30	+0.05	0.182	0.009
4	0.40	+0.15	0.470	0.071
PSI				0.228

A PSI of 0.228 falls in the watch zone, just below the action line. An empty bin makes a term infinite, so shares are floored at a small value and the plan says so. The two-sample Kolmogorov-Smirnov statistic, the largest gap between the two distribution functions, complements it without bins; on large samples its p-value is almost always tiny, so the deck reads the gap itself.

Indicators on the inputs and on the score can be read at once; performance (AUC, calibration, cohort default rates) has to wait until a cohort matures, which for a loan means months. Two dashboards, two rhythms.
Each indicator gets a threshold, an action and an owner, or it is not monitoring.
Retraining runs on a schedule, on alert, or as a challenger running alongside the champion; in every case a retrained model is a new model, revalidated, with the old one kept for rollback.

Back to contents.

Conformal prediction: an interval with a promise

Instead of one number, the model gives a range that contains the truth at least nine times out of ten, whatever the model, as long as tomorrow's data look like today's.

Split conformal in four steps (Papadopoulos et al., 2002; Lei et al., 2018)

Split. Keep a calibration slice of $n$ rows that the model never sees.
Train. Fit the model on the rest, as usual; any model will do.
Score. On each calibration row compute a nonconformity score, in regression the absolute residual: the bigger, the more the row surprises the model.
Quantile. Take the $\lceil (n+1)(1-\alpha) \rceil$ -th smallest score as the margin $\hat q$ and return the prediction plus or minus $\hat q$ .

C(x) = \big[\hat y(x) - \hat q,\ \hat y(x) + \hat q\big], \qquad 1 - \alpha \le \mathbb{P}\big(Y \in C(X)\big) < 1 - \alpha + \frac{1}{n+1}

(5)

Worked example. Nine calibration residuals, already sorted: 0.1, 0.3, 0.5, 0.8, 1.2, 1.4, 1.9, 2.5 and 3.0, target coverage 80%, so $\alpha = 0.2$ . The rank is $\lceil 10 \times 0.8 \rceil = 8$ , and the eighth smallest residual is 2.5, so the interval is the prediction plus or minus 2.5. The guarantee comes from exchangeability alone: the new score is equally likely to take any of the ten ranks among the ten scores, and it is covered when it lands in the first eight, hence a coverage of exactly 8/10. The deck also shows the trap: with nine scores and $\alpha = 0.05$ the rank would be 10, there is no tenth score and the interval is infinite.

The promise is marginal: on average over future clients, not for each subgroup. Coverage can be high for clients without arrears and poor for clients in arrears while averaging the target, which links back to the fairness audit; calibrating group by group (Mondrian conformal) helps if each group has enough rows. A weak model still gets honest coverage, but wide intervals. And drift breaks exchangeability, so the guarantee goes with it.

For credit the same idea gives prediction sets. The score is one minus the probability of the true class; a new client keeps every class whose score is below $\hat q$ . A set containing only "no default" means grant, only "default" means refuse, and both classes means "I don't know": the application goes to a human, and the abstention rate becomes a monitored indicator.

Back to contents.

Putting the model in euros

A threshold is worth what it saves compared with doing nothing, brought back to one figure per year that a decision-maker can sign.

C(t) = c_{FP}\,FP(t) + c_{FN}\,FN(t), \qquad \Delta(t) = C_{\text{no model}} - C(t)

(6)

The deck reuses the illustrative costs of the threshold session: €600 of lost margin for a good client refused, €4,500 of expected loss for a default accepted on a €10,000 loan. These are course assumptions and the memo must say so.

**Table 3:** Illustrative figures: 10,000 applications including 200 defaults.
Policy	False positives	False negatives	Cost
Accept everyone	0	200	€900,000
Cost-based threshold, about 0.12	800	50	€705,000
Gain			€195,000

That is €19.50 per application; at an assumed 50,000 applications a year, €975,000. The memo then shows what moves the figure: if a default costs 20% less, the gain at the same threshold falls to €60,000 per 10,000 applications, a less favourable assumption that divides the result by more than three. The baseline matters as much as the model: if the bank already has a rule, the gain is measured against that rule, not against accepting everyone.

A figure in a memo is a promise, and an A/B test checks it. To detect a fall in the default rate of accepted loans from 10% to 8% (illustrative rates) with a 5% type I error and 80% power, the two-proportion formula gives 3,211 accepted applications per arm. Halving the gap to detect multiplies the size by four, and because defaults appear months after the loan, the test lasts months. The memo itself is one page in five blocks: the decision asked for, the gain per year, the assumptions, how it will be checked, and what can go wrong, with no unexplained acronym.

Back to contents.

The AI Act, then the model behind an API

A credit score is a high-risk system under EU law, and serving it means a strict input contract, a trace of every decision and tests that the served model is the one you evaluated.

Regulation (EU) 2024/1689 lists among high-risk systems, in Annex III point 5(b), those that assess the creditworthiness of natural persons, fraud detection excepted. It entered into force on 1 August 2024; the deck records that the date of application of the Annex III obligations was moved by a 2026 amending regulation, and insists on going back to EUR-Lex for the current wording rather than copying a date from a slide. Other texts already apply: GDPR Article 22 on solely automated decisions (the CJEU SCHUFA judgment, C-634/21, December 2023), the deployer duties of Articles 26 and 27 of the AI Act, and the Consumer Credit Directive (EU) 2023/2225.

The workshop is a gap analysis: for each of Articles 9 to 15 (risk management, data governance, technical documentation, record-keeping, transparency, human oversight, accuracy and robustness), which file of the project answers it, and whether it is covered, partial or missing. Writing "missing" honestly is the expected answer for a course repository. See also the governance session of the Data Analytics course.

GET /health tells operations the service is alive; POST /predict returns the probability, the conformal set and the decision; GET /model-info tells the auditor which version, git tag, data hash and threshold produced the decision.
A pydantic schema is the input contract: one type and one domain per variable, taken from the datasheet. An age of 250 is rejected with a 422 before the model sees it.
A property-based test with Hypothesis checks a law on hundreds of generated clients, here that one more month of arrears never lowers the predicted risk; boosting is not monotonic by construction unless you ask for it.
Two tests close the most common deployment bug: the reloaded model predicts exactly like the trained one, and the API returns the same probability as the model called directly.
Latency is measured, not declared: p50 and p95 over a thousand calls, written in the model card with the machine and library versions.

Back to contents.

Key takeaways

A neural network is a logistic regression sitting on variables that its hidden layers learn; the output gradient is simply $p - y$ .
Backpropagation is the chain rule applied from the output backwards, reusing each product; it costs about two passes, whatever the number of weights.
On medium-sized tables, boosting usually wins; a network enters the same comparison, with the same folds, seeds and champion rule.
Drift is measured on inputs at once and on performance only when labels mature; every indicator needs a threshold, an action and an owner.
Split conformal prediction gives a finite-sample coverage guarantee for any model, marginal over clients and void once the data drift.
A model's value is a cost difference against a stated baseline, brought to one figure per year with its sensitivity, and checked by an A/B test.
A served model is a contract: validated inputs, a traceable version, tests that it equals the evaluated model, and measured latency.

Glossary

Forward pass: Computing a network layer by layer from the inputs to the loss, keeping every intermediate value for the backward pass.
Backpropagation: Computing all the gradients of the loss by applying the chain rule from the output back to the inputs.
Activation function: The non-linear function applied to a neuron's weighted sum: sigmoid, tanh, ReLU, softmax.
Multilayer perceptron (MLP): A network of fully connected layers; in scikit-learn, MLPClassifier.
Early stopping: Stopping training when the loss on a validation slice stops improving, and keeping the best weights.
Embedding: A learnt vector per category level, tuned by backpropagation with the other weights.
Data drift: A change after deployment in the inputs, the target rate or the relation between them.
Population stability index (PSI): A binned divergence between a reference and a current distribution, equal to the sum of two Kullback-Leibler divergences.
Conformal prediction: A method that turns any predictor into intervals or sets with a guaranteed coverage under exchangeability.
Coverage: The probability that the interval or set contains the true value.
Nonconformity score: How much an example surprises the model, for instance the absolute residual or one minus the probability of the true class.
Exchangeability: The assumption that the order of the observations does not matter; drift breaks it.
Gap analysis: A table of the regulation's requirements against the documents that answer them, with a status for each.
Property-based test: A test that checks a law on many generated inputs rather than on one example.

The other notes of the course

One public summary per deck, in the order of the course.

IntroductionAI winters, jobs and method
ToolsKaggle, GitHub, Python: the tools
Session 1Project framing and least squares
Session 2Maximum likelihood and logistic regression
Session 3Gradient descent, ridge and lasso
Session 4Decision threshold and reproducible ML
HomeworkA defensible baseline model
Session 5Kernels, SVM and decision trees
Session 6Forests, boosting, honest comparison
Session 7SHAP, fairness, PCA and clustering
HomeworkA documented champion model
Session 8Neural networks, drift, conformal prediction
HomeworkA delivered machine learning project
Session 9Defending a machine learning project
ReferenceCredit default scoring project
ReferenceAsset returns: XGBoost and sentiment
ReferencePython code for every ML model
ReferenceDeep learning essentials
ReferenceOpen data ML projects by sector
ReferenceData and AI engineer handbook
ReferenceData engineering, SRE and cloud
ReferencePanorama of machine learning models

All the notes, as a reading list

Attention and transformers, step by step

About these notes

Written by the lecturer from the slides of the deck, as a public summary: the key ideas, the formulas and one worked example. The exercises, their solutions, the tests and the grading stay in the deck, for the students of the course.

Revision: 16 October 2026.