~/teaching · machine learning · session 9, friday 12 march 2027

Defending a machine learning project: the pitch, the live demonstration, the questions, and what comes after

Five blocks to show, four traps to avoid, one way to answer what you do not know.

Sitraka Forler, Centrale Méditerranée

Revision 16 October 2026 · about 11 minutes

Abstract

This is the summary of the last deck of the Machine Learning course at Centrale Méditerranée (option DDEFI, 2026-2027): the day on which each team defends its final project in front of a jury. The deck is less about models than about judgement. A defence checks that a repository runs on someone else's machine, that the model serves a decision with numbers attached, and that every member of the team can answer a question alone.

The note keeps what is useful to anyone who has to defend a data science project, at school or at work: the five blocks of a short pitch, the live checks a technical jury runs, the questions that come back every year, the traps and their counters, and a method for the question you cannot answer. The last part looks beyond the course: how to tell the project in an interview, how to make a public repository readable without you, which jobs in finance it speaks to, and what to read next.

It assumes the earlier sessions: the cost-based decision threshold, the honest comparison of models, SHAP and fairness audits (session 7), and conformal prediction and drift (session 8).

The slides

Final presentation day: the defences

Session 9, Friday 12 March 2027. Machine Learning, Centrale Méditerranée, option DDEFI, 2026-2027.

Open the slides (password from your teacher)

The deck itself is encrypted for the students of the course; this page summarises it for everyone.

The course page · Attention and transformers, step by step

Plan of the deck

The day and the format. How a defence is built: a short pitch, then a live run of the handed-in repository, a predicted change, a derivation and open questions.
What a defence must show. Five blocks in order: the problem in one picture, the decision served, three models compared, the limitations, the next steps.
Questions and traps. The four mistakes seen every year, the predicted change, the likely questions by theme, and how to answer without knowing.
Deliberation and group debrief. While the jury deliberates, teams collect three ideas borrowed from other projects; the debrief draws the common lessons.
Afterwards. The project in an interview, the public repository as a CV line, internships and data jobs in finance, references to keep going.
What the course leaves. A model is judged by the decision it serves, the uncertainty it admits, and whether someone else could rebuild it.

What a defence checks, and how it runs

A defence does not test whether your model is the best; it tests whether it runs elsewhere, whether it serves a decision, and whether each of you understands it.

The jury looks for three things and no more. First, a repository that reproduces on a machine that is not yours: the jury clones the last commit pushed before the deadline, runs make reproduce and starts the API with make serve. Whatever was fixed afterwards does not exist for the day; say so if it matters, but the handed-in version is the one that runs. Second, a decision: the model is there to accept, refuse or flag something, with a threshold, an uncertainty, a value in euros and limitations the team states itself. Third, a team in which everyone can answer: a member who only knows their own part is visible within a minute.

The shape of a fourteen-minute defence in this course

Pitch. Five minutes, ten slides at most, every member presenting at least one block.
Demonstration. The jury runs the handed-in repository on its own machine; no USB stick, no laptop swap, no link sent that morning.
Predicted change. The jury alters one input of an example client and a member drawn at random predicts the effect before it is run.
Derivation. A member drawn at random works through one derivation from a list published months in advance, at the board.
Questions. Addressed to anyone, about the threshold, intervals, conformal sets, fairness, monitoring or the AI Act.

The practical consequence is simple: what you show is what you handed in. The repository must contain the slides as a PDF, a README that explains how to reproduce and serve, and an example client file to query the prediction endpoint. A demonstration that only works on the author's laptop is not a demonstration.

Back to contents.

The pitch in five blocks: problem, decision, comparison, limitations, next steps

Start from the decision and what each error costs, because the threshold, the metric and the value in euros all follow from it.

The problem in one picture. One figure, readable from the back of the room: who the client is, which decision is taken about them, and what each of the two errors costs. On the course's running example (credit default scoring), refusing a good client is assumed to cost €600 and accepting a client who defaults €4,500, two course assumptions set in session 4. Without this picture, a ROC curve means nothing to a jury.

The decision served. Three numbers, each with its origin. The threshold, derived from the costs and applied to calibrated probabilities. The abstention rate: the share of files for which the conformal set holds both classes, sent to a human. And the yearly value, always against something: against accepting everyone, or against the rule the bank already applies. A gain announced without its reference point says nothing.

t^{*} = \frac{c_{\mathrm{FP}}}{c_{\mathrm{FP}} + c_{\mathrm{FN}}}

(1)

Worked example: the threshold of the running example

Costs. $c_{\mathrm{FP}} = 600$ euros for a good client refused, $c_{\mathrm{FN}} = 4500$ euros for a default accepted.
Threshold. $t^{*} = 600 / 5100 \approx 0.1176$ , so a file is refused as soon as its calibrated probability of default exceeds about 12%.
Reading. The threshold is low because a missed default costs 7.5 times more than a lost client; it is a business choice, written in the configuration, not an output of the model.

Three models, compared honestly. One table, three rows: a naive reference (on the running example, the logistic regression), the champion, and the heaviest model tried, boosting or a network. Each with its AUC and Brier score and their 95% intervals, plus a paired test with the Nadeau and Bengio correction against the reference. The sentence under the table matters most: which selection rule was written before seeing the numbers, and which model it picked. If nothing beats the logistic regression in an established way, keeping it is a result, not a failure.

\hat\sigma^2_{\text{corr}} = \left(\frac{1}{J} + \frac{n_{\text{test}}}{n_{\text{train}}}\right)\hat\sigma^2

(2)

Here $J$ is the number of resampled splits and $\hat\sigma^2$ the variance of the score differences across them; the second term corrects for the overlap between training sets, which the naive variance ignores.

**Table 1:** The four limitations a pitch should state before anyone asks.
Limitation	What to say about it
Data	Origin, period, and what it does not cover (the running example applies data from a Taiwanese bank in 2005 to a European bank)
Conformal coverage	The guarantee is marginal; report what it is worth per subgroup
Fairness	What the per-group audit showed, and what it could not show
Drift	What would break the model, and which monitored indicator would catch it

Next steps. A good one names a precise action, what it should change and how it will be measured, and why it comes first; for instance, constraining the boosting to be monotone in the latest repayment delay because a property test fails today. "Try other models" or "get more data" are not wrong, they are empty: they do not show where the weak point is.

Back to contents.

The predicted change: show that you know your model

The jury changes one input of a real case and asks you to predict, before running it, whether the score moves, by how much, and whether the decision flips.

This exercise checks understanding, not luck. A case is taken from the validation set and one input is changed, typically the most influential one (on the running example, last month's repayment delay). Three things are expected: the direction of the move, its order of magnitude, and its consequence for the decision, given the distance to the threshold and the conformal set.

The tools exist already in the repository. SHAP values give the sign and size of each feature's contribution for this client, because they decompose the prediction additively around a base value:

f(x) = \phi_0 + \sum_{j=1}^{p} \phi_j(x)

(3)

A monotonicity test, if one was written, guarantees the direction; the threshold says whether the case crosses it; the conformal set says whether the service would abstain. A wrong prediction followed by a sound explanation of the gap is worth more than a correct guess without reasoning.

Back to contents.

Questions, traps, and how to answer without knowing

Most failures in a defence are predictable, and the question you cannot answer is coming anyway, so prepare the way you will answer it.

**Table 2:** Four traps seen every year, each with its counter.
Trap	Counter
Reading the slides aloud	Keywords on screen, the full sentence spoken
A demonstration that breaks	The day before, one member clones the handed-in tag on another machine and follows the README alone (usual culprits: a dependency missing from the lockfile, an absolute path)
"We did not have time"	"We chose X over Y because...": a choice, not an excuse
A number without its method	Say on what data, how, and with which interval, e.g. an AUC of 0.78 with its interval from nested cross-validation, not a bare 0.78

Answering a question you cannot answer

Rephrase. Check that you are answering the question actually asked, and gain a few seconds.
Separate. Say what you measured and what you did not, e.g. coverage checked by sex but not by age band.
Propose. Say how you would check it with your own code, concretely: which function, which split, which table.

Never make up a number: the jury can check it in the repository, and it is the one answer that truly hurts. Do not answer a different question, and do not let one member answer for everyone. "I do not know, here is how I would find out" is an engineer's answer, in a defence as in a job interview.

Threshold: what happens to it if a default becomes cheaper?
Comparison: why this champion rather than the boosting?
Conformal prediction: what does the interval guarantee, and for whom?
Fairness: which feature could act as a proxy for a protected attribute?
Monitoring: a PSI of 0.3 on the key variable, who decides what?
AI Act: which obligation does the repository cover least well?

None of these is a surprise: each comes from a session and from a file the team wrote. The best rehearsal is cheap: the day before, each member asks another member two questions. While other teams defend, listen with one question in mind: what did they do that we did not, and why?

Back to contents.

After the defence: interviews, a readable repository, jobs, reading

Once handed in, the project becomes the story you tell in interviews and a public repository that recruiters read without you.

Interviewers rarely ask for an AUC. They ask you to tell a project and listen for three things: whether you know what it was for, whether you took a real decision, and whether you know what it does not do. Four beats in about a minute cover them: the decision in a sentence, the most difficult choice with its reason, the result together with its interval, then one limitation and the next step it calls for. Prepare the follow-up question too: how was it validated? The answer is already there: an auditing team, a reproducible run, a selection rule fixed in advance.

The README opens with the problem and the decision, not with the installation.
A licence is declared and every dataset is credited with its own (the running example: UCI, Yeh and Lien, 2009, CC BY 4.0).
No personal data and no secret anywhere in the git history, which keeps everything.
The README says who did what; a recruiter will ask about your share.

**Table 3:** Five families of data jobs in finance, and what the project demonstrates for each.
Job family	What the project shows
Credit risk modelling	A cost-based threshold, calibration, an honest comparison
Model validation (second line of defence)	Reproducing, challenging, reading a model card
ML engineering and deployment (MLOps)	A tested API, continuous integration, drift monitoring
Data engineering	A frozen split, a schema contract, a datasheet
Model risk and AI compliance	An AI Act gap analysis, limitations written down

The deck gives no salaries and no number of openings, for lack of a source it could cite for 2027; the advice is to read real internship offers and count how often they mention validation, MLOps or the AI Act. For more on these roles, see the data and AI engineer handbook and the note on data engineering, SRE and cloud.

To keep going, from the most practical to the most mathematical: the free Kaggle Learn micro-courses; Géron, Hands-On Machine Learning (O'Reilly, 3rd edition, 2022); fast.ai's free Practical Deep Learning for Coders; James, Witten, Hastie and Tibshirani, An Introduction to Statistical Learning (2nd edition, 2021, free PDF); Hastie, Tibshirani and Friedman, The Elements of Statistical Learning (2nd edition, 2009, free PDF); Murphy, Probabilistic Machine Learning: An Introduction (MIT Press, 2022). For networks, Goodfellow, Bengio and Courville (2016); for conformal prediction, Angelopoulos and Bates (2021). The best next step remains a second project on data that interests you.

Back to contents.

Key takeaways

A defence checks three things: a repository that runs on someone else's machine, a model tied to a decision with numbers, and a team in which each member can answer alone.
Open with the problem and the cost of each error; the threshold, the metric and the value in euros follow from that picture.
Compare a naive, a best and a heavy model with intervals, and state the selection rule written before the results.
State the limitations (data, conformal coverage, fairness, drift) before you are asked; discovering them during questions is what costs.
Facing a question beyond your reach, restate it, distinguish what was measured from what was not, offer a concrete way to check, and do not invent a figure.
After the course, the project is a one-minute interview story and a public repository that must be readable without you.

Glossary

Decision threshold: The probability above which the model's output triggers the action; derived here from the two error costs.
Abstention (reject option): The service declines to decide when the conformal set contains both classes, and sends the file to a human.
Conformal prediction: A method that turns a model's scores into prediction sets with a coverage guarantee, marginal over the population.
Champion model: The model designated by a selection rule written before the test results are seen.
Nadeau and Bengio correction: A variance correction for paired comparisons over resampled splits, which accounts for overlapping training sets.
Predicted change: Changing one input of a real case and predicting the direction, size and consequence of the score's move before running it.
SHAP values: Additive contributions of each feature to one prediction, around a base value.
Drift: A change in the distribution of inputs or of the relation to the target after deployment.
Second line of defence: The independent function, such as model validation, that reproduces and challenges models built by the first line.
Reproducibility: The property that another person, on another machine, obtains the same numbers from the handed-in repository.

The other notes of the course

One public summary per deck, in the order of the course.

IntroductionAI winters, jobs and method
ToolsKaggle, GitHub, Python: the tools
Session 1Project framing and least squares
Session 2Maximum likelihood and logistic regression
Session 3Gradient descent, ridge and lasso
Session 4Decision threshold and reproducible ML
HomeworkA defensible baseline model
Session 5Kernels, SVM and decision trees
Session 6Forests, boosting, honest comparison
Session 7SHAP, fairness, PCA and clustering
HomeworkA documented champion model
Session 8Neural networks, drift, conformal prediction
HomeworkA delivered machine learning project
Session 9Defending a machine learning project
ReferenceCredit default scoring project
ReferenceAsset returns: XGBoost and sentiment
ReferencePython code for every ML model
ReferenceDeep learning essentials
ReferenceOpen data ML projects by sector
ReferenceData and AI engineer handbook
ReferenceData engineering, SRE and cloud
ReferencePanorama of machine learning models

All the notes, as a reading list

Attention and transformers, step by step

About these notes

Written by the lecturer from the slides of the deck, as a public summary: the key ideas, the formulas and one worked example. The exercises, their solutions, the tests and the grading stay in the deck, for the students of the course.

Revision: 16 October 2026.