The code of every model: import, download by API, fit, predict

Six building blocks are enough to read a listing

Almost every line of a machine learning script is one of six things: a name for a value, a collection, a function, a loop, an import or a table.

The first part is for readers who have hardly written code. It does not teach programming in general; it teaches what you need to follow the rest of the deck, each block with a tiny listing on temperatures and its real output.

Step by step

1. A variable. A label attached to a value. The sign = assigns, it does not compare (comparison is ==). Three types cover most cases: text (str), decimals (float, always with a point) and whole numbers (int); type() tells you which one you hold.
2. A list and a dictionary. A list keeps values in order and is read by position, from 0; a dictionary keeps key and value pairs and is read by name. A dictionary is exactly the shape of the JSON that APIs send back.
3. A function. A named recipe declared with def, whose indented body returns a result. Arguments can have defaults, and changing one by name is what you do all day with scikit-learn, as in LogisticRegression(max_iter=1000).
4. A loop and a test. for repeats a block for each element, if chooses between two blocks, while repeats until a condition fails, which is how you walk through the pages of an API.
5. An import. A package is someone else's toolbox: installed once with pip install, opened in every script with import, usually under a short alias such as np or pd.
6. A pandas DataFrame. A table with named columns and an index. Three reflexes on any new table: .head(), .shape and .describe().

Three short habits complete the part. An error message (traceback) is read from the bottom: the last line says what went wrong, the line quoting your own file says where, and everything inside the library is noise. The usual suspects are KeyError (a misspelt column), NameError (a notebook cell skipped) and ModuleNotFoundError (a package missing from this environment). Documentation is one help() or one question mark away. And the same code runs on Kaggle, on Colab or in VS Code; the tools session covers the accounts.

The part ends with objects and classes, because they explain the rest of the course. A class is a blueprint, an object carries its own data (attributes) and actions (methods), and a child class inherits everything from its parent and changes only what differs. Every scikit-learn model inherits from the same base classes, and XGBoost's classifier was written to inherit from them too. That shared contract is why one loop can train a logistic regression, a tree and a boosting model with identical calls.

Back to contents.

One skeleton and three verbs: create, fit, predict

Every model in the course is used the same way: you choose it, you let it learn from examples whose answer you know, then you ask it about rows it has never seen.

Each model listing in the deck follows the same eight steps, always in this order: import the tools, download the data, build the input table X and the target y, split into a training part and a test part, fit on the training part, predict on the test part, measure with a metric, print. When a listing looks opaque, asking which of the eight steps a line belongs to usually unlocks it.

Inside that skeleton, scikit-learn reduces to three verbs. You create a model object with its settings, you call fit(X, y) on the training rows, and you call predict (or predict_proba for a probability) on new rows. What the model has learned is stored in attributes whose names end with an underscore, such as coef_ and intercept_. To try a forest, an SVM or a neural network instead, only the creation line changes. The listing below, written for this page on simulated data, shows the pattern.

# The same three verbs for any scikit-learn model
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

X, y = make_classification(n_samples=400, random_state=1)    # simulated table
X_tr, X_te, y_tr, y_te = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=0)

model = LogisticRegression(max_iter=1000)  # 1. create (the only line to swap)
model.fit(X_tr, y_tr)                      # 2. fit on the training rows
proba = model.predict_proba(X_te)[:, 1]    # 3. predict on unseen rows
print(model.score(X_te, y_te), proba[:3].round(2))

The skeleton on real data

The first full listing predicts the daily maximum temperature in Marseille in 2025 from the minimum and the humidity, with 365 days downloaded from Open-Meteo and a quarter of them drawn at random for the test. The fitted plane, as printed on 12 October 2026, is:

Reading the result

1. Coefficients. One more degree of minimum adds about 0.96 degree of maximum at equal humidity; one more point of humidity removes about 0.12. For a day at 10 °C minimum and 80 % humidity, the plane gives 15.53 + 9.57 − 9.20 = 15.9 °C.
2. Error. On the 92 test days the root mean squared error (RMSE) is 1.91 °C, against 5.77 °C for a naive model that always predicts the training mean. A metric means something only next to such a point of comparison.
3. The trap. Neighbouring days look alike, so a random split gives each test day near twins in the training set and the error is slightly optimistic. For forecasting, the split must follow time.

The same model through statsmodels gives the same coefficients plus standard errors and p-values: scikit-learn is the tool for prediction, statsmodels for inference. The derivation of least squares is in the first session.

Back to contents.

From binary to video: every kind of data becomes an array of numbers

A machine only stores numbers in base 2, so a text, a photo, a sound or a film always ends up as an array of numbers; only its shape and its size change.

A bit is 0 or 1, a byte is eight bits and can hold 256 values, from 0 to 255. Each position in base 2 is worth twice the one to its right, so the byte 01000001 is 64 + 1 = 65, the code of the letter A.

Two traps follow from this, and the deck shows both in code. A numpy array of one-byte integers (uint8, the type of images) silently wraps around: 250 + 10 becomes 4, with no warning. And a floating-point number cannot write 0.1 exactly in base 2, so 0.1 + 0.2 == 0.3 is false; floats must be compared with a tolerance (np.isclose), never with ==. Memory follows the type: a table of 30,000 rows and 24 columns in 64-bit floats takes 5.76 MB, half that in 32-bit floats, which keep about 7 significant digits instead of 16 and stop telling consecutive integers apart beyond $2^{24}$, so identifiers should stay integers or text.

Text goes through two layers. Unicode gives every character a number and UTF-8 writes that number as one to four bytes, which is why a CSV opened with the wrong encoding shows garbled accents. A language model then cuts the text into tokens, pieces of words from a fixed vocabulary, and reads their integer codes; a rare word is split into several pieces. The attention article shows what a transformer does with those tokens next.

An image is a three-dimensional array, and what a convolutional network does with it is shown in miniature: a 3 × 3 filter slides over the pixels, multiplies and adds, and responds strongly on an edge. A sound is sampled 44,100 times a second, enough to keep frequencies up to about 20,000 Hz; two seconds of mono sound make 88,200 integers of 2 bytes, plus a 44-byte WAV header:

A Fourier transform of that vector puts the peak at 440 Hz, the note that was generated. Raw video is so heavy that it is always compressed, and a model reads decoded frames, a few per second, resized. For tables, the usual case in the course, the last step is one-hot encoding: a text column with three possible values becomes three columns of 0 and 1, so that each client is a plain vector of numbers.

Back to contents.

Downloading data by API, then freezing a dated copy

An API is a web address that a program can question; it answers with structured text that pandas turns into a table, and that table is saved once, with its date, and never edited.

A web API takes its question in the address itself: after the question mark come name=value parameters separated by &. The server answers with a status code (200 for success, 404 for not found, 429 for too many requests) and a body, almost always JSON or CSV. JSON is built from the two containers of the first part, dictionaries and lists, so it is read like a Python dictionary.

Three gestures make a download robust: pass the parameters as a dictionary to requests.get, set a timeout so the script does not wait forever, and call raise_for_status() so that a refusal stops the script at once with a clear message instead of failing three lines later. The listing below is written for this page; it follows the deck's pattern on another city and variable.

import requests
import pandas as pd

url = "https://archive-api.open-meteo.com/v1/archive"
params = {"latitude": 49.61, "longitude": 6.13,          # Luxembourg City
          "start_date": "2024-01-01", "end_date": "2024-12-31",
          "daily": "precipitation_sum", "timezone": "Europe/Luxembourg"}
r = requests.get(url, params=params, timeout=30)
r.raise_for_status()                       # stop now if the server refuses
rain = pd.DataFrame(r.json()["daily"])     # JSON dictionary to table
rain.to_parquet("data/raw/rain_lux_2024.parquet")  # raw copy, never edited
print(rain.shape)

Each source has its own quirks, and the deck states its licence and terms in the slide notes. The World Bank returns one page at a time and mixes countries with aggregates such as the euro area; FRED leaves empty days when the US market is closed, which should be dropped rather than filled with zero; Yahoo prices are for personal use, so a public repository shares the script, never the price files. Since online sources get revised, the raw result is written to data/raw/ in Parquet, a typed column format, with the download date in the file name, and that folder stays out of Git.

Back to contents.

A tour of the model families, one listing each

The deck runs every family of the course with the same skeleton, on real public data, and reads each output in plain words.

Each family opens with a one-sentence idea, then the listing, then what the output means, and points to the session that derives the model. Two penalised versions of least squares come first: ridge adds the sum of squared coefficients, lasso the sum of their absolute values, which pushes some of them exactly to zero.

Logistic regression passes a weighted sum through the sigmoid to get a probability of default; its listing, with numeric columns scaled and categorical ones one-hot encoded inside a Pipeline, becomes the template of every classification part. A probability decides nothing until a threshold is chosen, and that choice is a matter of costs, the subject of the fourth session.

Several runs are there to show a failure rather than a success. The nearest-neighbour model with one neighbour learns the training set by heart. The same SVM, fed an unscaled credit amount, sees only that one variable. An unlimited tree reaches 146 leaves and does worse on the test than a single question. The small perceptron jumps from last to first of the deck when its early-stopping patience changes, a reminder that on a 750-row table boosted trees are the safer bet. On the unsupervised side, k-means always returns the number of groups you ask for, and the Gaussian mixture's BIC says these country indicators form a continuum. On the euro-dollar, an ARIMA(1, 1, 1) is almost a random walk and a walk-forward AR(1) does no better than forecasting a zero return, while a GARCH(1, 1) shows how persistent volatility is.

The derivations live in the sessions: regularisation, kernels and trees, forests and boosting, PCA and clustering, neural networks, and the asset-price project for series and FinBERT. The model overview places them all on one map.

Back to contents.

Comparing honestly, then turning the notebook into a script

Several scores on one test set are mostly noise; models are compared by cross-validation on the training data, and the test is opened once, for the chosen model only.

Once every classifier has run on the same split, the temptation is to crown the highest test AUC. The deck refuses it: the best SVM was picked among six settings by looking at that very test, the network depends on its patience, and with 75 defaults and 175 good clients the standard error of an AUC near 0.8 is about 0.03 (Hanley and McNeil, 1982). Gaps of one hundredth say nothing.

The honest method is stratified five-fold cross-validation on the training set: five fits, each measured on the fold left aside, with the same folds for every candidate, so that the comparison is paired. The decision rule is written before computing.

The gaps are smaller than the standard deviations, so the rule fixed in advance applies: keep the simplest model. The logistic regression, readable and defensible for credit scoring, is refitted on the whole training set and the test is opened once, giving 0.794. That single number is the one that goes into a report.

The last part moves the same skeleton from a notebook into a train.py script that reads every setting from a config.yaml file, saves the fitted model with joblib and writes its metric to a JSON file, so that one command reruns everything. Changing a depth or a seed then happens in the configuration, in a commit that says why, never by editing code. A model saved with one version of scikit-learn should be read back with the same version, one more reason to record versions. Each kind of listing then has its place in the project repository described in the project baseline.

Back to contents.

Key takeaways

1. Six Python building blocks (variable, list and dictionary, function, loop, import, DataFrame) are enough to read any listing of the course.
2. Every model follows the same eight steps, and scikit-learn reduces them to three verbs: create, fit, predict; only the creation line changes from one model to another.
3. Text, images, sound and video all become arrays of numbers; mind integer overflow in uint8, never compare floats with ==, and keep identifiers out of 32-bit floats.
4. Download with parameters, a timeout and a status check, then save the raw result once, dated, and share the script rather than the data.
5. Distance-based models (neighbours, SVMs, neural networks, penalised regressions) need scaling inside the pipeline; trees do not.
6. A metric is read next to a naive baseline, and models are compared by cross-validation, with the test opened only once.

Glossary

Package

A library written by others, installed once with pip and opened in each script with import.

DataFrame

The pandas table: named columns, one row per observation, an index on the left.

API

A web address a program can query with parameters; it answers in JSON or CSV with a status code.

Fit and predict

The scikit-learn contract: fit learns from labelled rows, predict answers on new rows.

Pipeline

A chain of preparation steps and a model fitted as one object, so preparation is learned on training data only.

Feature scaling

Bringing each numeric column to zero mean and unit standard deviation, required by distance-based models.

One-hot encoding

Replacing a text column with one 0/1 column per possible value.

Test set

Rows kept aside and used once, at the end, to measure the chosen model.

Baseline

The simple reference a model must beat, such as always predicting the mean.

Cross-validation

Fitting several times, each time measuring on a different held-out fold of the training data.

Overfitting

A model that learns the training rows by heart and does worse on new ones.

Token

A piece of a word from a fixed vocabulary, the unit a language model reads as an integer.

Floating-point number

The machine's approximate decimal, about 16 significant digits in 64 bits.

Walk-forward validation

Refitting on the past only, day after day, and forecasting the next day.

The other notes of the course

One public summary per deck, in the order of the course.

Previous: Asset returns: XGBoost and sentimentNext: Deep learning essentials

1. IntroductionAI winters, jobs and method
2. ToolsKaggle, GitHub, Python: the tools
3. Session 1Project framing and least squares
4. Session 2Maximum likelihood and logistic regression
5. Session 3Gradient descent, ridge and lasso
6. Session 4Decision threshold and reproducible ML
7. HomeworkA defensible baseline model
8. Session 5Kernels, SVM and decision trees
9. Session 6Forests, boosting, honest comparison
10. Session 7SHAP, fairness, PCA and clustering
11. HomeworkA documented champion model
12. Session 8Neural networks, drift, conformal prediction
13. HomeworkA delivered machine learning project
14. Session 9Defending a machine learning project
15. ReferenceCredit default scoring project
16. ReferenceAsset returns: XGBoost and sentiment
17. ReferencePython code for every ML model
18. ReferenceDeep learning essentials
19. ReferenceOpen data ML projects by sector
20. ReferenceData and AI engineer handbook
21. ReferenceData engineering, SRE and cloud
22. ReferencePanorama of machine learning models

All the notes, as a reading list

Attention and transformers, step by step

About these notes

Written by the lecturer from the slides of the deck, as a public summary: the key ideas, the formulas and one worked example. The exercises, their solutions, the tests and the grading stay in the deck, for the students of the course.

Revision: 16 October 2026.