Predicting an asset price: time series, XGBoost and news sentiment, measured honestly

Frame the problem: predict a return, never a price

Yesterday's price is always close to today's, so any model that copies it looks brilliant on prices while knowing nothing; the honest target is the change.

The deck starts with the decision, as the credit scoring project does. Three uses are possible: adjusting a position, hedging an exposure, or briefing a committee with a daily note. The project picks the third: a report every evening, no real order, and a simulated strategy only to put a value on the signal.

Then comes the trap that ruins most stock-market projects. On a simulated random walk of 2,520 days, where nothing can be predicted, the forecast "tomorrow equals today" reaches an $R^2$ of 0.993 on prices. The same idea applied to returns gives $R^2 = -1.02$, worse than predicting zero, because the difference of two independent returns has twice their variance. A predicted curve that hugs the real one with a one-day lag is therefore no evidence at all.

The target is the next-day log return of the adjusted close, and the direction is read from the sign of the forecast. Everything placed in the features must belong to the information set $\mathcal{F}_t$ available at the Paris close of day $t$.

Log returns are preferred because they add up over time, so a five-day target is just the sum of five one-day targets. Simple returns do not add up, as the deck shows on five invented prices.

Back to contents.

Walk-forward validation, baselines and leaks

On data ordered in time you learn on the past, test on the block that follows, step forward and repeat, exactly as if you had lived through the period day by day.

Shuffled cross-validation would let the model learn from the future to predict the past. The deck replaces it with walk-forward validation: train on everything up to a date (expanding window) or on the last few years only (rolling window), predict the next block untouched, move forward one block. The test blocks, laid end to end, make the out-of-sample record. In scikit-learn this is TimeSeriesSplit.

The arithmetic of the deck's split

1. History. Ten years of trading make about 2,520 sessions.
2. First training window. Three years, i.e. 756 sessions, before the first test.
3. Test blocks. One quarter each, 63 sessions.
4. Folds. $(2520 - 756)/63 = 28$ folds, covering $28 \times 63 = 1764$ test days, the same days for every model.

Every model is compared with baselines written before anything else. Predicting a zero return is the hardest benchmark in RMSE; repeating yesterday's return (persistence) roughly doubles the error variance. ARIMA (Box and Jenkins) and GARCH (Bollerslev, 1986) complete the time-series toolbox: ARIMA usually finds tiny coefficients on daily returns, which is efficiency rather than failure, while GARCH forecasts the size of tomorrow's move, not its sign.

With a five-day target, neighbouring rows share four days of their targets. López de Prado (2018) recommends purging the training rows whose target spills into the test block and adding an embargo after it when training data follow the test. With a one-day target and a pure walk-forward, the purge shrinks to a single row.

Three families of leakage are listed: look-ahead bias (a feature dated $t$ but known later, such as a US series that closes after Paris), survivorship bias (choosing stocks from today's index), and statistics recomputed with the future (standardising over the whole sample, revised macro data). The most useful test in the repository truncates all raw data at a date, rebuilds that day's row, and checks that it is identical to the row in the full table.

Back to contents.

XGBoost on lagged features

A tree model does not know about time, so each day becomes a row whose columns describe only what happened before the close, and whose label is the next day's return.

Lagged features give the past to a model that only sees rows and columns: the day's return and the four before it via shift(k), a 20-day cumulative return for momentum, a 20-day rolling volatility and the GARCH forecast, a 14-day RSI (Wilder, 1978), the day of the week, and a few macro series from FRED lagged by one day. The target is moved up one row (shift(-1)), and a single sign error there means the model learns a return it already knows. The last row has no target: it is the one predicted tonight, so a global dropna() must not remove it.

import numpy as np, pandas as pd, xgboost as xgb

r = np.log(px["Close"]).diff()                # log return, known at the close
X = pd.concat({f"lag{k}": r.shift(k) for k in range(5)}, axis=1)
X["vol20"] = r.rolling(20).std()               # trailing window only
X["mom20"] = r.rolling(20).sum()
y = r.shift(-1)                                 # target: tomorrow's return
ok = X.notna().all(axis=1) & y.notna()          # tonight's row is kept aside
model = xgb.XGBRegressor(max_depth=3, learning_rate=0.02,
                         subsample=0.8, n_estimators=500)
model.fit(X[ok].iloc[:-250], y[ok].iloc[:-250])  # one fold, for reading
pred = model.predict(X[ok].iloc[-250:])

XGBoost (Chen and Guestrin, 2016) adds small trees one by one and charges a price for leaves and leaf values. On returns, where the signal is faint, that regularisation matters: shallow trees, a small learning rate, subsampling, and early stopping. The validation set for early stopping is the last stretch of the training window, never the test block, and hyperparameters are frozen in a config file once chosen over all folds. The details of boosting are in the note on random forests and gradient boosting.

Two more points. Permutation importance and SHAP values say what the model used; volatility features often rank first because they help with magnitude, not direction. And a tree predicts leaf averages, so it can never go beyond the largest value it has seen: on a rising price it stays flat at its edge leaf. A stationary target, the return, removes the problem.

Back to contents.

News sentiment: lexicons, FinBERT and GDELT

Each headline gets a score between minus one and plus one, the scores of a trading day are averaged, and those averages become a few extra columns in the table.

The deck climbs the same ladder as for models. A finance word list (Loughran and McDonald, 2011) counts positive and negative words; it was built because general dictionaries treat words like liability or tax as negative when, in a financial text, they are just accounting items. VADER (Hutto and Gilbert, 2014) adds rules for negation and intensifiers but was designed for social media. FinBERT (Araci, 2019) is a BERT model further trained on financial text; thanks to attention it reads each word in context, so "costs fall" and "sales fall" are not scored alike.

FinBERT returns three probabilities (positive, negative, neutral); the score keeps the balance of the two extremes, and the largest probability serves as a confidence. A rerun by the teacher in October 2026 found that a terse headline and a full sentence with the same meaning can get opposite scores, so test the model on your real headlines first. It also says whether news is good, not whether it was a surprise, and the surprise is what moves prices.

The alignment rule: a headline belongs to the first session whose close comes after its publication time, all times converted to UTC. A story published at 18:45 Paris time on a Friday goes into Monday's row (merge_asof with a forward direction does it). Wrong time zones are the most common leak in sentiment projects.

The weighted mean is higher because the positive headline is the most recent. The day also gets a headline count, so the model can tell "neutral" from "no news". GDELT, an open database that codes the world's press into events, themes and tone every fifteen minutes, adds geopolitical columns: average tone of events in oil-producing countries, material conflict counts compared with their 60-day level, tone of energy-themed articles, and attention volume. Lists of countries and themes are fixed before looking at results.

Back to contents.

Combining the signals and measuring what the news adds

The simplest combination is one table with prices, macro, sentiment and geopolitics side by side, then a careful check of whether removing the news changes anything.

Sentiment features join the lags and macro columns of the same row, each with a documented time of availability, and a single XGBoost decides whether to use them. The project compares three rungs: a naive model (zero return, sign persistence for direction), a more complex one (ARIMA, or ridge on the same lags, which isolates the effect of non-linearity), and the heavy XGBoost with every family. The rule deciding whether the heavy model wins is written in the decision log before any computation.

The team writes down its expected outcome before running the ablation study. If C does not beat B, that is a legitimate finding, consistent with the semi-strong form of efficiency. Stacking (Wolpert, 1992) is offered as an alternative: specialised base models whose forecasts are blended by a ridge, provided the weights are learnt only on out-of-sample forecasts produced walk-forward; otherwise the blend trusts whichever model overfits most.

Back to contents.

Honest metrics: tests, costs and the number of attempts

A model that is right 52 days out of 100 may just be lucky, and a strategy that looks profitable may vanish after fees or after counting how many variants were tried.

Three metrics run from model to business: RMSE against the zero forecast, the share of correct directions against 50%, and the annualised Sharpe ratio of a simple long-or-flat rule after costs. Each gap must pass a test. The Diebold-Mariano test (1995) asks whether the average daily difference in squared errors between two forecasts is larger than noise.

Worked example from the deck: naive model (1) against XGBoost (2), eight invented days

1. Loss differences. Day 1 gives $1.2^2 - 1.0^2 = 0.44$, and the eight values are 0.44, −0.17, 0.81, 0.12, −0.19, 0.57, 0.20 and 0.25 (in units of $10^{-4}$).
2. Mean and spread. $\bar d = 0.254$ and $s_d = 0.347$.
3. Statistic. $DM = 0.254 / (0.347/\sqrt8) = 2.07$, above 1.96: tempting to declare XGBoost better.
4. Small-sample correction. Harvey, Leybourne and Newbold (1997) multiply by $\sqrt{7/8}$, giving 1.94, to compare with 2.36 for a Student law with 7 degrees of freedom: no conclusion. Only a long test period can decide.

For directions, the binomial test treats the model as a coin. Over the 1,764 test days, 52.0% correct means 917 hits and a one-sided p-value of 0.050, borderline. Turning the formula around, a 52% rate needs about 1,691 trading days (6.7 years) to pass at 5%, and 51% needs 6,764 days. If the asset rose on more than half the days, "always up" is the right benchmark, not 50%.

Costs decide many projects. In the deck's illustration, a gross gain of 0.04% a day with a 1% daily standard deviation gives a Sharpe of 0.63; paying 5 basis points on 40% of days halves the mean gain and the Sharpe falls to 0.32. Finally, the deck simulates strategies with no skill at all: over one year, the best of 100 has an average annualised Sharpe near 2.5. Hence the deflated Sharpe ratio of Bailey and López de Prado (2014), and the minimum rule of declaring every configuration tried.

Back to contents.

Pitfalls, ethics and going live

Markets change regime, headlines can describe the very move you predict, sources move under your feet, and a public prediction about a real stock has legal limits.

• Regimes. The 2020 crash, when the VIX closed above 80 in March, and the 2022 energy shock changed volatility and the oil link of an energy stock. Report results by period and compare expanding and rolling windows.
• Leakage through headlines. Many headlines narrate the session ("shares slide 4%"). Attached to the wrong day they predict perfectly. The deck treats directional accuracy above 60% as an alarm, a personal rule of caution, and suggests shifting the news by one more day: a collapsing score is suspicious.
• Moving data. Adjusted prices are recomputed after each dividend, delisted stocks vanish from free sources, macro series are revised (ALFRED keeps the vintages), RSS feeds forget. Raw files are dated, hashed and never overwritten.
• Law and disclosure. The EU Market Abuse Regulation (No 596/2014) forbids insider dealing and spreading misleading information. The project uses public sources only, places no real order, and states at the top of the README and of the interface that it is a teaching project, not investment advice.

Going live means an evening batch, about an hour after the close, that calls the same functions as the backtest on a single day and writes a dated report; a test compares the batch's prediction for a past day with the backtest's. Drift, treated in the note on drift and conformal prediction, is watched at three levels: feature distributions (PSI), source volumes (a feed at zero is broken, not calm), and the cumulative hit rate, whose pure-chance band is about ±12.7 points over 60 days. A small Streamlit page only reads the reports, so it cannot introduce a leak.

Back to contents.

Key takeaways

1. Predict a return or a direction, never a price level: on prices even a random walk gives an $R^2$ near 1.
2. Every feature has a time of availability, and a headline belongs to the first session that closes after it was published.
3. Validate walk-forward, with baselines (zero, persistence) computed on the same days and under the same cost assumption.
4. Sentiment enters XGBoost as a few daily columns; an ablation decided in advance shows whether it adds anything.
5. A gap is a result only after a test: Diebold-Mariano for errors, binomial for directions, and enough test days to detect a small edge.
6. Count every configuration tried and report the Sharpe net of costs; the best of many useless strategies still looks good.
7. A clean, well-tested conclusion that the naive model is not beaten is a valid outcome in finance.

Glossary

Log return

The logarithm of the ratio of two consecutive prices; log returns add up over time.

Walk-forward validation

Training on the past, testing on the block that follows, then moving forward; the time-series replacement for shuffled cross-validation.

Lagged features

Columns holding past values of a series (shifted by one or more days) so that a tabular model can see the history.

Data leakage

Information from after the decision time slipping into the features or the training set, producing results that cannot be reproduced live.

Purging and embargo

Removing training rows whose target overlaps the test period, and a margin after it, when targets span several days.

Baseline

A naive forecast, such as zero return or persistence, that every model must beat on the same days.

Sentiment analysis

Turning a text into a score of tone, here between minus one and plus one per headline.

Ablation study

Removing or adding one family of features at a time to measure its marginal contribution.

Stacking

Combining the forecasts of several base models with a meta-model trained on their out-of-sample forecasts.

Diebold-Mariano test

A test of equal predictive accuracy based on the daily differences of the losses of two forecasts.

Deflated Sharpe ratio

A Sharpe ratio corrected for the number of strategies tried, compared with the maximum expected by luck.

Drift

A change over time in the inputs, the sources or the performance of a model in production.

The other notes of the course

One public summary per deck, in the order of the course.

Previous: Credit default scoring projectNext: Python code for every ML model

1. IntroductionAI winters, jobs and method
2. ToolsKaggle, GitHub, Python: the tools
3. Session 1Project framing and least squares
4. Session 2Maximum likelihood and logistic regression
5. Session 3Gradient descent, ridge and lasso
6. Session 4Decision threshold and reproducible ML
7. HomeworkA defensible baseline model
8. Session 5Kernels, SVM and decision trees
9. Session 6Forests, boosting, honest comparison
10. Session 7SHAP, fairness, PCA and clustering
11. HomeworkA documented champion model
12. Session 8Neural networks, drift, conformal prediction
13. HomeworkA delivered machine learning project
14. Session 9Defending a machine learning project
15. ReferenceCredit default scoring project
16. ReferenceAsset returns: XGBoost and sentiment
17. ReferencePython code for every ML model
18. ReferenceDeep learning essentials
19. ReferenceOpen data ML projects by sector
20. ReferenceData and AI engineer handbook
21. ReferenceData engineering, SRE and cloud
22. ReferencePanorama of machine learning models

All the notes, as a reading list

Attention and transformers, step by step

About these notes

Written by the lecturer from the slides of the deck, as a public summary: the key ideas, the formulas and one worked example. The exercises, their solutions, the tests and the grading stay in the deck, for the students of the course.

Revision: 16 October 2026.