Skip to content
Menu
Finabase Learn

Hedge funds and AI · AI in investing

Machine learning in quant funds: features, overfitting and regimes

How quant funds turn data into features and labels, split it into training, validation and test sets, and fight overfitting, data snooping and markets that change.

On this page
  1. The parts of a quant model
  2. Training, validation and test
  3. Overfitting: more predictors, worse forecasts
  4. Why financial data is small and noisy
  5. Non-stationarity and regime change
  6. What quant funds do about it
  7. Where machine learning in quant funds fails
  8. How Finabase does it
  9. Frequently asked questions
  10. Summary
  11. Check your understanding

Machine learning in a quant fund is less glamorous than the name suggests. Most of the work is deciding what to measure, lining the data up so the model never sees the future, and then distrusting the result. The models themselves, from linear regressions with a penalty to decision trees and neural networks, are widely available. What separates a useful one from an expensive mistake is how it is tested.

Financial data is unusually hard for machine learning. There is little of it compared with images or text, it is mostly noise, and the patterns in it change as markets and the people trading them change. This lesson walks through the pieces of a quant model, shows with a simulation how adding predictors makes the past look better and the future worse, works out how many years of data it takes to tell a real edge from luck, and explains what funds do about it.

The parts of a quant model

Every supervised machine-learning model in investing has the same skeleton.

  • Features are the inputs: numbers known at the moment of the forecast. Examples are a stock's return over the past 12 months, its price-to-earnings ratio, the change in its trading volume, the size of its last earnings surprise or the tone of its latest earnings call.
  • The label is what the model is asked to predict: next month's return, the return relative to the market or to the stock's industry, or simply whether it beats the median stock.
  • The model is the rule that turns features into a forecast. It may be a linear regression, a penalized regression such as the lasso or elastic net, a collection of decision trees, or a neural network.
  • The portfolio step turns forecasts into positions: buy the stocks with the highest forecasts, short the lowest, size them by risk, and hedge out the market so the result depends on the ranking.
  • Costs and capacity come last but decide the most. A forecast that needs heavy daily trading in small stocks can lose everything to spreads and market impact.
Piece A simple example A common mistake
Feature Return from 12 months ago to 1 month ago Using a figure that was restated or published later
Label Next month's return minus the market's return Predicting raw returns and so mostly predicting the market
Model Elastic net on 20 features A large network on a few hundred observations
Portfolio Long the top 10%, short the bottom 10%, rebalanced monthly Ignoring trading costs and borrow costs on shorts

Many of the classic features are the factors described in Factor investing: five factor funds against the S&P 500, tested: value, momentum, size, quality and low volatility. Machine learning adds the ability to combine many of them and to find interactions, for example that momentum works differently in small and large stocks.

Training, validation and test

A model has to be judged on data it did not learn from, so the history is split into three parts.

  1. Training data fits the model's parameters.
  2. Validation data chooses the settings the training does not fit, such as how many trees, how strong a penalty or how many layers. Trying settings and keeping the best is itself a search.
  3. Test data is kept locked until the end and used once, to estimate how the finished model might do. If the result disappoints and you go back and change the model, the test data has become validation data, and you no longer have an honest test.

In finance the split must follow time. A random split, standard in many other fields, would let the model train on March and be tested on February, learning from information that did not yet exist. Overlapping labels cause the same leak more quietly: if each label is a 12-month return, two neighbouring months share 11 months of outcome, so a gap is left between the training and test periods. A fund might train on 1990 to 2009, validate on 2010 to 2016 and test on 2017 onward. Out-of-sample and walk-forward testing, worked on SPY shows a single split and a rolling, walk-forward version on real data.

Overfitting: more predictors, worse forecasts

Overfitting is a model learning the noise in its training data as if it were signal. The more freedom a model has, the easier this is. The chart comes from an invented setting chosen to resemble stock returns: 240 months of training data and 240 new months for testing, 60 candidate predictors, and only 3 that matter. Together the 3 real predictors explain 3% of the variation in the outcome, a generous amount for monthly returns. The other 57 are pure noise. A linear regression is fitted with the first 1, 2, 3, then up to all 60 predictors, the real ones first, and the whole exercise is repeated on 200 invented datasets.

More predictors: a better fit to the past, a worse forecast (simulation) -40% -30% -20% -10% 0% 10% 20% 30% 40% 0 10 20 30 40 50 60 No better than the average Training data (in-sample R²) New data (out-of-sample R²) 2.1% 27.3% −30.8% Number of predictors in the model (3 real ones first, then noise)
Finabase simulation, averaged over 200 invented datasets: 240 training and 240 test months, 60 candidate predictors of which 3 are real and together explain 3% of the variance. Out-of-sample R² is measured against the training average.

The two lines move in opposite directions:

  • On the training data, every added predictor improves the fit, real or not. With all 60 the model explains 27.3% of the past, nine times what is really there.
  • On new data, the model with the 3 real predictors does best, at 2.1%. That is below the true 3% because even the real coefficients are estimated with error. Adding noise predictors makes it worse: by 10 predictors it is −0.8%, worse than simply forecasting the training average, and with all 60 it is −30.8%.

Real research shows the same pattern. In Gu, Kelly and Xiu's study of nearly 30,000 US stocks from 1957 to 2016, an ordinary regression on their full set of more than 900 predictors had a deeply negative out-of-sample R². Methods that restrain the model did better: the elastic net, which shrinks and drops coefficients, reached 0.11% a month, and tree models and neural networks reached 0.33% to 0.40%. Their neural networks did best with three hidden layers and worse with more, which the authors put down to the small amount of data and the tiny signal-to-noise ratio compared with fields such as computer vision.

Overfitting also happens across researchers. Thousands of people test ideas on the same few decades of US stock data, and the successes get published. Campbell Harvey, Yan Liu and Heqing Zhu counted 316 factors proposed in the research literature and argued that, after so much searching, a new factor should clear a t-statistic of about 3.0 rather than the usual 2.0. A fund that reads the literature, adds its own search and keeps what works is several layers deep in selection. Overfitting and backtest bias: 4,094 rules tested on SPY shows the arithmetic of many tries with trading rules.

Why financial data is small and noisy

Sixty years of monthly data is 720 observations per stock. Daily data gives more rows, but daily returns are noisier still and neighbouring days are not independent experiments. Image and language models train on millions or billions of examples with strong, stable patterns. In markets the signal is small next to the noise, and the noise is large.

The chart shows what that means for anyone trying to prove an edge from its own track record. If yearly excess returns are independent and roughly normal, the t-statistic of the average return is the Sharpe ratio times the square root of the number of years.

Years of returns needed before a real edge stands out from luck To reach t = 2 (the usual test) To reach t = 3 (a stricter hurdle) 64y 144y Sharpe 0.25 16y 36y Sharpe 0.50 7.1y 16y Sharpe 0.75 4.0y 9.0y Sharpe 1.00 1.8y 4.0y Sharpe 1.50 Arithmetic: t = Sharpe ratio × √years, yearly returns independent and normal.
Arithmetic, not a fund’s record: years = (t ÷ Sharpe ratio)², assuming independent, normally distributed yearly returns.

Sharpe ratio: return per unit of risk, calculated on real funds explains the ratio itself. The point here is that most real edges are too small to confirm from returns alone in any reasonable time. That is why quant researchers lean on an economic reason for a signal, on testing it across many stocks and countries at once, and on skepticism about anything that looks too good.

Non-stationarity and regime change

A stationary process has statistical properties that do not change over time. Markets are not like that. The relationship between a feature and future returns can drift or break because:

  • conditions change, such as interest rates moving from near zero to several percent, which changes how investors value distant profits;
  • market structure changes, through decimal prices, electronic trading or new rules;
  • other traders learn: once an edge is published or widely traded, prices adjust earlier and the edge shrinks; and
  • the data itself changes, when a vendor revises its method or a company changes how it reports.

FINRA's 2026 oversight report uses the term concept drift for a related problem in AI tools: outputs going wrong because the data a model learned from has gone out of date. In trading, the remedy has a cost. Retraining on recent data keeps a model current, but recent data is even scarcer, which makes overfitting worse. Retraining on a long history gives more data but mixes regimes that may no longer apply. Why trading strategies stop working measures how edges fade on real data.

What quant funds do about it

There is no cure, only discipline. The common defences:

  • Start from a reason. A feature with an economic story, such as investors underreacting to news, is less likely to be a fluke than one found by search alone.
  • Restrain the model. Penalties, few features, shallow trees and early stopping trade a worse fit to the past for a better chance in the future.
  • Combine many weak signals. Averaging several models, or many small signals, reduces the damage any one fluke can do. Jim Simons and Renaissance Technologies: the Medallion Fund record describes a firm reported to have built its record on very many small edges rather than one big one.
  • Test honestly and count the tries. Split by time, leave gaps, use the test set once, and record every version tried.
  • Start small. Paper trading and small live positions show costs, delays and data problems a backtest cannot.
  • Watch it live. Compare live results with what the model expected, and cut risk when they diverge.

Where machine learning in quant funds fails

  • Leakage. Restated figures, today's index members or a random split slip future information into the past and produce results that cannot be repeated.
  • Costs and capacity. Many forecasts are strongest in small, illiquid stocks, where trading costs are highest and little money can be invested.
  • Crowding. Funds using similar data and models hold similar positions and can be forced to sell together.
  • Opacity. When a complex model loses money, it can be hard to tell a bad stretch from a broken model.
  • Regime breaks. A model trained on calm years has never seen a crisis, and the opposite is also true.

How Finabase does it

Finabase does not train machine-learning trading models for its users or run a fund. Its research tools make the same testing problems visible on a small scale. The Backtest tool (Backtest indicator signals and rule-based strategies on Finabase) trades every signal of the indicators you switch on, on one symbol over the range you choose, with a fixed $10,000 per signal and no trading costs. Each time you change a parameter and run it again, you are searching, exactly as described above, and the tool does not count your tries. The Strategy tool replays if/then rules over two years only, a very short sample. Treat a good result in either as a description of the period tested, then change the range and see whether it holds. Practise with Finabase paper trading on the web lets you follow a rule with practice money before anything real.

Frequently asked questions

What is a feature in machine learning for trading?

A feature is an input the model uses to make a forecast, such as a stock's past return, a valuation ratio or an earnings surprise. It must be something that was known at the time of the forecast.

Why can't you split stock data randomly into training and test sets?

Because markets unfold in time. A random split lets the model learn from later periods and be tested on earlier ones, which leaks future information and makes the test look better than real trading could be.

Does a more complex model predict returns better?

Only up to a point. In the simulation here, and in published research, restrained models beat unrestrained ones on new data. Extra flexibility mostly fits noise, because financial data is small and noisy.

How long a track record is enough?

Longer than most people think. With independent yearly returns, a Sharpe ratio of 0.5 needs 16 years to reach a t-statistic of 2. A record of a few years cannot separate a modest skill from luck.

Summary

A quant fund's machine-learning model turns features known at the time into forecasts of a label such as next month's relative return, then into a hedged portfolio. The hard part is testing: splitting data by time, keeping a test set untouched, restraining the model and counting every try. In a simulation, adding noise predictors lifted the in-sample fit from 4.4% to 27.3% while the out-of-sample result fell from 2.1% to −30.8%. Financial data is short, noisy and changing, so even real edges take decades to prove from returns alone, and the funds that last are the ones most suspicious of their own results.

Check your understanding

4 questions on this lesson. Each answer comes with the reason, and nothing is scored.

  1. A researcher adds 50 more predictors to a stock-return model. Its fit to the training data jumps, but its R² on new data falls below zero. What happened?

    1. The extra predictors let the model fit noise in the training data, which does not repeat
    2. The new data must contain errors, because a better fit always means better forecasts
    3. The model needs even more predictors to recover
    Show the answer

    A. The extra predictors let the model fit noise in the training data, which does not repeat Every added predictor improves the in-sample fit, whether it is real or not. In the simulation, all 60 predictors fitted 27.3% of past variation but scored −30.8% on new data. A negative out-of-sample R² means the forecasts did worse than simply predicting the historical average.

  2. Why should a quant researcher split stock data by date rather than at random?

    1. A random split lets the model learn from later periods and be tested on earlier ones, leaking the future into the test
    2. Random splits are slower to compute
    3. Regulators require that training data be older than five years
    Show the answer

    A. A random split lets the model learn from later periods and be tested on earlier ones, leaking the future into the test Markets unfold in time, and features and returns in neighbouring periods overlap. Training on March and testing on February uses information that did not exist yet, so the test looks better than live trading could be. Splitting by time, with a gap, avoids that leak.

  3. A strategy has a Sharpe ratio of 0.5 and its yearly returns are independent. About how many years of results are needed for its average return to reach a t-statistic of 2?

    1. 16 years
    2. 4 years
    3. 2 years
    Show the answer

    A. 16 years The t-statistic is the Sharpe ratio times the square root of the number of years. 0.5 × √T = 2 gives √T = 4, so T = 16. A stricter hurdle of 3 would take 36 years, which is why track records alone rarely prove a modest edge.

  4. What does it mean that financial markets are not stationary?

    1. The relationship between a signal and future returns can drift or break as conditions, rules and other traders change
    2. Prices never move on days when markets are closed
    3. Machine-learning models cannot be used on market data at all
    Show the answer

    A. The relationship between a signal and future returns can drift or break as conditions, rules and other traders change A stationary process keeps the same statistical behaviour over time. Markets change as interest rates, market structure and participants change, and as edges become known and are traded away. Models are retrained and tested on data they have not seen for that reason, at the cost of having even less relevant data.

Sources and further reading

  1. Shihao Gu, Bryan Kelly and Dacheng Xiu, “Empirical Asset Pricing via Machine Learning,” NBER Working Paper 25398 (December 2018, revised 2019); published in the Review of Financial Studies 33(5), 2020
  2. Campbell R. Harvey, Yan Liu and Heqing Zhu, “…and the Cross-Section of Expected Returns,” NBER Working Paper 20592 (October 2014); published in the Review of Financial Studies 29(1), 2016
  3. FINRA — 2026 Annual Regulatory Oversight Report, GenAI: Continuing and Emerging Trends (December 9, 2025), on model testing, monitoring and concept drift
  4. Finabase calculations: a seeded simulation of 200 invented datasets (240 training and 240 test months, 60 candidate predictors, 3 of them real), and t-statistic arithmetic