TL;DR: Tabular Foundation Models (TFMs) are to machine learning what Large Language Models (LLMs) are to language—and both are still in their infancy.
Unlike the models we use at P123, TFMs do not require hyperparameter tuning, and remarkably, you don’t even need to specify a model (e.g., Boosting or Elastic-net). They are true zero-shot models capable of handling classification, regression, and time-series forecasting out of the box. Rather than training and validating on your specific dataset from scratch, they are pretrained on millions of datasets and synthetic examples—mirroring how LLMs use massive examples of text from the web.
I wanted to test a TFM in practice—combining an LLM with a tabular foundation model—using Antigravity and TabPFN.
I tested it on a time-series market timing model. I chose this because current TFMs struggle with large row counts; the 4 million+ rows in a typical P123 stock model are simply too large for today's TFMs. Plus,TabPFN would start charging for use of its API if I used too much data.
So everyone will have to find their own use case and decide whether the paid API is worth it I might try it on my slippage data as slippage data is not really linear in my experience.
Here is my first attempt using price series and macroeconomic data (Fed data):
Honestly, the results remain limited here by the difficulty of market timing (or possibly by a poor choice of features in this example), so this might not be the best use for TFMs.
However, what stands out is that it outperformed LightGBM with zero hyperparameter tuning. This matches what the literature is suggesting: TFMs can outperform gradient boosting without any tuning.
Not just can be better but often are:
