I have noticed that my AI models do not benefit from including MarketCap as a factor. I have tried normalizing it across the entire dataset as well as by date.
However, if I train the model on a universe of 1000 or more stocks, 9 times out of 10, I achieve significantly better results when backtesting with the predictor adding a 5% MarketCap weight or so in the ranking system.
Does anyone have a good theory as to why MarketCap does not work well as a factor? Or am I using it incorrectly?
I have seen the AI system pick up other features that I think is beeing used as a size factor, typically factors I wouldn't use as a size factor my self.
Thanks for sharing. I'll offer a hypothesis:
I think part of the issue is that AI models decide which factors to include based on statistical significance, while ranking and backtesting don’t work the same way. It’s kind of like comparing apples to oranges.
It might help if you share how you tested MarketCap—did you use rank in both cases? That could explain some of the difference you’re seeing.
Also, the small-cap effect hasn’t really held up for a while now. The S&P 500 has been crushing small-cap indexes for years. In my experience, MarketCap is better as a pseudo filtering tool to find less efficiently covered stocks. That seems to match what the P123 community has seen too.
Like, just backtesting or even ranking the way P123 does it doesn't prove statistical significance. We're often left guessing and using subjective measurements (not sure why).