For backtesting purposes, quick and dirty, I’ll generally say that 15% average daily stock total should be more than my anticipated average portfolio position. So if my average port position is going to be $20k…
(MedianDailyTot(120) * .15) > 20000.
I’m actually able to trade with less liquidity than this in the real world without effecting slippage too much.
I have rarely scrapped a model for outright losing money. A few times, mainly because I had a thesis that I built a model around and the thesis was wrong, it really had nothing to do with over fitting the model, per se. Or maybe more accurately it wasn’t wrong, I just grew frustrated with it or wanted to put my money into something better. One example is I built an entire ranking around Sin/Vice stocks like tobacco/gambling/booze under the conventional wisdom they would make good defensive hedges against my high flying microcap model. Well, turns out my main microcap multifactor model actually performed even better in the downturns than my “defensive stock” model once you start putting things in like industry momentum. If you have a lot of factors cash flow growth, industry momentum, low accruals the model will just kind of naturally move to whatever industries the economic regime is moving into. I have chased performance trying to build models around high flying themes of the moment, like a cloud industry model. I didn’t stay with it, but it most certainly would have blown up … not because the model was “over fit” but because the entire concept would have blown up. These concepts would also not work if I were buying industry based ETFs or actively researching and picking stocks under the same premise.
The main changes I make are because new P123 keeps releasing new features, factors, markets, etc. that I play around with and want to add. The P123 community keeps sharing new findings. I keep finding better testing approaches. Sometimes I realize those approaches might not be best and take a step back. Like if you’re constantly seeing how successfully you can trade a certain liquidity of stock , you might keep pushing the boundaries of liquidity lower until you’ve realized you’ve gone too far and it’s hard to get out of positions or blowing up the price on the buy. Then you got to back up. Lowering turnover has been a constant theme as I’ve morphed from someone who originally didn’t mind high turnover in order to achieve maximum alpha, to now trying to minimize transaction costs and trading as much as possible. Sentiment factors and price action momentum (and valuations) can really drive turnover. It’s all trial and error, even with live money, and getting to know your temperament as an investor and/or trader.
I started out with the same public available ranking systems everyone else starts out at and built my way up. Generally what has cost me money is not overfitting models. It has been poor execution on trades. It has been poor attempts at market timing (both “panic selling” and systemic market timing). It has been not fully exploring and utilizing whatever tax advantage possibilities I had available to me. The compounding effects of slippage and taxes definitely eat away.
My main ranking system has 3 previous successful ranking systems that I’ve used in real life that I’ve successfully traded all sandwiched together. Each ranking system has about 35-60 factors. I equal weight the three together to give me an average, attempting to get a signal from the noise of what makes a good business. I’m pretty much at the node limit of what P123 allows. If we ever expand node limits, I would like to do more industry specific rankings within the same ranking system. There is a lot of overlap in the factors. Many of the factors are in all 3 rankings, but just grouped in a different way or in composite factors. I’m sure if an actual statistician looked at it their head would explode with all the correlations. But I’m not running an academic exercise or training an ML model off it. I’m just picking good businesses, just like Warren Buffett, I just like my ranking system save me time by going through the fundamentals of thousands and thousands of companies in a few seconds to pick them for me in a systematic rules based way. And I’m extremely satisfied with the results for several years and several different regimes.