Would it be possible to allow the AI factors to be backtested over the full period and not the 5 year max? This would allow full testing of different universes if the AI factor was trained on a different universe. It would also allow for some robustness testing of adding buy rules or testing only on a fraction of a universe.
1 Like
P123. Any thoughts here?
It will be useful or course. If I not recall bad sooner or later @portfolio123 will do it.
I have an interesting thought on this. Allowing AI factors to be used pre 2020 may be sub optimal. The reasoning is that alot of technology we would be using now to back test for the 2010s was not available as readily at that period of time so arguably there is more capacity in those periods than there really would be looking forward. Just a curious thought of mine.
My use case is really for stress and robustness testing. I would want to backtest on the full period, and exclude sectors, restrict industry weights, split universes, use completely different universe for backtesting than what the model was trained on, etc.
Any further thoughts on this. Being able to test over the full period is really holding me back from using. Even though the backtest would run over most of the learning testing/learning period, I want to see how it holds up with various buy and sell rules, rank tolerance, etc.
1 Like
Would it be a big no-no in that case to use AIFactorValidation(…) to use saved validation predictions from the holdout period from something like k-fold to do your backtest to review what it would look like with your other rules in the absence of longer AIFactor(…)?
AIFactor() function is limited to 5 years of backtesting because prediction is an expensive, slow-running operation that involves sending the dataset for the chosen universe to the AI backend, normalizing the data, and calculating predictions. The limit is more of a resource/cost imposed limit than an implementation issue.
1 Like