How are your results using AI Factor out of sample?
I had an unexplained large discrepancy between backtesting using AIFactorValidation & AIFactor so decided to run some results forward without investing capital into it. The 2 systems that I selected 2 months ago have both unperformed. I selected these after many tests using predominantly using K fold cross validation with a hold out set using both selected universes and open universes and with selected factors and an open factor list to minimize the chance of overoptimizing. This is too short a time period to make a conclusion but so far my results do not seem usable.
Please offer suggestions if you have had better results
I have 1 live (20 stock) strategy that uses AI Factor and that strategy went live exactly 6 months ago. It is only 6 months of data, but during those 6 months it has been my best performing live strategy. These are all my live strategies for comparison:
I described the process I followed back when I created this AI Factor. See this forum post: system design
Finding free time has been an issue, but I am looking forward to creating another AI model soon and utilizing functionality we have added since I created my first model like model level feature importance, grid search and normalization options. I plan to document my steps in hopes it will help others and also to get feedback from P123 members who are much more knowledgeable then I am.
Thank you for your useful & humble response. Great job on your ports. I used your framework for building the 2 sub optimal ports that I've been running out of sample without capital but will revisit your post.
There's nobody else that is satisfied with their out of sample AIFactor returns?
It's up 2% today when market is down around 1-2%. So ~5% in two months, which is in line with my testing across various p123 tools.
However, the volatility is too high (you can see the drawdowns) and 85% of the shorts are in healthcare. That's hard to accept.
So with that kind of volatility, a black box AI factor, and no ability to control risk (I'm forced to use the screener which doesn't have the same features), it's essentially foolish to actually use the system.
This is why I keep pushing for a risk model.
I wish I could test and manage the system by hedging sector exposure, controlling beta (you can see the realized beta is -.87), or target volatility. The screener indicates it's supposed to average to 0, historically over time.
