I am satisfied with the ranking system I currently use. However, especially after reading several posts from Algoman, that my AI-factor system is also becoming increasingly relevant. What I'm looking for are ideas or possibilities to combine these two.
I have attempted two methods:
-
* **Integrating AI-factor as part of the buy criteria:**
RatingPos("091025-eu-easyu-") < 300
* **Directly incorporating AI-factor into my primary ranking system and assigning it a weight, for example, 20%.**
However, neither of these solutions has provided better returns when I test them. This could be due to my ranking system being over-optimized for my specific universe, making any changes to the ranking system overly sensitive to alterations. Yet, when I conducted a test on a different, larger universe—one that my ranking system is not typically adapted to—I observed similar results.
Therefore, the question is: Has anyone had success in achieving effective combinations between two good systems, where one is a ranking system and the other is the AI-factor?
What I've done until now is to use my ranking system as the primary assessment for which stocks to buy. However, I also look at the AI-factor score, and if the number is very poor (e.g., higher than 400), I then choose the next stock with a better AI-factor ranking. I'm unsure if this is the best approach.
The optimal approach seems to depend on the specific system, but my approach is fairly close to your first method.
Buy rule: Avg(ml1, ml2, ml_3, .... ml_i) > 5 and Rank > 90
Sell rule: Rank < 80
I use the average of the raw predictions from X locally trained ML models—where the predictions represent the expected percentage future return—as an additional buy threshold.
For some periods, there are no candidates that meet the buy criteria. As a result, there are periods when my portfolio is not fully invested.
Interestingly this approach (raw predictions) allows to silently incorporate macro factors / market timing as one of the feature in ML model. You can see that my exposure actually decreased before market boom in March/April. But then unfortunately stayed lower during rebounds so the system is not yet perfect 
1 Like
Thank you for the feedback.
How have you managed or would you manage to backtest such an idea to determine if it actually improves the ranking system or your simulation?
I think that combining two different systems, especially when the backtesting methods behind them are so diverse, should be optimal to avoid overfitting
I compared the out-of-sample results of:
Blended system
Buy rule: Avg(ml1, ml2, ml_3, ..., ml_i) > 5 and Rank > 90
Sell rule: Rank < 80
vs.
Ranking system only
Buy rule: None
Sell rule: Rank < 80
This blended approach works well for Polish stocks, mainly because my linear ranking system has been underperforming the rapidly rising Polish stock market recently. The ML component seems to help in this environment.
For US stocks, however, my linear ranking system has been performing really well, so the additional ML filter does not seem to be as necessary.
1 Like
I’ve been fighthing this battle for a while in testing. I generally put Rating(“StaticRank”)>95 in a Buy rule with AI Feature as the ranking system in the porfolio. However, more importantly, I’ve generally found that to get a true 1 + 1 = 3 effect of combining AI Factor and Ranking System their features/formula nodes need to be orthogonal to each other. If my AI factor was just trained on features contained in my ranking system I don’t get much additive effect adding them together in my backtests.
