I have done a lot of testing, like:
- many features (500), or a few of which perform well in RS (90)
- I have tested large universes and small universes
- I have tested different periods
- I have tested with Frequency Every 4 Weeks and 1 week
- I have tested with all validation models and validation methods
- I have removed features with low Target informations regression
- I have removed features with high NA
- and more
This gives OK results when I run it with settings like this:
So, I have trained the predictor on the entire period of 2004-2019 but excluded the last 5 years to test it out of sample in a simulator and screen backtest.
Here is where the problem arises - it yields very unstable and volatile out of sample performance:
This models gives a good performance, but moves sideways most of the time. When testing this in the simulator it gives even worse scenario. This is pretty much the same on a lot of the models.
What settings do you use, and is there something I am doing wrong since the results turn out like this? Is there something I have misunderstood?
Is there any settings that brings you closer to the results