@danparquette @marco I see this:
And then I see this.
How do I reconcile this with anything greater than 13 week predictions? Thank you.
I'm wondering if my question makes sense since I haven't heard back. Do I misunderstand these notes and how longer terms are tested here?
Hello korr123,
I am not sure I understand what you are asking. Selecting a long term target (like 12MRel) is unrelated to the setting you select for the frequency of the data in your dataset.
Or maybe you are asking for the ability to use a 1 year frequency in the validations? I would expect that limiting the model to so few data points during training would lead to poor training results. Do you have reason to believe that it would be useful?
If you plan to have 1 year holding periods in your live strategy and want to backtest that, then you could create simulations with 52 week rebalances which use a series of rolling predictors each created so there would be no data leakage. Functionality to accomplish this is something we are looking into adding.
Ideally, the sampling should be unrelated, but the text indicates that the portfolio results are rebalanced more frequently. This means the results don’t adhere to the one-year predictions. In this scenario, the target is one year out, with the "target lookahead" automatically set to 52 weeks, yet the rebalance frequency is capped at 13 weeks. Does this explanation clarify things better?
As I understand it, the sampling frequency is simply about sampling. Best practice is to sample as frequently as possible when testing for empirical results, as we are doing here. I don't see why anyone would want to sample less frequently. Just a side note.
So, yes, I think I understand you are saying that the 12-year target testing isn't complete yet?
Setting the target to 12 month returns doesn't mean it will use the 12 month returns in the performance results. It just means that the 12m return is what the model is trying to predict. If the target is 12 month returns, that target is calculated at every rebalance date regardless of the rebalance frequency.
The part that is not complete yet is the ability to run a simulated strategy as a way to see the results if you wanted to hold each stock for 1 year.
How do you suggest we test the 12 month return targets, or anything longer than the rebalance windows?
Its confusing to use the term target because the target is totally unrelated to the return calculations. I think what you are asking for is a separate setting for 'holding period' ie run the validation at every date according to the Frequency setting but the performance data should show the returns assuming every stock was held for the 'Holding period' ie 12m in your case. If we add that holding period functionality, then the only limitation is that the holding period would have to be a multiple of the sample frequency because the dataset would have to contain all the required prices to calculate those returns. Would that meet your needs? It seems like a good idea and certainly easier for the user then the other alternative of running a simulated strategy that I mentioned above.