Does Past Performance Predict Future Returns: A look at designer models

[Jrinne]

So I am with you I think on at least some of the backtests being misleading. I think the data is noisy at best. My study was slightly different though. It actually did not include backtest data for example. We happen to be discussing two different topics.

My study looked at 4 years of out-of-sample data (not just 3 months) to see if it could predict the next year’s return. It was done in a rigorous way with the data we have. The only problem being survivorship bias.

I think many are just asking for the removal of survivorship bias, BTW.

I wonder if 3 months of out-of-sample data would show anything different. But please, redo the study as you wish. I initially made this a Claude Cowork Artifact to share (and fork or customize), but for some strange reason Chat Artifacts can be published but not Cowork Artifacts. I tried to do that so that people could look at several periods if they wished to. If you want me to I will produce an Artifact for a 3 month period. But I think you can get Claude to do that without me independently which may be more convincing whatever you find in the data.

I agree with you on several important points for sure:

1. You have asked for excess returns (everywhere) before. I agree that would help including with assessing Designer Models as well as machine learning models.

2. We should have access to more raw data. Maybe let Claude, ChatGPT or Gemini sort it out. The Designer model Excel Spreadsheet could have more data in my opinion.

3. I am not against not allowing people to invest for 3 months on a model. I am fine with that or actually I do not care. I do not Design models or sub to them so you guys decide that. But the idea of including of all models submitted before the blackout and removed as part of a designers record is important in my opinion–as many have said above:

   @ScifoSpace

   This should be a must for transparency @marco

4. I hope I have given a heart to everyone who wants to asses all of the Designer models without survivorship bias. P123 and FactSet do a lot to avoid survivorship bias in their data. I don’t understand why designer models should be any different.

5. Above are just a few examples of data we could use. But more raw data of any kind would be beneficial. Here is another request:

Why the poor live results versus backtesting?

Great analysis! As a supplement, we can calculate the performance decay using this simple formula: Performance Decay = 1 - (Live Excess Return / Backtest Excess Return) Using the numbers from your example: 1 - (-3.5% / 21%) = 1.1667 (representing a 116.7% decay) @marco, what are your thoughts on implementing a decay metric directly into the platform? I understand, we are still waiting on the 'Return 10Y' feature, so I understand it might not be a quick update, but it would be very helpful…

TL;DR: I like your ideas and cannot see a single thing I disagree with. I don’t really care what members decide to do with regard to any embargo or blackout. I would prefer that were transparent however.. As for my study, the data did not behave the way I wanted it to. I am truly surprised by it. Make of it what you wish. Or do your own study if you want. I would love to see the results.