So, a simple question: Is P123’s data PIT now or not? The above would suggest that maybe it is and there would be no better news for me than to find this out.
I appreciate that P123 is working hard on this. Working hard on a difficult task. But I shut down all ports based on any FactSet data not too long ago. P123 has become a method for using InList for me. InList to import data this has been proven out-of-sample. For the simple reason that out-of-sample data is PIT data.
I continue to see differences in my sims and ports out-of-sample. Significant differences in the performance, differences in the tickers selected and differences in the ranks recorded for the tickers between the sim and port in the transactions. Differences in the ranks for transactions on the same day.
I had stopped posting on this because I figured P123 had done all it could. I gave up and went to using InList.
I post now only because it seems possible that the differences in the sims and ports are something trivial like the 3 am update or an error on my part.
Just tell me the data is PIT now with minor differences in sims and ports due to something trivial like the 3 am update if you can, please.
I saved my ports–stopping rebalance–if I made an error that P123 wants to show me. And if I did make an error, my apologies. It is not my purpose to make any point on this forum. I would just prefer to know for sure if I can use P123 for more than importing data using InList.
Put in the simplest terms, my sim continues to perform well. Continues to do well out-of-sample. Overfitting the sim is not a problem here. But the port did not do as well as the sim continues to do, and honestly, I do not think the sim is realistic (image below). But I leave it as a question: Realistic?
If my perception is wrong on this I would be happy to simply go back to using it. Granted, it is a 5-stock model but it is very liquid and very tradable. If it were real, it could be (and should be) a small part of a diversified portfolio. Average of High Low, liquid universe with variable slippage.
And would the 25-stock sim–which also does well–be a good port?
Is there a type of data I can remove to get realistic sims? I am happy to do some cross-validation to get a more robust port, but if it is data problem I will never be 100% sure the port will be as good as the sim no matter how many cross-validations I do. I will be left guessing if the port is good and will bail on any significant drawdown. Even if it is a good port it will not end up working for me because of any doubts I have about the data.
Happy for any information on this. Confirmation, prove me wrong, doesn’t matter as long as I learn the best path forward for this sim/port and ones like it in the future (if I ever run another sim).
Jim