All,
I have a “standard model” for comparison. It is simply what I would like to beat to make moving to a more complex machine learning model worthwhile. Details of the standard model would be difficult as I do not wish to share it. And let me stipulate that it probably has nothing to do with what others are doing with their models. But it is all I got. I offer it as comparison for this post only and encourage people to make their own comparisons.
I wanted to compare my standard model to an extra trees regressor model available in Sklearn: ExtraTreesRegressor
This uses k-fold cross-validation with a 3-year embargo period and this is not directly comparable to a backtest. I find backtests will show higher returns. Also this uses the close and there is no slippage. So the numbers should be used as comparison only. Anyway, the results:
Discussion: There is some improvement for this ‘screen-like test’ with no slippage using closing prices for models with 30 or less stocks. The improvement is not super-large (you can judge for yourself).
I look forward to when P123 implements its AI/ML. I think they will do something similar and provide further information regarding drawdowns, turnover, slippage, buy/sell rules based on rank etc. I.e., something you could put into a sim or live port.
Sorry for the length of this (I worked on reducing the length while still being clear). I do think it is totally supportive of what P123 is doing with machine learning and I think it is an appropriate topic for the forum.
Thank you P123 for the DataMiner download whatever your AI/ML implementation looks like going forward. But I am pretty sure it will be awesome and useful based–to some extent–on my findings presented here which stated simply are: AI/ML does seem to work and I have presented some data to support that. Factual data based on around 5 million rows of data using cross-validation and a long embargo period. A start at least. Totally supportive of what P123 has done and is doing.
Jim