Thank you Yuval.
And in purely mathematical terms: pretty darn good.
-Jim
5/24/2017 to 1/31/2019 is a tough period for the market. I don’t think that there are many models that did well over this period.
I actually like seeing the backtests and factor discussions. I learned a lot from those. if we move these to a separate group, i fear newer folk will not be able to learn by observation.
I would also like it the way it was, with free discussions and posting of ideas, charts and results.
Much to learn for everybody. Would be a shame to change that.
RT,
I am okay with some backtests myself.
But I believe Yuval’s Crazy Returns Mico-cap Model (his only public micro-cap model) says much more about micro-cap strategies than any backtested micro-cap model I would try to present on the forum.
If I present a great backtested model and try to convince you to switch from Yuval’s model, switching would be a mistake.
Does not mean you shouldn’t take a look at one of my backtests—if you don’t like the Super Bowl commercials this week-end and have nothing else to do.
Uh… Honestly, you should look at the Super Bowl commercials.
-Jim
I completely agree with the above.
I also particularly like the back tests and charts that George posts.
I now own a domain address for this, just in case.
P123BACKTESTS.COM
If the P123 community wants a forum away from P123, please let me know so I can get somebody to set it up for us.
Just complete the Golden Cross & Recession Timer (Stocks-Bonds) 1999-2020, but I can’t show it to you under the new forum regulations.
Not that it matters to this discussion but:
A machine learning specialist would not argue against backtesting. They might do some things to limit how much overfitting you can do but within those constraints they would want you to backtest a lot and get the best backtest possible.
They would always validate that, however, even if that meant paper-trading the model for while.
We see how that works with designer models. I suspect every one of the Designer Models backtested well.
Anyway, backtesting is just a tool–like a hand gun–that can be misused.
A discussion of backtesting in isolation does not advance the discussion much IMHO.
But it is fair to say everyone—including de Prado–backtests and tries to get the best backtest possible (within any constraints).
AND AT LEAST IT IS SOME EVIDENCE.
-Jim
Also here is what P123 has on their front page:
Be a cold blooded investor.
Create, backtest and invest in your own rule-based strategies.
There would not be a need for backtest if there was a perfect theory to select stocks based on functions, ranking, etc. and always make profit on every trade. If such a perfect rule based theory existed, and only a few insiders know it, they would be the richest people on earth.
If it’s not used to sell a DM, I like seeing backtest results, too. It motivates me to explore techniques that I would otherwise overlook. I gotta thank Georg for the use of ETFs and recession indicators in portfolio construction.
Walter
Maybe that should be the deciding factor.
Also I would not mind if P123 actually took some concrete steps to help us identify over-fitting.
I mean some of the methods the AI specialist P123 hired used.
And the steps Yuval took: no survivorship bias in his models. DOCUMENTED FACTS OF HOW IDEAS ARE WORKING OUT-OF SAMPLE.
-Jim
And I am very exited about mixing stock trend and recession indicator models. I would have posted the rules and backtest here,
but unfortunately the community will not be able to see them under the new forum regulations.
But all is not lost, I will post the article on Seeking Alpha shortly.
Backtesting doesn’t tell you a model whether a model will work, but it does tell you what hasn’t worked. I like backtesting, but I try to just throw something together without tweaking values to the upteenth degree. I think Marc Gerstein mentioned on his backtesting in the design guide that he’s not looking for perfection in his backtests but rather does an idea yield some alpha. He tried to take top percentiles like take top 30% value and apply a core quality + core sentiment ranking.
Not all Marc’s strategies have performed well but on the whole they are the better performing strategies.
It’s better to be in vaguely right than perfectly wrong (Marc’s quote). Investing is not a precise endeavor so I think we need to avoid precision in our backtests to get better trading strategies. Easier said than done. It’s natural to want to improve.
I would also never trust my money with one strategy even if it OOS tested well. 3 or 4 strategies at least. Then at least if a couple flunk you can maintain capital.
Jeff
Jeff,
I like this and have said it myself.
But do we know that Marc is beating the benchmarks? I think we do not despite the fact the there is much to like about his methods.
Are all of his sims and ports meant for serious investing?. Was Piotroski meant for real money?
Which ports are serious?
So if you take the serious ports (whichever ones those are) which ones should we be in? Which should we be out of now? I do not know.
I like Marc and his ideas. But I make no claim whatsoever about having an accurate idea of how all of that might work for me.
Maybe someone has seen more evidence than I have one way or the other.
The only evidence I have is that on average his ports are underperforming the benchmarks I think last time I checked. Saying some of those models were not serious is only an argument that I know nothing.
Okay I know nothing on this. But I do not find convincing evidence for much that is said—or any evidence at all for that matter.
-Jim
Jim,
I do think the last two years have been a difficult investment environment. There’s a lot of strategies that just haven’t worked or underperformed. I have a couple strategies that have outperformed in backtesting but they are either momentum or quality based. Definitely doesn’t include value or traditional growth. I can get a value strategy to perform if I compare within industry, but there is a lot of value in financials and energy now which the market has been avoiding. Even still that value strategy has underperformed the benchmark the last two years. This is a bond market and quality names serve as bond proxies. Value performs best in high growth, high rate environments. We are not in that domain right now. Also consider that most stock market appreciation has been due to market Goliath Faang and Microsoft stocks.
So I think you are going to find it nearly impossible to develop a viable value strategy that has beat the market the last couple years. That might be a good thing because the more value gets beat down the more it will be coiled for a rebound. Depending on how much the market corrects and whether a recession occurs this change might happen sooner than later.
Maybe look for a value strategy that has shown outperformance in the past including before 2008 and accept that now is not values time but it could be poised to spring back at any time. I think that’s as good as we can do. Not every factor is going to outperform all the time.
Jeff
Agreed.
Not what I would call evidence supporting Marc’s over Georg’s methods however. Which was the only point I was making.
I have no evidence on this at all and prefer not to assume.
-Jim
Can we have a poll to decide what the P123 users want to see in terms of backtests, and backtested theories? And if we want to as a group, be required to post a disclaimer? This may absolve P123 of responsibility while at the same time informing unsophisticated people (like me) of potential pitfalls.
I vote we keep Marco in charge of this and that he provide any clarification.
And Yuval too has been very professional with his models. Not many backtest (none that I remember) etc. I am happy to have him decide or relay any P123 policy.
I did not hear anyone say no backtests so maybe there isn’t really a problem.
I do prefer backtests over claims with no evidence at all which we have been seeing. We do need some backtest.
-Jim
Why not put a disclaimer on every post that is not in the official capacity of Portfolio123. Problem solved. I think most users understand that a forum is a free exchange of ideas, right or wrong.
I am for disclaimers and Designers should post the average annualized excess returns for all of their models at the bottom where my quote is. It should be attached and updated by P123, IMHO.
I do think that would provide a reality check for some of this.
-Jim