Hi Mike:
I sympathize with the challenge you’ve encountered finding models you like. In addition to the tips others have given about studying the designer’s descriptions of the methods, I also like to get a handle on the track record of the individual designers.
Here’s how I did this most recently (beginning of 2018).
I downloaded the full designer model data set from P1213 into Excel
I highlighted the column for 1 year returns (ie 2017 returns)
I highlighted the column for 2 year returns (ie 2016 and 2017 returns)
I created and highlighted a column for 2016 results (2 year return - 1 year return, a rough estimate for 2016)
Since I also have 3 live models of my own (real results not back tests), I added these to the designer models list.
Then sorted by Designer.
I created 3 new columns in Excel.
. AVERAGE 2016 returns (which was the average returns for ALL the models by a designer)
. AVERAGE 2017 returns (which was the average returns for ALL the models by a designer)
. AVERAGE 2016-2017 returns (which was the average returns for ALL the models by a designer)
I had to calculate these averages manually because I don’t know how to write Excel Scripts.
If a designer had fewer than 3 models (with at least 1 year history), the designer did NOT get a score.
Then I sorted by AVERAGE 2016-2017 results.
I only considered models by top 5 designers (actually only 4 designers since I ended up being in the top 5)
Of these models, I rejected any that did not have at least 2 years of history.
Some models largely duplicated my own, so they got excluded.
Some models did great in 1 year but not the other, so they got excluded.
Some models were fully subscribed so they ended up on a watch list (got one of them after a couple months of waiting).
One of these designers got excluded because 2 of his 3 of his models were loosers for 2016-2017. His other one was big winner, so big that his average score put him in the top 5 designers. He has 5 more models with less than 1 year’s history so next year I’ll have 8 to use to evaluate him and then things might look better. But for this year I passed on his one good model because it just might be luck. Time will tell.
I ended up signing up for a total of 3 models (2 from one designer and 1 from another).
So far I’ve not put money into any of them. I subscribed not to immediately trade them but to have the option of trading them in the future, if they prove themselves over the next year or two. If I haven’t designed the model myself, I like to have several years for history to build my confidence.
By the way, none of the 3 has done very well since I subscribed (two are flat and one has lost money).
So my approach doesn’t guarantee every model will be a winner.
Brian
PS: To get a handle on the bias you mentioned (designer’s deleting their poor models), I plan to keep my Excel sheet so I can refer back to see if a designer has deleted models.