Why are there so few subscribers to designer models is the question?

General
1

This is a long post so I appologize if it’s not appropriate but I would love feedback and maybe there are some quick wins.

I don’t have any answers just more questions and some food for thought. I will frame my analysis around customer life cycle management and the subscribers journey. The following article provides analysis for each stage and I have provided my own thoughts. There are many different frameworks to evaluate the customer journey so feel free to add what ever you want. This may not be the best way to analyze it but it highlights some issues. I hope you do the same type of analysis. This could be a complete waste of time and the only thing that matters is OS performance because in the end that’s all the matters but I think there is more to it. If you think there is more to it then please read and comment.

Reach

The potential subscriber is doing their research and comparing P123 against collective2 or any other subscriber based subscription and after they compare all the features of both via the internet and social media they decide P123 is for them. These are digital marketing processes which I am not an expert at and I am a long time user so I have major bias.

Questions during Reach:

  • Is a designer model easy to find and how do we know?
  • Is it the designers responsibility to do this or a joint effort?

Acquisition

The potential subscriber is now on the P123 site trying to get more educated and evaluating the designer models. This is not easy you need very good knowledge of trading and what KPI make a good trading system and after all of that there is a lot of faith in the designer. If the prospect has questions about P123 they will get answered in a reasonable time frame but if they have questions about a designer model it may take some time. There are some designers who don’t respond and this many or may not reflect badly on other designers. I’m sure there is a lot of opportunity during this stage but I am not an expert.

Questions during Acquisition:

  • What is the profile of a subscriber?
  • Is P123 setup to target these prospects?
  • Is it the designers responsibility to target all potential subscribers?

Conversion

This is where one click signup would help. It does not stop there each designer needs to ensure the subscriber is getting good value and has entered a relationship with the customer. This is not at all how the financial industry works. Even a financial advisor cannot guarantee success so what is the value proposition for the subscriber and what is the role of the designer? It cannot be use at your own risk but I feel it is today.

Questions during Conversion:

  • What is the onboarding process from start to finish?
  • What level of knowledge do I need to be successful. If you have no experience this is not for you.
  • As a subscriber becomes a customer do they feel supported?
  • Does the customer understand the value proposition of the designer model?
  • What skills does a subscriber need to evaluate a model? This is very difficult you need to know how to evaluate a trading system with your trading style and profile.

Retention

Retention starts with finding out how the customer feels. This is a loaded question with a designer model. If the customer is making money they feel great but if they are losing money they hate you. Making it even more complicated some designer models lose money over many years. A good example of this was 2018-2020 most value based models underperformed while growth was beating the market. Retention is usually done with surveys and voice of customer. I don’t think any designer on P123 is doing this.

Questions around Retention:

  • If a designer cares about customer retention how do they proactively retain there customers?
  • Not every model performs well all the time so having a narrative that supports portfolio construction would help. This would require giving financial advice and designers cannot do that. So what if anything can be done?

Loyalty

Subsribers are an important asset to the Designers brand. Once this is destroyed it is very hard for it to comeback. A perfect example is the Designer graveyard. This is a very touchy subject because no one wants to call out designers who have curve fitted models that bomb OS. That being said I don’t see a lot of advocacy for the designer models that have beat the market over the last 5 years and have a good sharp ratio. I am guilty also since I have not recommended designer models to my friends. I tell them which ones I use but I never say use this model since I am not there financial advisor.

Questions on Loyalty:

  • How is brand loyalty for a designer model nurtured and cultivated just like your grandfather who only drove a Buick?
  • As a subscriber where can I post that a Designer is amazing and has met all my expectations?
  • Why are designers not advertising how they have beat the market for over 5 years with a sharp ratio greater than 1? Would this not create loyalty?

Other questions:

  • If you were a financial advisor what model would you recommend for a client that you did not build yourself? Or if your not what model would you recommend for a friend?

  • Why do so few designers subscribe to other designer models? This is a big red flag. Does it come down to trust or does every designer think they can do it better?

  • How would you go about selecting a model, exiting a model and what are the considerations before you put up real money.

  • Your knowledge of trading needs to be very good in my opinion.

  • Your trading style? This is rarely mentioned in designer models but trading a microcap with 30 trades per month is very different than trading a large cap model with 5 trades per month.

  • What are the exit plans for a model when it stops working? A good example would be growth vs value last year. Or do you want to create a book of models and ride it out?

Out of the box ideas:

One of my ideas to create more subscribers is via live trading seminars. I participate in a few and find them very helpful. Just seeing how a pro sets up IB and executes trades is worth watching. Especially for option traders. A Video tutorial showing the basics like setting up and IB account to the more advanced topics like creating a book of different designer models and why each was selected. The topics are endless but so valuable.

Live trading seminars every Monday showing a simulated trading account on IB with a book of designer models. The meeting could review each model and if it is performing as expected. Last year would have been very hard depending on the models choosen.

Seminars on how to choose a designer model with an open forum would really stimulate conversation about the entire issue. This would be very interesting to see which models were selected and 5 years from now how they performed. Every year the subscribers could vote on which ones to add or drop.

Obviously someone has to take the time to do this and that is hard problem to solve. Just putting yourself out there on live video would take a lot of courage.

Conclusion

In conclusion I have not solved the low subscriber issue but I hope I have given some food for thought. The solution will involve P123, designers and subscribers being innovative, logical and honest with each other. Hopefully we can beat the market or why else are we doing this?

Thank you P123 all the designers and subscribers.

Cheers,
Mark V.

2

Great post Mark.

There may be few DM subscribers, but are there too few? I believe the typical website conversion rate is about 3%. I don’t know how many visitors p123 gets but I’m guessing it’s not very many. When I attended local investments meetups, I never met anyone that knew about p123. And these were well attended meetups. The sales funnel problem may start at the very top.

I’ve always considered p123 members as the target audience for my DMs. I can’t imagine how anyone else would find them.

As for retention, p123 does an exit survey when unsubscribing. They’re not shared with the designers, so I don’t know why people left my models. Maybe there’s some good insight there. Dunno.

1 Like
3

I also think this was good post.

I would it say the following is an important factor:
When I first came to know P123, I really thought that getting into the language (or ‘code’) of P123 would be a big hassle. I also thought that understanding all the exact accounting terms was going to be tough. Then you also have all the options to backtest that seemed so extensive, I thought I would never be able to completely wrap my head around it.

Of course, looking back, I’m just really glad that I just started exploring and went for the ride.

The thing with designer models, in my opinion, is that if I want tot trust the model, I kind of want to have some basic knowledge about P123. Like Walter mentioned, I think most subscribers will be active on P123.

To put a long story short: I think the conversion doesn’t have all that much to do with the number of ‘clicks’ it takes on the website to subscribe (I think some users have suggested this). It is much more a trust thing. This can mean that designers must (be able) to gain trust on external websites, so people trust the designer enough to use the designer model on P123 ánd be able to easily buy a subscription on that external website. Either that, or P123 would have to make the process of understanding how P123 works and how designer models are constructed way easier to digest for someone new.

Best,

Victor

4

A factor overlooked so far is Designer participation in the discussions here. I have subscribed to a few models but only from people that i “know”, which means I have a feeling of what they think, how they operate and what their participation here is. Of course, a model can speak for itself but if the designer chimes in and participates here in the discussions, maybe with some constructive proposals, strategy design etc. I am much more willing to subscribe to a model. I would not subscribe to a model that looks very good but the designer is mute here.

1 Like
5

All,

I do not have a designer model and maybe I do not understand the sign-up process. So take this for what it is worth.

But I have highly paid friend who is in medicine. She has expressed some interest in what I do. She took calculus as a pre-med and is extremely intelligent. But any programming or the math that is used is just not all that interesting to her. Kind of but not really. She has papers to write and other things to keep her mind busy (e.g. saving people’s lives giving talks).

I think, as a simple one step process like Andreas describes, I could create a model for her (and others) that I could recommend to her and she would sign up for it. It would probably be highly liquid (maybe just SP 500 universe) so that her wealth as she pays down her medical school loans would not interfere with my small and micro-cap purchases (I do not want her buying the same stocks as I do in the morning). She is not so good of a friend that I want her increasing my slippage. There would be enough liquidity in my models that I would probably open them to everyone.

This could be highly appealing to her considering that P123 will be making it possible to have automatic trades. She is busy. Taking call at nights and often in the operating room at 7 or 7:30 in the morning. Sometimes after taking call. Maybe she wants to make some trades in the call room. Probably not something she really wants to most days, however.

I would like her to sign up with P123, run some screens, talk about what factors to use. Develop some models together. She is certainly capable of learning that.

But it is not going to happen. Not this year anyway.

If I understand the sign-up process at all and Andreas’ (JudgeTrade) post, you should do this:

Make it so that people like my friend can go to P123, easily find a model that someone has recommended to them, give P123 a credit card number and with “one click” as Andreas suggests you are good to go. Maybe facilitate all the stuff to complete the automatic trading with as many brokers as possible. I will not be recommending that she pay IB any commission for SP 500 stock trades.

Maybe the present process is a little like the above but I think it isn’t. In any case, I have not built a model for her, and today at least she will be going to the OR or the clinic without looking at P123’s web site despite my discussing the site with her and her (limited for now) interest.

I think she will not go through the sign-up process. I think I will not be building a model for her.

TL:DR: With my present understanding of the process, I think Andreas has an excellent point. Crucial really for full participation.

Also, this is just a dream but when I retire I might get a CFA (we will see). I would love to send clients to the site. Maybe I would make it so I have no financial interest or conflict of interest (any price they pay being P123 costs). I do not think a bunch of retirees are going to start coding and backtesting.

Jim

6

I hope this thread isn’t dead yet. I have some DMs in incubation that I’m really excited about and this would be a great time to work on promotion.

How about creating a landing page that summarizes all the designer’s current offerings? Each designer would have their own page. Potential subscriber could then be given a page link for review and designers could work on their own sales funnel. It could look something like the current Designer Dashboard. Here’s a mockup - please excuse the crude appearance. Of course, the display fields are negotiable. This is just an example.

final
final1920×1131 175 KB

7

Most designer models are about small and micro caps. In my whole trader network (about 50 traders, investors outside p123), I do not now many who trade them (exactly 1!).

I pitched small cap value momentum (let’s say with three different tilts: quality, value and eps growth) and I had interesting conversations about it. Small and micro caps are the best for small accounts. Everybody agreed.

But people do not want to use the “small account advantage”. The want to do what the big guys with big accounts are doing.

Day trading SP500 futures, Breakout strategies (CANSLIM and other stuff), macro-based ETF Trading, etc. The exitement is a big part of the game!

I still remember around 2006, 2007 when small and microcaps and value strategies where very prominent (after a huge run from 2000-2007, up, up, up!), just before the deflationary bust in 2008!

Yes, we can improve, but at the end of the day, it is an uphill battle.

And let’s face it, it is very good news for our niche!

Best Regards
Andreas

8

I think people forgot the real story behind Designer Models, which began in 2013 with various members creating interesting strategies. However, the competition for the best simulation became so fierce a couple months later, because good money could be earned, that some started producing bogus R2G strategies with insane returns with various trickeries that cheated the P123 system in clever ways. Back then, there was a real demand for designer models, I remember having at some point almost 30 subscribers at 100$/month each on a single model. Then the bubble burst after the scandal erupted that many of those strategies were more than just overfitted but completely fraudulent… Since then it’s been downhill for R2G strategies and it never really recovered in popularity. So I would say the scars of the past are still there somehow sadly. P123 made changes to avoid this, such as hiding simulated returns, but it was too late, the reputation got eroded.

9

I don’t recall any fraud. How? Who?

10

I don’t remember the exact details but I think there was a way to forward look the data or it was all coded in some long string of bogus code that had similar outcome of giving a signal at certain dates chosen by the designer (i.e. wanting to purposely sell stocks at date A and purposely buy stocks at date B, knowing the past outcome at these dates), making it the perfect doctored market timing method. For example, if on Date A, you create some technical “DNA” for that date by noting the exact RSI value, the exact Stoch value, the exact price value of SPY, you can pinpoint the date in a custom formula somehow that match these criterias, and there would only be one date matching it. Then you stacked such a doctored market timing string formula with a decent ranking system and you would get 90-150%APR model. Novice investors who came to P123 thought they found a gold mine and all subscribed to these. I don’t want to name any name but some of the oldest members will probably know what I am talking about.

2 Likes
11

I’ve been following this thread with interest and decided to go look at the designer model offerings which currently number 165. Excepting less than a double handful, they are basically unappealing. I’m relatively new (mid-July 2022) and moved into models like being offered in mid-September which means I only have 3 month real money returns on a 35-38 position strategy. But only 22 models beat my 3 month return. Only 12 of those have 1 year returns that exceeded the 3 month returns. Three models were full. Several had designers that had not logged on for years. Others had high to very high risk.

Overall, not a promising place to look for models that exceed what I am currently doing with a moderate risk level.

Regards,
Rich

12

All,

So maybe Rich is different than I am. But I have a Port that I am very happy with also. We may be the same in that regard.

BUT I am well aware of what benchmark my port is correlated with and how much alpha it provides for that benchmark. I choose to diversify.

Some (not necessarily Rich or even me) might want to diversify their portfolios with some designer models.

To help the with that P123 could give us this information:

  1. which of the benchmarks the designer model is most correlated with.

  2. the alpha on the benchmark the model is most correlated with.

This would be helpful because I would select models that provide diversification and VERY SIMPLY I would want the model to provide the greatest alpha for the type of model I am wanting to diversify into.

A very simple algorithm for choosing a model.

Not a complex decision. P123 should not want the site to be complex.

Now you can look at alpha for a designer model but you do not know if the benchmark that was chosen has any importance to anything. Alpha on an uncorrelated benchmark can be pretty random and dependent on……uh well, random market conditions.

You could even have a correlation matrix of the backtest to several benchmarks. With alpha to all of the benchmarks in the correlation matrix, perhaps, but definitely alpha to the most correlated benchmark.

This is information you would get from any serious advisor when building a portfolio.

Also, Kurtis Hemmerling and Andreas are right I think. P123 should ultimately have an independent site with easy sign-up. To keep people at that site, serious advice about diversification could prevent unpleasant volatility and drawdowns that will cause people to leave.

To be a serious site it should start providing some information that would help in building a serious portfolio. Correlations to benchmarks and the alpha on the most correlated benchmark would be a start. Just the beginning, perhaps. But that could easily be added to the designer models if anyone else think it would be useful information.

E.g., I like my port but it is most correlated to the SP 1500 Pure Value Benchmark. I provides good alpha but it can underperform the SP500 for significant (and unpleasantly long) periods of time. This is despite having multiple growth factors (and other factors) in the ranking system.

It is what it is, and I can be objective about what I can expect from my port. Not all ports will be all-weather even if someone might judge that to be desirable for some reason.

Jim

13

A lot of great ideas. So something like;

  1. separate site that can act as a feeder to the main site - or not.
  2. access to Designer Models
  3. access to P123 Models
  4. access to ETFs
  5. access to books - simulated and live
  6. access to a broker

And I would still like a direct link to my models. Perhaps an embeddable link so that I could display my models on my site.

1 Like
14

I’ve been a member since 2010 and a daily user since 2013. The only case I recall involved an unsubstantiated claim of cheating. P123 examined the model and found no improprieties. That’s my recollection.

On the other hand, over-fitting was rampant. The increased incubation time addressed that issue.

P123 keeps getting better with time.

1 Like
15

There was the knowledge and demonstration that sims could be played by excluding certain stocks in the exclusion list (and perhaps elsewhere)—especially with the ability to do so over certain timeframes (e.g., when in retrospect a stock tanked). My memory is hazy on how P123 addressed this hack but it may have been fully addressed (not 100% sure).

No one is perfect and there will always be smart hackers, but Marco is pretty computer savvy. My computer can be hacked but I am not throwing it away.

16

Yeah the best way to address all of this was the longer incubation period and hide sim stats. In 2014-2015 people were judging a model only by its sim as the selling point which is why these trickeries happened.

17

“On the other hand, over-fitting was rampant.”

There is nothing wrong with over-optimization as a strategy. (I do it all the time.) But one has to understand why he/she is overoptimizing and it becomes problematic when presented to potential subscribers as performance.

18

Over-fitting (aka fitting to noise) is a path to failure. It’s probably OK to optimizes a model that is difficult to over-fit through the use of few rules or ranking factors. Here “few” is very subjective but, in my experience, it’s in the few score smart ranking factors, no buy rules and maybe four-six sell rules.

Anyone would go-live with the best simulation results. What’s the alternative?

19

Steve and Walter,
I think the concept of “robusteness” comes into play here.
There are many ways to “de-optimize” a strategy. Varying the parameters and see how they affect results, using even/odd numbers for stock selection, fewer rules etc. etc.
Optimizations is practically unavoidable but in the right context and done with caveats in mind, it is ok.

20

All,

Over-fitting will occur any time you run more than one optimization. p < 0.01 means that there is a 1/100 chance that your result would occur randomly. If you run 100 backtests using RANDOM() as your ranking system the result would occur 1/100 times is the simple meaning of that.

How many backtests do you think you have run? Speaking for my self, I have easily run over 1,000. Using random() as my ranking system, I will have a pretty good backtest after 1,000 tries (I have tried this and 1,000 tries will give you an amazing sim). And if I had a random seed I could probably sell it as a designer model. Of course it would not do well out-of-sample!!!

AND it is not just over-fitting that is the problem. There is regression-to-the-mean. And the fact that if a factor is truly useful others will begin to use it: e.g., Zacks is using some of the factors we have backtested at P123. Others will discover it and start using it.

So, whatever you are doing you will do this as Walter correctly points out: Anyone would go-live with the best simulation results. What’s the alternative?

People will have backtested multiple times and go with the best result. If they were using random() as their ranking system they would still go with it but it would not do well out of sample.

The designer model backtest will not be realistic no matter how you try to explain it. (e.g., explanations above) So, I will stop trying to explaining it now, IT SHOULD BE SELF-EVIDENT AND NOT REQUIRING ANY FURTHER EXPLANATION!!!

We should SHRINK our expectations. Shrinkage has a meaning in statistics. As an individual one should somehow shrink the expectations they have gotten from a backtest. P123 is not going to do it for you!!! It would be difficult to arrive at a consensus of how to do that. This is understandable. Empirical Bayes might be the best method but not trivial to perform. Bootstrapping may be as good and it is trivial now.

This thread made me want to arrive at a realistic expectation of my own model. No one else has to do this. Here are some stats on my model.

Stats
Stats908×394 41.1 KB

Walter wants the distribution of the results. I want to use bootstrapping of those results (other recent posts).

Here are the bootstrapped results of this sim’s weekly excess returns downloaded from P123. The bootstrapping is done easily in JASP (confidence levels or Cl here):

Bootstrap
Bootstrap686×598 23.2 KB

The lower-bound of the confidence interval (a pretty wide confidence interval of 99.9%) for the bootstrap’s weekly excess return is 0.346. Unfortunately this is compared to the benchmark and not the universe but a start nonetheless.

Edit: regarding the 99.9% confidence interval. Above I said I have probably run at least 1,000 backtests (exactly 1,000 would put this as about the right confidence interval). I have probably run more than that (so it should probably be wider still). I should probably use a wider confidence interval and I could start keeping track of how many backtests I have run. Sadly, I am not that OCD but I really should do that.

To annualize that is 1.00346^52 = 1.1967

Or the best I can hope for is a 20% excess return on the benchmark for my port going forward out-of-sample. Maybe less. Maybe I needed a wider confidence interval. Maybe a whale just started using the same factors I am using. Maybe there will be crazy market conditions making the backtest totally irrelevant for the next several years.

TL;DR: You can never use the raw backtest for prediction. On your own you can get more realistic expectations. At least do it before you figure out how big of a mortgage to take out on your house to invest with (or pay for an expensive designer model).

BTW: How many backtests do you think went into creating the designer models? I think about 162 designers now. But you also have to count the discontinued designer models. Make your own estimate. But you should consider that there might have been 500,000 or 1/2 million backtest total I think. Or a confidence interval of 99.9995% for the designer models. Less for your own models. Use that for your bootstrap of any designer models if you can get the information you need in a P123 download (not 100% sure on how easy that would be).

Oh and most people start with ranking systems that have already been tried somehow (e.g., searched in P123’s public ranking systems). I am not sure how to incorporate that. But don’t sell the farm based on your projected results based on a backtest: that I am sure of. Very sure.

And BTW, I argued with Marc Gersein at the time about the inclusion of backtests in the designer models. He came at this in a different way. But I am going to just say: Sorry Marc, I was wrong. That is not to say I really have a position today. More information is better, would be another way to look at it. But I clearly did not fully understand the issues at the time. Marc had a better grasp of the issues then.

Edit: the above is pretty good but it can probably be improved. I will avoid going in to specifics but an example of an improvement might be to use log returns as that will be more accurate for getting annualized returns. I will avoid the details. The error is small by not doing that but the log returns would clearly be better. The log of the raw returns rather than of the log of excess returns would be much better if you decide to use log returns. I will be thinking about what confidence interval to use. The above clearly puts you in the ballpark but the numbers for a hard cutoff are not clear to me yet. There are texts on this and I will probably review them.

Bootstrapping is well established for this and is included in SPSS too. Literature on it can be found in a lot of places. It does have the advantage of getting around all of the issues of normality. Bayesian statistics can also get around fat-tails by using the t-distribution but the issues of skew remain with the t-distribution.

I will be using this but not today to edit this particular post and will not start a new post unless there is a dramatic improvement (or error in this post).

I am not asking P123 to do this. They won’t being the main reason. But it is probably appropriate that they do not at the end of the day. You can do it without P123 and the debate in the forum would be intense. There is no AI person at P123 who posts on the forum to moderate such a debate or implement any consensus opinion were any consensus to ever actually occur.

Jim