As requested, below is a chart of my investment gains and losses (real money, not simulated) since I started using P123’s ranking systems around Halloween 2015. The red line is IWC, a microcap index fund. Annualized, this is 47%.
1 Like
Interesting thread…
Actually I don’t want to give away all my secret ingredients, but I learned also new things at P123 and this forum and so I want to give something back.
-
Of course, it’s a pity that so less ranks are made public nowadays, but analysing the old ranks is nevertheless more helpful, because you have some years OOS performance. So you can find even more easily robust ranks and learn how they are created. Never underestimate the value of knowing years of OOS performance!
-
To create more robust systems and use your time productively you should keep in mind what improvements are really helpful: 1) It is far better to find new, un-correlated and sound ranks than to use effort to optimize weights. 2) It is better to find new ways or perspectives to measure value, growth and momentum than to find the right thresholds in the buy/sell-rules.
-
If you have different ways to measure your core-strategy (value, growth, smallcap etc.), than use them all in the ranks, even when you assume a strong correlation. Overweight the nodes which are more sensible and should have less noise. If you don’t have an understanding which are more sensible or have less noise, then weight them equally. To measure i.e. value by all valuation metrics you know is more rewarding than to find the “best” valuation metric. By doing that you solve following problems: 1) You reduce noise by some kind of “averaging”. 2) You don’t have to find out which factor is the “best” one. 3) Even if you have a wrong factor (by miss-understandings or wrong calculation) or a wrong data point, it doesn’t damage your system completely. 4) You make your strategy more plausible by looking on it on various perpectives. A real value company shouldn’t have only one good valuation metric, but several. 5) If you don’t optimize your weights, you will create a robust system (even when statistics says, only 1 or 2 factors should lead to non-optimized system), because you just put most of your knowledge about stock analysis into the system.
-
Also bring well-known factors in question. One example, which alarms me how often it is used by model designers, is “EPS%ChgPYQ”. If you think about it, you find some problems. According to the help this value is calculated as the most recent quarterly EPS minus the EPS for the same quarter 1 year ago divided by the absolute value of the EPS for the same quarter one year ago, multiplied by 100. I assume the absolute value in the denominator is used in order to handle negative EPS. Also other investors, not at P123, using similar calculations. But does that really lead to sensible results? From my point of view it looks more like a workaround. It may solve the problem with negative EPS, but not with small numbers. Assume two companies which earned an EPS = zero (or almost zero) last quarter, but had an EPS of 1 dollar vs. 5 dollar one year ago. Both of them have a an EPS%ChgPYQ of -100%, but in my opinion the second company is worse than the first one. Similar problem: A company earned an EPS of 0.01 dollar last year and earn now 0.02 dollar. Is it really a growth 100%? I would say if the P/E is around 10 it may be a real growth, but if the P/E is around 100, than the change of the EPS is just noise, because it seems to be a big company just earning not enough money. If you find mis-calculations like that in the investing world, it is fertile to find better measurements. You can think about this formula in a more mathematic way and gets to the conclusion that the difference of current EPS vs. EPS last year is “normalized” by the EPS last year in this formala and then try to find a better normalization. You may use the marketcap (like the P/E) to compare the EPS grow with the size of the company. So you can create your own formula like: (EarningsPQ - EarningsPYQ)/mktcap. This formula doesn’t have any problems with negative or small earnings - so it leads to better a rank. Improved factors like that, which quite “repair” common factors give you a good advantage about other investors using the common calculations. You may argue, that I combine here a growth factor with valuation - which may right (like it is also done by GARP strategies). But if you see any sense in it, you may combine it with EPS%ChgPYQ and any valuation in a rank anyway. Especially, the combination in a rank with the classic EPS%ChgPYQ is a good way to reduce the noise in the denominators (the noise in mktcap is quite a different noise than in the earnings last year). So… (EarningsPQ - EarningsPYQ)/mktcap is one of my ingredients of the secret sauce, but more important is the way how to find such factors.
-
Think more about noise in the markets. Keep always in mind, that the factors have different levels and forms of noise. Most noisy is price (and therefore also the market cap), followed by estimations (price targets more noisy than recommondations, which are more noise themself as the sales and EPS estimations). Factors from the sheets are general less noisy, but also here is some kind of rank. EPS may be more noisy (because of manipulations) than Sales and Cashflow. Keep such a list always in mind when creating/improving a system. Keep also in mind that some calculated factors lead to even more noise (as you see in the previous point). You can reduce noise only by averaging in respect to time (i.e. sma(20,190) instead of close(200), amounts of data (like number of analysts) or different ways to measure the strategy (like different valuation metrics in one ranking system).
-
Find the small disadvantages which the P123 systems have, and some solutions/workarounds for it. For example, if you use mainly the rankPos variable in your sell-rules, than you should use it also in buy rules. Even if it is the only buy rule. For example, if you have a port with 20 stocks, you may sell at ranks above a rankPos of 50. But you could consider also to buy only stocks with a rankPos of smaller than 15 (so you don’t buy stocks with a rankPos of 15 - 20). Sometimes it’s better to wait a little bit for an other great stock instead of buy just another one (this is only true, because of the construction of the portfolios. If P123 changed the buy/sell-rules in a way, that we can sell stocks if there are new, very high-ranked ones available, you may change the rules again. But the current systems just sell according to the sell-rules and after that, the system looks for the buy-rules to find new stocks).
-
One of the strongest single factor (with a reasonable turnover) I know in the SP500 universe is the Research-and-Development costs / marketcap. On the one hand it’s a reasonable factor for some sectors. On the other hand it doesn’t work in all universes - which is the reason why I don’t use it so far. I prefer factors which work in all universes (of course, I know, many inverstors are successfull with specialised SP500 or microcap systems), because it proves the robustness of a factor.
I hope this encourages more members to share some of their secrets 
1 Like
Sebastian -
This is an absolutely wonderful post. I agree with everything you write, and it really provides a great perspective. It is exactly this kind of response that I was hoping for when I started this thread. Thank you!
- Yuval
Sebastian, thanks for sharing
As an exercise I’ve implemented Yuval’s system as described (filling in the details where necessary) and it results in enviable excess returns over 10 years in simulations even with minimal tuning of weights and Universe rules (50 stocks in portfolio, 3000+ stocks universe, buy/sell only based on ranks and minimal liquidity). Mind that this is happening 4 years after it has been shared (possibly 5-6 years since inception) and that my implementation is certain to not fully capture Yuval’s. I think this is a great statement as to its rigor and validity.
Huge thanks to Yuval for his work and for sharing it with the investing public.
1 Like
Thanks Yuval, that’s terrific. I haven’t invested as much effort in ranking systems, perhaps it’s time I start.
Hi all,
since trend is the investor’s best friend, this buy rule filter also helps:
RSI(48)>55
Thanks for sharing some secret sauce! As I’m trying to get more familiar witht the 123 syntax, can someone show me how to calculate the deviation of sales?
Thanks for a great post!
It’s the first time I’ve seen someone weighting sectors like that. A very interesting idea. How exactly do you do this? Do you do it as a buy rule, or do you give the sector a weight in the ranking system? Have you tested one and one node against each sector, or do you test the entire factor theme against one sector (quality, value, growth, etc.)?
One way is to test your ranking system on various sectors or subsectors by changing universe rules. If it works really well (high values on the right, low values on the left), give the sector or subsector a high rank; if it doesn’t work well (middling values throughout), give it a low rank.
So something as simple as this?:
Before:
After, with a 10% weight in the overall ranking system and each sector weighted after its historical performance:
The overall performance goes down…
If it’s not your IP,may I ask, how is your weight in sectors and subsectors?
Or, if not to weigh each sector according to their historical performance with my rankingsystem, any other ideas I can try? Or just add or subtract 2-3% from each sector until it improves the performance in the overall simulation?
Just a general extension of this. It NEVER works for me to weight by the historical returns no matter what I am doing. And this would be just another example that proves this rule to me.
AND this is in-sample. I think it gets much worse out-of-sample. Done out-of-sample it would be a clear example of fitting to noise and overfitting as there is guaranteed to be some noise in the returns. Pure and simple. Overfitting to some extent is guaranteed with raw returns and there are know solutions.
There is information in the returns that can be use. It is just not necessarily a good idea to overfit that information.
An example of a solution is Ridge Regression that uses the information but shrinks it (Lasso regression might discard some of the data). My experience and it is growing every time I look at this is shrinkage does work and information like this can be used while the raw data does worse than nothing.
There are a lot of good shrinkage methods with ridge regression being just one of them. Empirical Bayes is another.
Empirical Bayes sounds complex. But here is a P123 resource that addresses some (but not all) of the reason it is used (e.g., regression-toward-the-mean) with the first link being to a paper that uses Empirical Bayes https://blog.portfolio123.com/thoughts-on-is-there-a-replication-crisis-in-finance/
And here is a link to the original paper if you just want to go there: Is There a Replication Crisis in Finance?
Here is a quick a dirty way to shrink things that seems to work with not a lot of justification and certainly nothing like a mathematical proof. But shrinking things to the lower confidence bound (i.e., the the lower side of your usual 95%confidence intervals) gets you pretty close. And JASP and other programs can calculate that number easily enough (menu driven and free). And if you squint, think about variance etc you can see how it sort of looks like a Bayesian approach as the confidence interval will widen (with more shrinkage) when there is less certainty (more variance or less data).
So, IMHO, Yuval has 2 good points. First, I suspect the sector return information can be used!!! And I give him credit for the “The Factor Zoo….” post which actually has a lot of good points. I think almost all of those points are related, but if anyone wants to say things like regression-toward-the-mean is something different, I would not mind how one decides to outline the general topic of raw returns out-of-sample. Or what particular solutions they may use if they too find some problems they want to address.
BTW, I do not mean to imply that once you shrink things you will get a dramatic improvement. In my experience you will not. But you are actually guaranteed to be less overfit out-of-sample and you will get some improvement in my experience.
Jim
I think your ranking system is giving all stocks the same value no matter what sector they’re in. Try it and see.
This is what I use:
eval(rbics(xxxx),9,eval(rbics(xxxx),8,eval(rbics(xxxx,xxxx),6,eval(rbics(xxxx,xxxx,xxxx,xxxx),5,eval(rbics(xxxx,xxxx,xxxx,xxxx,xxxx),4,eval(rbics(xxxx,xx,xxxx,xxxx,xx,xxxx),3,eval(rbics(xxxx),2,eval(rbics(xxxx),1,7))))))))
Why is this so complicated? We should have a SWITCH() function, similar to Excel. Then we can code it simpler. Like this:
SWITCH(rbics, nnnn, 9, nnn, 8, nnnn, 6, nnn, 5, nnnn, 4)
That’s a lot less work and less error prone.
I hardly ever optimize ranking weights. Maybe that’s just me. But I wanted to throw that out here just in case new users get the impression that it’s necessary.
Below is my longest running live strategy. Factors are organized into composite groups and everything on a particular level is given the same weight. It went live 12/03/2018 and has an Annualized Return of about 40%.
I think my time was better spent building new rankers.
Yes, this is one way to do it.
A less error-prone method is Eval(RBICS(5678),9,0) + Eval(RBICS(2674),8,0) + Eval(RBICS(7828),7,0) + . . .
Ok, I will do that 
One question, the number behind the sector code, “(5678),9,0)” what is that? the weight? So here, give it a weight of 9%.