I came across this some time ago. I had tested a ranking with and without the composite nodes and found an improvement without the composite nodes. I saw this with a couple rankings so to verify I selected one of the P123 ranking systems, I can’t recall which right now, and removed the composite nodes and compared the results to the original and found an improvement of a couple percent in a sim based on this ranking.
I have been working on a ranking for some time and it is easier to do with the composites, so it has been on my todo list for a while to test it without the composite nodes. I just did and the sim based on this ranking has: Annual return improved by 9.6%, Max drawdown improved by 3.2%, Sortino ratio improved by 9.1%, and gain/stock/day improved by 8.3%. Based on my earlier comparisons this is much higher than I expected, but the ranking that I updated is complex and had a number of composite nodes. Still, even for the possibility of a couple percent improvement it is worth recalculating the weights to remove the composite nodes and see if it improves performance. I’ve seen this enough times now to believe it is not a fluke. I have saved my original ranking with the nodes in case I want to make updates in the future, but will switch my trading to be based off the ranking without composites.
Are you saying that if you remove composites and apply the EXACT same weights to represent the individual factors/formulas that you get a different result?
I’m more concered with why than I am with seeing an improvement. Can someone shed some light on this please.
That is exactly what I am saying. Take for example the Balanced ranking which I have posted below. Within the first composite the factor Surprise%Q1 has a weight of 15%. So for each stock this factor is ranked and then weighted and combined with the other factors within the EPSRank. The EPSRank is then weighted at 30% and combined with the other composite nodes to form a rank. This is similar but not the same as weighting Surprise%Q1 at 4.5% and combining it directly with all the other factors.
I suggest you test it. I don’t have time right now to set up an example and I can’t share the private ranking that I reported on.
The way I look at it is that without composites, each factor/function is ranked on it’s own merits as part of the overall ranking, whereas a factor/function within a composite node is only ranked on it’s own merit within the composite node. Within the overall ranking factors/functions within a composite are ranked based on the overall score of the composite. The important point is that this is not the same. Now maybe the differences are random and the improvement I have seen with several rankings was a fluke, but I think that perhaps it may be superior to rank each factor/function on it’s own merits rather than to dilute them within a composite ranking.
If you don’t like it you don’t have to use it … but I bet it will improve the performance of your ranking.
Here are a couple examples I ran for fun so take what you will from it. The ranking did not improve and I’m convinced that it’s a fluke yours did. However test for robustness both ways and pick the one that works better in your sims.
There’s definatly a difference in how a rank is determined within the composite vs on it’s own with an equivalent rank.
That is VERY interesting and informative. Since you have already set up these 2 test ranking systems, will you post the results of running the Ranks using “Include Composite and Factor Ranks”?
That way we can access the different rank values between the two systems.
It looks to me like whether or not removing the composite nodes improves the rankings may be based more on the weighting of the composite nodes and whitch factors or functions are included in each node.
I think this merits a closer look. When I have more time I will re-test one of the P123 ranks and post here. In the meantime, here is my rank w/ and w/o the composites.
I tried flattening the composite nodes on a value ranking I’ve been working on. The results are inconclusive I think. I understand that using composite nodes is functionally different from the flattened version but it looks like it didn’t make any difference for this model.
There is a difference there. Like mine the results are similar but the fact that your bucket total you see at the bottom of the graph are different is what is the concerning part. It really has nothing to do with if the system perfoms better or not at this point. We need to answer the “why” before we command the “do”.
Why the results are different is simply because the rankings are tallied in different ways. Maybe I am not explaining it well. Take APPL in the above example. It has a low score for InsOwnerSH%, but all other factors score fairly high. In the ranking without composites it still does fairly well. Although the factor scores only 50.34 the factor has only a 10% weight, and this is compared against the other stocks ranked by the other single factors. In the ranking with composites, APPL does not do as well. The composite containing this factor had a relatively high SIRatio but the test4 composite is weighed down and scores only 72.4 when compared against stocks ranked against the other composites. As a result APPL is ranked 10th instead of 2nd. Different comparisons yield different results.
To me the question is whether the method without composites may in some circumstances have some advantage. The examples above show that it does not in all cases.
I agree with Don. It is obvious from the Rank value comparison above that the actual placement of the factors within the composite nodes varies the effective weighting on a given factor.
By recombining which factors are together in a node there should be cases in which the performance is improved and cases in which the performance is hurt.
Whereas the Rank values for the case without composite nodes weighs all factors the same.
Hi,
Strange that it seems every time I remove composites I see such a big improvement and yet so quickly others found no improvement. Still I think it is worth a look for most. I removed composites from 3 P123 ranking systems and for the 99th bucket I get an average improvement of roughly 20%. That’s a big improvement for such a simple task. For those that may be interested I made the rankings public, they start with NoComp. Comparisons below:
I did some simulations with the flattened version of a ranking model versus the flattened one. This ranking model is based on Magic Formula. I deliberately paired FCFF and EBIT yield with ROIC in a composite node because I wanted those reranked independently first before ranking with the rest of the model.
The difference in returns might be fluke but if anything, using the composite nodes was beneficial.
Ok, last one. I told Oliver that I thought removing the composites would improve his ranking, so I had to check and see. I have a copy of his Generations 3.3 and Generation 4.0 that I combined years ago and call Olikea’s Generations 3.3.4. Now I also have a NoComp version which is higher by about 18% in the top bucket (which is where I do most of my trading) though the slope of the top buckets are not as good. Some randomness is to be expected but I have seen improvement in every single long ranking that I have removed composites from. It did not improve my weekly short ranking.
Nobody is disputing that you got better results without the composites for you examples. All I’m saying and I believe Denny agrees … is that you will get DIFFERENT results based on the rank of the composite vs individual. The composite is a combined rank composed of what lay within it. When a factor/formula is on it’s own then it is ranked against the other factors/formulas or combined composites one on one. IMHO whether the end result is better or worse is mute because it completely depends on how the rank system is set up.
I’ve shown you a system that performs worse - Fact!
You’ve shown a system that performs better - Fact!
End result is the same. You’re going to use one that performs better and is more robust.
Hot date tonight!!
