a newbie's exploration of the core combination ranking system - part 3

Manage
1

All,

This is a continuation of my efforts to understand P123, the “Core : Combination” ranking system and how it performs and why. It also documents my admittedly amateurish efforts at improving the performance. Part 1 documents my test of the “Core : Combination” ranking system using the S&P 1500 as my stock universe, initially using a moderate (25 stock) portfolio and then a concentrated (5 stock) portfolio. Part 2 documents my analysis of using the 6 components of the combination (growth, low volatility, etc.) individually, to see how each performed. There were definite surprises, at least for me. Then I decided to see what the contributions in performance of each of the subfactors of each core components were. This post, part 3, documents that effort. In order to keep this from being a lengthy post, I will limit my documentation of each subfactor to CAGR, Max Drawdown, and Sharpe ratio. This is for a 5-stock portfolio on the S&P 1500 with a sell rule of “Rank < 95 // Sell low-ranking stocks”.

Growth
EPS … 4.06%, -63.59%, 0.23
Sales … 7.61%, -74.37%, 0.36
Operating Income … 1.66%, -77.80%, 0.14

Then I decided to adjust the weights of each factor for Growth, first by CAGR and then by Sharpe ratio, to see how that impacted performance of Growth. There were modest improvements using both methods of weighing. Here are the results:

Original … 9.73%, -76.12%, 0.43
CAGR-weighed … 10.26%, -73.77%, 0.45
Sharpe ratio-weighed … 10.06%, -70.30%, 0.44

Low volatility
Beta … 9.03%, -41.89%, 0.53
Price volatility … 9.63%, -41.31%, 0.70

Original … 8.91%, -31.14%, 0.67
CAGR-weighed … 8.91%, -31.14%, 0.67
Sharpe ratio-weighed …, 7.89%, -36.33%, 0.61

Momentum
Price changes … -4.32%, -87.84%, 0.03
Technical indicators … 13.14%, -50.71%, 0.59
Quarterly returns … -2.34%, -86.45%, 0.04
Industry momentum … 13.93%, -64.08%, 0.55

Original … 3.26%, -70.89%, 0.21
CAGR-weighed … 3.80%, -71.46%, 0.24
Sharpe ratio-weighed … 10.18%, -69.82%, 0.29

Quality
Margins … 5.89%, -41.76%, 0.36
Turnover … 7.14%, -59.68%, 0.40
Return on capital … 8.58%, -56.39%, 0.44
Finances … 5.91%, -62.70%, 0.32

Original … 9.96%, -62.83%, 0.49
CAGR-weighed … 12.25%, -56.03%, 0.59
Sharpe ratio-weighed … 9.91%, -61.41%, 0.48

Sentiment
Estimate revision … 15.69%, -55.36%, 0.69
Surprise … 8.30%, -67.40%, 0.39
Recommendations … 3.89%, -62.26%, 0.23
Short interest … 16.09%, -32.48%, 0.85

Original … 20.83%, -47.62%, 0.88
CAGR-weighed … 22.35%, -38.24%, 0.98
Sharpe ratio-weighed … 21.92%, -42.37%, 0.98

Value
Earnings … 11.13%, -75.36%, 0.45
Sales (*) … 15.87%, -85.98%, 0.28
Free cash flow … 5.17%, -90.55%, 0.27
Assets … -6.69%, -94.07%, -0.01

(*) I made the following note in my file regarding sales: “The above results are very deceptive regarding the CAGR - there was a HUGE gain near the end of the testing period that accounted for almost all of the total gain.”

Original … 18.18%, -77.37%, 0.61
CAGR-weighed … 15.97%, -77.82%, 0.56
Sharpe-ratio weighed … 10.79%, -80.17%, 0.42

I then used the same method described above to adjust the weights of the 6 nodes in Core : Combination. Alas, I did not save the results, so I will have to post that tomorrow.

Everyone should get the idea as to what I am doing. What kind of comments and guidance can you provide?

Cary

2

In my opinion, your first goal should be to find out what test parameters and what metrics were predictive out of sample in order to maximize the predictive ability of your backtests, and minimize the noise (a.k.a. randomness).

Hints: Maybe thirty or more stocks instead of 5, focusing on metrics that are predictive oos, ignoring results that were caused by a huge gain in a single period, etc.

3

I would also advocate using the performance tab of the ranking system to test individual factors rather than using the performance of 5 stocks. 5 stocks–even 30 stocks–are going to be rather unrepresentative of how well an individual factor works. If you combine a bunch of factors with generally upward slopes you’ll get better out-of-sample results than if you try to weight factors according to the performance of a few stocks on each factor. Remember that a factor whose 15th or 20th percentile outperforms its 1st percentile–as very many factors do–can still be very valuable when combined with other uncorrelated factors.

4

Chippers6: how would you go about assessing the predictability of a node or a factor?

5

You can split the backtest period into two. Run tests on the first period for various factors. Record the results in a spreadsheet. Run tests on the second period. Compare the various metrics for the first period with the out of sample performance in the second period.

Yuval has done that and publicized his findings.

6

Chaim,

Well said! I cannot think of anything to add. Thank you.

Well I guess I could add that I hope P123’s AI specialist (developing XGBoost and other ML algorithms for P123) is reading your post.

Jim

7

All,

Sorry to be slow to respond. I have been doing farm chores today and only now (15:20 CDT) am I reading this thread.

Thanks to all for your comments. They are quite insightful. They correctly point out that I did not perform out-of-sample performance testing, that my sample size of 5 stocks was way too small, etc. In light of this, I won’t bother with a part 4, i.e., showing the results of my improvement efforts. Instead, I will read the material that you provided earlier in the different posts, and then ask questions regarding testing, etc. in other threads.

NEWBIE ALERT: For those of you who are new at Portfolio 123 like me and also new at backtesting, etc., note well the comments by others to my posts. Also, please read the book Evidence-Based Technical Analysis by David Aronson (or some other book that others on this forum can recommend) on how to do backtesting and take to heart the requirements there. Treat my threads on the “Core : Combination” ranking system as entertainment and how NOT to backtest.

Thanks again to everyone.

Cary