a newbie's exploration of the core combination ranking system - part 1

Manage
1

All,

This is my first post documenting my exploration of the Core Combination ranking system. I decided to use that ranking system as a learning exercise as anything else. Eventually, I wanted to examine the individual components.

I used as my stock universe the S&P 1500. The reason is that some of my investing friends were not interested in trading small caps and micro caps, fearing that a small company could go under suddenly. While I knew that the best performing screen on AAII over the entire period they examined was “O’Shaughnessy: Tiny Titans Screen” (see https://www.aaii.com/stockideas/allstrategies), I also knew that the second best screen, “Est Rev: Up 5% Screen”, covered stocks of all sizes, not just small cap stocks. So I figured that the S&P 1500 was a good start.

I wanted to consider as wide a time as possible, as I wanted to see how it performed under different market conditions. I didn’t want to use a ranking method that performed great in one market regime but awful in others. I used the longest time period I could, which was 01/01/2002 to what was then the present, 07/02/2022. (More on this test period later.)

I then ran the simulation. I don’t remember what it was but I was not impressed. I was running a small portfolio of 5 stocks. I decided to change the sell rule from “Rank < 60” to “Rank < 90”. My reasoning was that for every stock I hold represents an opportunity cost as well as an opportunity. If I am holding a stock that ranks, say, 62, then that means that in the S&P1500 universe, there are ((1-.62)*1500=) 570 stocks that rank higher than the stock I am holding. Holding that stock is an opportunity cost, just like a value trap type stock. So with the “Rank < 90” rule in place, I ran the test again and got substantially better results. Hmmm, let’s try it with “Rank < 95”. I got even better results. Then I tried “Rank < 98”. I got worse results. So, I chose to set my sell rule to “Rank < 95”.

So with these rules in place, here are the results I got using the original Core Combination ranking system. I have also included the results of holding 25 stocks based on comments from Ryan Telford (“rtelford”) to my post “Texas Aggie(*) I/Q getting in the way - why are my tranches so different?”

[font=courier new]
all factors, original weights … 5 stocks … 25 stocks
----------------------------- … --------- … ---------
Total Return … 564.38% … 921.61%
Benchmark Return … 393.03% … 393.03%
Active Return … 171.35% … 528.57%
Annualized Return … 9.68% … 12.01%
Annual Turnover … 77.91% … 132.93%
Max Drawdown … -54.42% … -57.71%
Benchmark Max Drawdown … -55.19% … -55.19%
Overall Winners … (50/93) 53.00% … (433/774) 55.00%
Sharpe Ratio … 0.50 … 0.68
[/font]

Over a 20+ year period, the strategy earned 9.68%. While that is better than what the S&P 500 did, 29 screens at AAII did better than that over the same 20 year period. So obviously this ranking system is simply meant as a starting point and not necessarily to be used as-is. I decided to look at each of the six major factor sets (growth, low volatility, etc.) to see how each performed had I only used that factor set. The results will be in part 2.

Cary

2

Yes, the Core system is probably just a start. There are several systems here that will give significantly better results than this. The best place for me to start was here: https://www.portfolio123.com/app/opener/RNK/search (Take a look at Yuvals ranking systems)
Look at the ranking systems used, see which nodes they emphasize, and se how they weight factors. I have been a member for almost a year and am still frustrated :blush:! But its getting better.

As for AAII’s screens, they are, in my view, not usable:

  • There is no price control,
  • there is no control of the volume,
  • there is no control on number of stocks in the portfolio
  • Such as Est Rev up 5% screen has a huge turnover and is nowhere near what is possible for a regular investor to achieve

My main point is that when the AAII strategies is tested in P123, they usually loose money.

I would test also the system on:

More than 10 stocks
10-15 years. I believe the market has change so much that the numbers for the last 10 years is way more important than the previous 20.

And why limit yourself to the S&P 1500, and not go on the larger universes, but control the universe with:
AvgDailyTot(120) >( 100* 1000)
Spread(20)/Price < 0.2
StaleStmt = 0 // Current statement is available and up-to-date
Price > 1

And it also seems that the best systems have a lot of factors. My working project have 73 nodes.

Just some inputs from another newbie.

3

Many of the nodes and factors are equally weighted in the Core system.

For a while at least, Yuval described an optimization algorithm that, it seemed, might take the weighting far from being equal for some of the nodes (I am not sure whether he uses nodes or not) and factors. He can address the methods he is currently using. And the Core system is also his system to address.

But a fair question of Yuval might be whether the systems he invests in are as balanced in their weighting of nodes (if he uses them) and factors as the Core system is.

Whatever system you look at from whatever developer 2 (or maybe 3) questions will have been addressed implicitly or explicitly. Should nodes and factors be equally weighted? If not, how does one determine the weights? And perhaps, how do you decide whether to use a factor in the first place?

To the extent that there might be any debate on techniques here at P123 it revolves around methods of weighting factors and determining whether a factor is a noise factor (part of overfitting possibly) or a real signal. Any maybe there is a debate about whether some of that can be automated. Also, if it can be automated, what is the best use of P123’s limited resources (computing power and personnel). P123 does have to decide what priority to place on different projects.

A lot can be done in this regard with the present automation and downloads assuming a member is interested in how much weight he or she should put on a factor.

4

Thanks, everyone, for your comments. I will make it a priority to read the references you have provided. As for the AAII screens, one of my next goals was to try to implement in P123 the top 5 or 10 performing screens to see how well they did. I know that whatever my implementations would be, they would not replicate exactly what AAII does. For example, in their modeling of a given screen, if in one month the screen has 5 stocks passing, and then in the next month the screen has 10 stocks, the screen assumes that all of the money that is in the previous 5 stocks will then be equally allocated to the 10 stocks, something that almost no one would actually do (or perhaps I should say, I would not do that).

As for shortening my testing period to 10-15 years, believe me, I have considered that. I do know that sometimes the markets can change. 20 years ago there were almost no ETFs, and now there are 1000s. As an example, back in the late 1990s, I heard of the “Dogs of the Dow” theory (at the first of each year, buy the 10 highest dividend yielding stocks of the DJIA, hold for 1 year, then sell and repeat) and how the data showed it easily beat buy-and-hope on the DJIA (or the S&P 500, for that matter). So for 3 years I tried it, and for 3 years I trailed the DJIA and S&P 500. I stopped using the strategy after 3 years, deciding that it no longer worked. I was right that it no longer worked. While the idea of buying value stocks wasn’t wrong, the problem was that the composition of the DJIA had changed. Previously, when the method was developed, the DJIA consisted of mostly mature “smokestack” industries that paid nice dividends. By the time I started using the method, the composition of the DJIA had changed to include a number of stocks that paid little to no dividend. Thus, using dividend yield as a proxy for value resulted in choosing the same 10 stocks over and over again and thus poor performance. So, yes, the market had changed, specifically the DJIA.

But sometimes the market goes through a phase, sometimes a lengthy one, where one style of investing trails another style of investing for quite some time, yet there is nothing wrong with the previous method. Witness the Internet “DotCom” stock market from 1994-1999, where Value investing trailed Growth investing badly. Was Value now out of fashion, like the “Dogs of the Dow”? No. Instead people had fallen victim to the four most dangerous words of investing “This Time Is Different!” Once some of the dotcom companies started going broke (I think I remember that it was “Doctor Coop . Com” [drcoop.com ?] announcing that they were running out of money that burst the Internet bubble in mid-2000, but I may be remembering wrong.)

So the advice of to only look at the last 10-15 years may be correct, but I would like to look at as much data as I can. I know I risk looking at data that no longer applies (e.g., Dogs of the Dow type data), but I would prefer to take that chance than assume that the previous 10 years (where an accommodating FED has had loose monetary policy) lulls me into thinking that whatever method I develop will do well in all market regimes, like the current bear market or maybe a REAL, lengthy recession, which will most certainly not be true.

Also, to the person who created the “Core Combination” ranking system (Yuval ?), please don’t take my comments in previous posts as criticism. I see now that the system was simply a way to show how P123 can combine numerous factors into one system.

Regarding adjusting the weights of the different factors, that is exactly what I did next and I will show my (naive ?) approach to adjusting the weights in part 2.

Cary

5

I wrote about the Core Combination system a few days ago on the P123 blog: https://blog.portfolio123.com/how-to-make-money-trading-european-stocks/ I think much of what I wrote there is relevant to this discussion, even if you’re not about to invest in European stocks.

6

Yuval,

I wish this forum had a “like” indicator, as I would definitely be selecting it for your posts (and many of the others, for that matter). Alas, it does not.

Thanks for pointing me to that article, Fascinating. Again, my apologies regarding my comments for the Core Combination ranking system. I now understand that the ranking system is meant as an example of how to use the different factors. I may decide to invest in the European stocks in the future, but for the time being, I will stick with the one I know, namely the US Stock market.

Thank you again.

Cary

7

When looking at longer time periods, I wanted to suggest breaking results out into longer ago and nearer term time periods just so you’re aware how the dynamics may change. My sense is that almost anything systematic quant (doesn’t matter if it’s value, growth, or any other flavor or combination of flavors) could be made to work a lot better the farther back you go. I doubt this is a always true, but it’s just my sense of things and it seems to be mostly true to me, and it may make sense to be aware of how much the early years of a trial might be influencing overall returns. all imho.

8

Thanks for the suggestion. While I had not planned on discussing in detail my studies, I did a detailed study of 2 of the investment factors, “Core: Sentiment” and “Core: Value”, studying rolling 3-year periods, i.e., 01/01/2002 to 01/01/2005, then 01/01/2003 to 01/01/2006, etc. The reason I did so is that I wanted to see how the performance varied in those rolling 3-year regimes.

The reasons I care about 3 year periods are two: (1) my only source of money to invest / trade are in my 3 traditional and Roth IRAs, my retirement nest egg, and (2) I am 70 years old and already retired. My investment horizon is measured in years, not decades. If I spend 5 years pursuing an investing / trading method that performs poorly during that time, those are 5 years I will never get back. So looking at an investment method that has streaks of great performance and then streaks of poor performance does not appeal to me unless there is a reliable way to discover when the great performance era is about to end and the poor performance era about to start.

“Beware the daring of a cautious man.” (*) I remember that quote and wonder if it applies to me. Also, the adage among traders about one of the surest ways to lose money at trading is to invest with scared money definitely applies to me. I will try to combat both through work on this site and reading as much as I can, living with the nagging thought that maybe the best thing I can do is simply put my money in dividend paying stocks and try to enjoy my retirement years puttering around my 160 acre farm, instead of fretting with learning new investing methods.

PLEASE PLEASE PLEASE keep the comments coming. While I may not understand the comments, or may not agree with them, I am much better off learning from people like yourself who know what the heck you are doing better than I do!

Cary

(*) I first heard that saying from a political commentator in response to the Iran hostage rescue disaster of 1980. For what it’s worth, in the mid-1990s I worked on the flight simulator / trainer system that was built at Kirkland AFB at Albuquerque NM in response to that disaster. We were adding an electronic combat simulation system (ECSS). Fascinating place and fascinating work.

9

Mr SpacemanJones, sir!

I fear that my previous reply to your comment only tangentially addressed your point, namely that factors in the past that performed well may lose their effectiveness later, so that a investing / trading method that works well over a lengthy time period, such as the 20 year period I used, may perform poorly in more recent years because market participants have recognized its effectiveness and have arbitraged it away. (I think I remember that has been seen for the factors price-to-book value and price-to-sales.) This point is definitely valid and I had intended to address it in subsequent posts. In my presentations thus far, and in my next post, I have committed an egregious sin for developing investing / trading systems: I don’t have any out-of-sample data to use – I have used up all of the available data. I will explain shortly why I did that.

NEWBIE ALERT: If you are new to Portfolio 123 like me and you do not know about in-sample / out-of-sample testing, I STRONGLY encourage you to learn before you start investing much money in different investing / trading systems that you develop or modify. The book where I first learned about in-sample / out-of-sample testing was “Evidence-Based Technical Analysis” by David Aronson (see https://www.wiley.com/en-us/Evidence+Based+Technical+Analysis%3A+Applying+the+Scientific+Method+and+Statistical+Inference+to+Trading+Signals-p-9780470008744 ). In the boock you will read excellent critiques of subjective technical analysis and of the efficient market hypothesis. But more importantly, you will learn about data mining bias (curve fitting) and ways to overcome that, such as using out-of-sample data sets. In fact, he makes a good case to use 2 out-of-sample sets, not just one. There are some really smart people on this forum and I am sure that they can recommend other / better books and articles that explain the principles. Just be aware that you really need to make a practice of using out-of-sample data when developing an investing / trading strategy or you will probably be deeply disappointed.

Cary

10

As Whycliffes mentioned - “why limit yourself to the S&P 1500”? That is an important question.

The fundamental question you need to ask yourself is what advantage you have over other investors. Different types of investors have different advantages.

Professionals:
Investment team with superior skills in picking stocks and monitoring the markets all day, every day.
Powerful tools/systems.

Individuals:
Can invest in low liquidity stocks that are less efficiently priced.
Can react to news and buy and sell quicker with no restrictions and small trades do not move the stock price. No tax concerns if investing in an IRA.
Do not need to replicate the sector weights of an index or stay fully invested.
No restrictions on what they invest in ie no ESG rules, etc.

Limiting yourself to the S&P 1500 means that you are competing directly with the professionals and not using the main advantage available to individual investors which is being able to buy low liquidly stocks that few people are watching. With small caps, you are much more likely to find stocks that are mispriced because you have a powerful tool (Portfolio123) that few other individual investors have. With large caps, the people you are completing against all have powerful tools.

You mentioned that individual small cap stocks are more risky then large caps. That is true, but if you are holding 30, 40 or 50 of them in different industries, the risk goes way down. I have been using Portfolio123 since 2006 and I think I have only had one case where a stock went to 0. That was a Chinese company which basically made up its financial statements.

You should consider expanding your universe to include smaller companies that meet your liquidity requirements.

11

Dan makes a good point, which I agree with. You should expand your research to include additional universes.

Having said that, it is possible to ride the tailwind of institutional investors if you can figure out where they are headed. So the S&P 1500 would be appropriate in that regard.

12

Thanks to both of you. You both make excellent points, and I was fairly certain that limiting myself to the S&P 1500 would result in less stellar performance compared to a broader range of stocks. (I ran a quick test to test this hypothesis, with the results below.) The sole reason I limited myself to the S&P 1500 is that I have some investing / trading friends with whom I have been corresponding, with some of them continually saying “No small cap stocks! No small cap stocks!”. So when I initially joined P123, it was to evaluate the site for their use as well as mine, and thus I limited my exploration and testing to the S&P 1500.

I know that the investing / trading opportunities are much greater with small-cap / micro-cap stocks than mid-cap to large-cap stocks for the reasons that you folks cited, at least when using financial information as part of the selection criteria. After all, the top performing screen at AAII from 1998 to present is “O’Shaughnessy: Tiny Titans Screen”, which limits itself to micro-cap stocks.

But investing / trading mid-cap / large-cap stocks is doable. Prior to retiring and moving to western Arkansas, I worked in the Dallas / Fort Worth, Texas area. There was an investment group that met regularly there, where the leader of the group taught and used CANSLIM. (You can see the group’s web site here: https://armchairinvestor.com/) She, along with other folks not actually in the group, traded primarily growth type stocks that CANSLIM selected. They were doing very well for themselves. They were not trading microcap stocks, as far as I know. I quit attending, though, because I strongly disliked the subjective parts of CANSLIM such as using the “cup-and-handle” charting method. I wanted mathematically driven methods that permitted backtesting using historical data. (One reason I love P123.)

Another group in DFW that did not limit themselves to small-cap / micro-cap was the technical analysis group of Dallas. (I am not sure if they still meet. Here is their web site, which hasn’t been updated since 2020: http://www.afta-dfw.com/) Some of the technical analysts who made presentations used charting (I disliked those presentations), but most used mathematically based technical indicators. Several folks traded large-cap stocks and made a decent income (although not enough to get rich, that I know of). Still, it was doable, and I wanted to learn.

So, to repeat, I know that the points you made about my limiting myself to the S&P 1500 are valid. But I still think my exercises are helpful to me, in that I am learning (1) how to use the tools of P123 better, and (2) how (or how NOT) to assess the different factors to adjust the different weights. For those of you who already know all of this, you can think of this as entertainment. But PLEASE continue to make your comments, as I continue to learn so much.

Now to the performance of “Core : Combination”, broadening the universe to Russell 3000 and No OTC.

Russell 3000 results, with sell rule “Rank < 95” (no better than S&P 1500):
Annualized Return … 9.72%
Annual Turnover … 122.03%
Max Drawdown … -60.56%
Overall Winners … (382/713) 53.00%
Sharpe Ratio … 0.52

Russell 3000 results, with sell rule “Rank < 60” (substantially better):
Annualized Return … 11.66%
Annual Turnover … 20.11%
Max Drawdown … -51.99%
Overall Winners … (87/141) 61.00%
Sharpe Ratio … 0.66

No OTC, with sell rule “Rank < 60” (similar to the Russell 3000):
Annualized Return … 11.60%
Annual Turnover … 17.66%
Max Drawdown … -47.48%
Overall Winners … (80/126) 63.00%
Sharpe Ratio … 0.67