ETF Ranking Systems

Research
7

Don,

Turning back on my 2 buy rules in your Sim improves both the annual return and the max drawdown for the 5 year period and the 11 year period.

Denny :sunglasses:

8

Thanks for all of the good points Denny and Don -

This is becoming more involved than I had anticipated and I probably should have explained my original thinking.

I started with the IVY portfolio. There are several references here:

http://dshort.com/articles/2009/ivy-portfolio-update.html

My thinking was that I would start with this portfolio and then see what improvements I could make. BTW - the 1Month moving average > 10Month moving average has been backtested over the various asset classes for several decades. Starting in 2006 is “out of sample”.

So I created the 5 ETF portfolio using the same ETFs as the IVY portfolio with one exception. VEU only has a history back to March 8, 2007. I substituted EFA which goes back to August 27, 2001. Whether it is the best substitution is the subject for another debate. The simulation with weekly rebalance is here:

http://www.portfolio123.com/port_summary.jsp?portid=879356

Annual Return: 4.95%
% Drawdown: 13.30%

This is with 0.25% slippage. So this sim is not pretty… Yes it has fairly sizeable drawdowns… BUT the moving average filter SMA(21) > SMA(210) did its job. Taking out the moving average filter I got this simulation:

http://www.portfolio123.com/port_summary.jsp?portid=879357

Annual Return: 3.9%
% Drawdown: 47.01%

The drawdown was significantly reduced using the MA filter (from 47% to 13.3%) with less capital deployed. This result is absolutely consistent with the decades worth of backtest results.

My thought was to try to increase profitability by using a ranking system to select the best ETFs.

Now new variables have been introduced… changes to the moving averages and adding another ETF. These are all interesting changes and I appreciate these mod’s. My only concern is not having the decades worth of backtest to support the changes to the moving average filter.

More considerations:

The start dates for the five ETFs are:
VTI June 16 2001
VNQ Oct 1, 2006
IEF July 31, 2002
EFA Aug 27, 2001
DBC Feb 6, 2007

Asuming that one year of data is used in the ranking system or buy/sell rules then by rights the sim shouldn’t start before Feb 6, 2008. So I am cheating by using a five year sim - DBC is not included for the first year. (But let’s ignore that :slight_smile:

Now if I add 0.25% slippage into Denny’s sim and add TLT into mine I get the following results:

Dennys weekly rebalance: 57.38% Total Return

Mine with 4 week rebalance:
#1 91.28% Total Return (5 yr)
#2 79.44% Total Return (5 yr shifted one week)
#3 49.66% Total Return (5 yr shifted two weeks)
#4 55.53% Total Return (5 yr shifted three weeks)

With regards to Don’s simulation - it does provide better results but I would have to go back to square 1 and test all assumptions over again. Things like using SHY instead of IEF, etc…

Again - thanks for the suggestions!
Steve

9

Steve,

I’m not sure what you did to my Sim when you added the 0.25% slippage, but when I added it I got an annual return = 11.07%; Max drawdown = 15.13%; Total return = 214.84%; 81% winners.
You can see my Sim here.

Anyway, I agree with your concern about deviating to far from the well tested prior IVY approach.

Denny :sunglasses:

10

Denny - It’s because you are testing from 2001. I am testing 5 years.
Steve

11

Hi,
With all due respect to Mebane Faber’s work, which I follow quite closely, his team has turned in a very unimpressive performance with their ETF: GTAA. Since inception returns for the GTAA ETF were: AR -1.7%, StdDev 9.4%, and MaxDD -14.4%. At the same time my live (non P123) rotation strategy that I’ve been trading for years has returned: AR 13.5%, StDev 10.2%, MaxDD -6.8%, and that is with a limited set of funds and 30 day trading restrictions. Personally I’ve found that including many funds in a rotation strategy can hurt returns, and I think that may be the issue with GTAA. Maybe he should have stuck with the simpler approach.

While I regularly get ideas from others, I generally test for myself. I have done my own testing on moving averages on multiple markets going back as far as the 1920’s and on rotation strategies going back over 20 years. While monthly averages were suprisingly effective, and Oliver’s buy above 50 or 200 day moving average approach works very well, there are no magic numbers. Momentum works. The research generally says lookback timeframes from 3 to 12 months are predictive one month forward. Sometimes some numbers work better, sometimes other numbers work better. I resolve such issues through diversification (of assets and strategies) more than trying to worry about the optimum approach or testing.

The OP asked about ETF rankings and Steve provided a simple and excellent example of one. Denny and I offered some examples that showcase some minor variations. I’ve found this to be one of P123 biggest strengths: discussions and sharing of various trading approaches. In any case, I wouldn’t recommend that anyone apply someone else’s trading rules without testing it for themselves and judging it by their own criteria. I take what I can from others’ work, including the IVY portfolio, but I don’t limit myself to that. I also include what I’ve learned in my own research, experience, and testing and build my strategies from that.

Don

12

Don - congratulations on your many years of outstanding returns. I am where you were umpteen years ago. I’m looking for a good system but 5 years backtest isn’t sufficient for me. I have also found that adding too many securities into the mix reduces performance. I’m not really sure how the GTAA compares to the IVY portfolio but was surprised to see allocations such as below:

The actively managed Cambria Global Tactical ETF (NYSEArca: GTAA) weights U.S. fixed-income at 59%, international fixed-income 5%, international emerging markets 2%, U.S. stock 13%, commodities 1%, U.S. Real Estate 11%, cash 8% and currencies 7%.

I have been looking a long time for extended series of historical data for the main asset classes, or if you wish asset class categories (for the more picky readers). I had prevously found monthly data but wasn’t end of month. I think it was a monthly average price.

Please let me know if you have a source of historical data that I can get hold of and play with in EXCEL. Then I can take the results and model the last 5 years in P123.

Thanks
Steve

13

I really appreciate the chance to see the different approaches to ETF ranking - ETFs are a new way of thinking for me.

One thing I never liked about the Ivy Portfolio was limiting it to 5 (or if you read the book, fewer than 5) ETFs. In creating my version, I decided to use 5 stocks for the US market and 5 REITs for the real estate portion of the strategy. This works well for me since it stays in the realm of stocks, not ETFs.

However, creating the ETF variant for Countries and Commodities has not been as successful. Using some of the ideas presented here, I came up with the following sim:

http://www.portfolio123.com/port_summary.jsp?portid=879091

Any suggestions/feedback are welcome.

By the way, I do not commit 20% of the strategy to Fixed Income - just 25% to the other 4 components and let Fixed Income be the default when not invested.

Jim

14

Hi,
Jim, excellent sim. It’s performance relies in part on strong performance from the commodity funds over the time frame, as running the sim with a 1 year rebalance still gives returns of almost 10%/year. It is also different from the others mentioned earlier in that it has a shorter hold time and tends to buy reversals, rather than trends, but that can be a very effective approach.

Steve,
I’ll follow up with you directly. What I will say here is that I simply downloaded data from Yahoo and tested in a spreadsheet. I had 36 funds with most going all the way back to 1989, and I tested thru 2008. In one set of testing, I ran 368 rotation test cases plus a buy and hold SP500, buy and hold a diversified 75/25 portfolio, and market timing the 75/25 (think I used the 10 month moving average, of the market, not the individual funds). The advantages of momentum is shown by sorting the results by risk adjusted return (AR/((StdDev-MaxDD)/2) and absolute return. The percentiles are below, where 100 was best, 0 was worst.

B&H MT: Risk adjusted 56, absolute 7
B&H Diversified: Risk adjusted 6, absolute 0 (2nd to last)
SP500: Risk adjusted 0 (last), absolute 0 (last)

In these tests I rebalanced monthly, but I varied the lookback period, the method for measuring momentum, the number of funds to hold, how aggressively to sell, and tested with and without market timing. That ALL cases beat the SP500 and perform so well relative to a diversified portfolio really sold me on the approach.

Overall results are below.

Don

RotationResults.PNG

15

Jim - I had a look at your sim. One of the issues I have with what you are doing is that there are at least 37 funds that meet your buy criteria.

When I looked at your realized trades it was apparent that there is a tendency for the system to select all of one fund type. For example, for the 5 trades closed on Feb 13, 2011, all five were bond funds.

This is not to say what you are doing won’t work, but there is no diversification…

Steve

16

Hi Don:

I appreciate your comments about asset rotation and testing. I very much like to do my own testing to confirm ideas. I find this helps me stay the course when an approach has one of its draw downs.

I’ve got a few questions about your table of test results:

  1. To what does the “avg top 100” refer? Is that a benchmark you constructed by averaging the top 100 etfs selected in highsight? Or is that one of your ETF selection methods? Or is the top 100 best runs in the 368 tests done?

  2. Simiarly to what does “avg All” refer? To the average of all 368 of the tests?

Thanks,
Brian

17

Hi Brian,
Avg all is the average of all tests, even the buy and hold (it was easier to do)

Avg Top 100 is the average of the top 100 sorted on risk adjusted returns, which is Annual Return / avg(StdDev, MaxDD). I think this is useful because some of the parameter selections I made were not very realistic, and with the hindsight of testing I saw advantages in some approaches over others. No guarantee that these advantages will hold up in the future, but that’s how I would bet, and so I think this avg is something of a benchmark for the rotation approach during this timeframe.

Don

18

Jim,

I feel in trading a Port of your Sim it will be difficult to get the performance the Sim shows. 2 reasons:

  1. The slippage is very low at 0.025%. For a liquidity of $5 Mil/day I have found my real slippage to be slightly > 0.1%. To be a little more conservative I run 0.2% slippage with $5 Mil/day & 0.1% with $10 Mil/day.
  2. The average gain/trade is only 0.81% (from the statistics, Trading Page), and any additional slippage will eat most of that up. I don’t consider trading a Sim that doesn’t show > 2% gain/trade using a conservative slippage.

Here is another ETF Sim to consider that has decent performance: ETF Rotation - Basic 2 Stks, 1 Wk, 10 Mil, AR=18%, MDD=18%, 65% winners

This Sim trades 2 long only ETFs, $10 Mil/day liquidity, rebalances weekly, and it uses 3 market timing rules.
It has an annual return of 18%, a Max drawdown of 18%, total return of 544% and 65% winners. The average gain/stock is 5% and the average days held is 70.

Denny :sunglasses:

19

Don, Steve and Denny - thanks for the feedback. All good and things I need to address.

I wasn’t trying reversals per se, just trying to avoid Faber’s use of DBC for trending. I generally use the weekly refresh to weed out stocks dropping in rank, but the turnover here is huge. Is there a way to check without the high refresh?

I looked at the bond fund periods as a signal to go to cash. The screen version of this turns out to be an interesting market timing mechanism. If I cannot find a mechanism to limit the commkodities to 2 or 3 per type, I was planning to create a list. Is it possible to detect the shift to bonds in a sim and buy nothing?

Because my stock sims generate few trades and have high average gain/trade, I have been pretty sloppy in examining slippage. Something I intend to change.

Again, thanks for the help.

Jim

20

Hi,
Regarding turnover. There is generally a tradeoff between performance and turnover. In my experience higher performing strategies tend to have higher turnover, and so trading costs have a much bigger impact. I’m not sure which ranking system you are referring to but simply holding to a lower ranking will lower turnover and likely decrease performance but also trading costs.

To identify when bonds are top performing, perhaps you could use forder to sort all ETF’s, select the top 1, and check it to see if it is a bond fund. I expect going to cash will underperform going to bonds.

Don

21

Using the P123 Piotroski system and adding liquidity criteria and EPSCurrYr timing gives me a long holding period (about 200 days if I remember correctly), avg return/trade over 30%. Increasing the turnover does not seem to help performance.

The ETF ranking systems seem different but it may be due to my lack of experience in thinking this way.

I am going to try the forder approach you suggest as soon as I have a little more time. Thanks.

Jim

22

Okay, so I didn’t remember the numbers very well. I made a public version that includes some trading liquidity and EPS Curr Yr timing.

http://www.portfolio123.com/port_summary.jsp?portid=881675

There is very low turnover for the period.

It seems like the sell rules (with the exception of trailing stops) are only evaluated at the rebalance. In that case, I need to incorporate something like what Denny demonstrated.

Jim

23

I’m starting to look into ETF ranking systems to use in my Fidelity 401k. They have a number of ETFs that trade commission free which is of use for low capital or higher frequency systems, though trading restrictions are 30-day minimum for the free ETFs.

Is anyone willing to restart this discussion or share a starting point for an ETF ranking that would work well under these constraints? Thanks!

24

Caley, some momentum factors actually do better with four week rebalance rather than weekly. Thirty days is close enough to four weeks to make it work.

25

I don’t have a ranking system but I do have some suggestions on designing an ETF book which did not exist a couple years ago.

  1. Diversify with stocks, bonds, gold, ( I don’t like resources)
  2. Diversify with many designers including yourself.
  3. Use different market timings and Hedging

Here is a book that would cost less than $100 and by playing with the Assets and Hedging you can get the drawdown lower than 8% with a return of 20%. This might not suit your needs for free trading but it would come close since it is low turnover.

https://www.portfolio123.com/port_summary.jsp?portid=1443293

MV

26

Thanks MV, but I don’t have access to that book…