What would Ed Thorp do?

Manage
1

I have been a huge fan of Ed Thorp ever since I read my first book as a child: “Beat the Dealer.” Okay maybe it was a few books after Dr. Seuss but early.

Anyway, I know he looked at a lot of different ways to beat the market and discarded many of them including technical patterns. He made most of his money developing the Black-Sholes formulas (before Black and Sholes) and not publishing them. I.E., he traded and hedged options and warrants.

But what would he say about what we do? His recent autobiography tells us (A Man for All Markets: From Las Vegas to Wall Street, How I Beat the Dealer and the Market):

“The Compustat database provided historical balance sheet and income information.”

“….indicators we systematically analyzed, several correlated strongly with past performance. Among them were earnings yield (annual earnings divided by price), dividend yield, book value divided by price, momentum, short interest (the number of shares of a company currently sold short), earnings surprise (an earnings announcement that is significantly and unexpectedly different from the analysts’ consensus), purchases and sales by company officers, directors, and large shareholders, and the ratio of total company sales to the market price of the company. We studied each of these separately, then worked out how to combine them. When the historical patterns persisted as prices unfolded into the future, we created a trading system called MIDAS (multiple indicator diversified asset system) and used it to run a separate long/ short hedge fund.”

I guess he hedged but pretty much what we do.

-Jim

2

I am also heard him an Investor’s Podcast interview say that while all investors try to outsmart the crowd, very few actually can and will. What stood out to me was his claim that – in order to stand apart from the crowd – one had to find some special and uncrowded niche.

That’s what I am trying to do, but am having some difficulty determining how I am really that unique.

Ed Thorp’s advice for me – and most people – would probably be to work hard, save smart, invest in ourselves, and then hand over anything left to John Bogle to invest. For >~98-99% of people, that’s probably the good advice.

As far as for how to beat the market, I am pretty sure Thorp did it by staying a few steps ahead of the crowd and the house.

3

He learned how to beat roulette by predicting where the ball would land with a computer worn in the casino. He beat Baccarat and of course Blackjack. Niches to be sure. He found them, in many cases, because no one else thought it was possible.

Everything was just pure math (he had a Ph.D and was at MIT for a time). He calculated an expected return. And if there was an edge he used it.

The higher math came with his managing risks. Starting with the Kelly Formula for Blackjack. Later his hedging of options was quite remarkable. He never had a quarter with a loss at his hedge fund. The S&P 500 Index fell 23 percent on October 19, 1987. He was well hedged when this happened and he made money when the S&P futures became mis-priced. He bought the futures while shorting stocks—having trouble getting enough shorts because of the uptick rule.

He was aware of and worried about fat-tails in the 80s.

Okay, discovering the Black-Shloes formula was pretty advanced and was not entirely about risk. Controlling risk was just a big part and I am exaggerating.

Still it was just math. Advanced math. Elegant math. Often, math that I cannot do. But just math.

And he was doing what we are doing with Compustat—along with other things to be sure. Can it still be done in 2017? Maybe we do need to find an uncrowded niche—I’m probably not the only one who read his book. Maybe that is what he was getting at. I will listen to that podcast and try find what niches he might be recommending.

Thank you. I will definitely listen to that podcast!!!

-Jim

4

I enjoyed that biography very much, and very much recommend it. The stories of what it was like in the early days of computing and trying to keep the computers in his Los Angeles from overheating were very vivid and amusing. The only qualm is I wish it would have gone into more detail about his health and nutrition strategies as it had about his blackjack strategies. I know he’s quite fanatical about health, and I assume his outlook on it is as empirical and independent as everything else he’s dabbled in.

As far uncrowded niches in equities, I’m assuming these are microcap and non-US markets. Also high volatility markets, because fund managers cant participate in them without losing weak stomached clients. This is probably more true than ever with the move to passive, as clients have very littlie tolerance for periods of underperfomance to the major indexes.

5

I suspect he wouldn’t be making public any proprietary information that still held substantial value. So publications and talks etc are more of a tour of age old magic tricks for fun and ego fulfillment.

His story is interesting though.

6

“Fortune’s Formula” by Poundstone is a good read about the Kelly Criterion and Thorp’s involvement. For a while, Kelly worked at Bell Labs in Murray Hill, NJ, my old haunts.

7

Primus,
Thank you for the link. Of course, I listened to it. As you recall Mr. Thorp was asked about using the formula for more complex bets than simple Bernoulli trials with fixed payouts—as you find with Blackjack betting. Mr. Thorp’s answer to that was that this is hard. This is a question I (Me not Ed Thorp) have never been able to nail-down.

Ed Thorp recommended a book that he helped edit: “The Kelly Capital Growth Criterion.” This is a collection of many articles. So chapter 52 has the following answer to the question in the podcast. I could not copy and paste directly so to summarize:

Your optimal fractional bet = (μ-r)/(σ^2) where μ = mean return, r = risk-free rate and σ = standard deviation. And:

“optimal growth rate = 1/2*(Sharpe Ratio)^2 + risk free asset”

I would be the last to say that it is really that easy or that this is the final word on this. And indeed the author himself later modified this using something like the Sortino Ratio. Furthermore, Ed Thorp recommended 1/2 optimal Kelly for most people. I would add my own 2¢ but I think I will leave the last word to Ed Thorp (Editor) and the author of the article (William T. Ziemba: “The Symmetric Downside-Risk Sharpe Ratio”).

Much appreciated.

-Jim

8

Interesting you bring up those formulas for optimum bet size. Sizing bets in accordance with expected outcome is actually one area where Las Vegas is ahead of Wall Street.

For example, here is an interesting on article on Kelly: The Kelly Criterion - Wizard of Odds

And here is an excerpt which supports using mean-variance analysis:

Bet sizing in accordance with expected outcome is one area where I believe P123 is lagging. And actually, according to Stochastic Portfolio Theory (an analogue of continuous Kelly growth criteria), sizing bets as an inverse of market cap is more optimum than using the market weights. Still, for a large portfolio, I think custom bet sizing makes a huge difference when controlling risk and reward.

9

Equal-weighting is OK for most ports since most are picking only five or ten stocks from a much larger universe.

But if people have built ranking systems with a true edge (the higher ranked, the better performance) and are running larger ports (20+), then they need to be able to weight their positions in accordance with rank.

For example the Guggenheim Pure Value ETF has about 100 stocks and weights according to rank.

And once you weight by expected return, adding expected volatilty and weighting by MVO becomes a possibility.

This is the final frontier for P123.

10

Parker and David,

As you both know, what you are suggesting is right out out of Kelly’s original paper. Kelly writes: ''The essential requirements….the ability to control or vary the amount of money invested or bet in different categories. The “channel” of the theory might correspond to a real communication channel or simply to the totality of inside information available to the investor."

I tend to think in terms of how much to place on a Blackjack bet. Or how much money to put into an individual stock or port. But both of you have pointed out that it was originally about betting on multiple, different outcomes. Weighing your bets according to your expectations.

-Jim

11

All:

 With Vince, Kelly turned into optimal f which is the last I heard of it.  FWIW:  Optimal f is a great metric for looking at system efficacy.  In general, for a good system, optimal f >> 2, which is what my broker is happy with.  No one sane trades optimal f. Fat tails, and black swans.

Bill

12

The thing I struggle with is turnover. I know that, mathematically speaking, the sample distribution approaches the underlying process as number of trades goes to infinity (due to CLT). This means that high turnover models will – in theory – expose their true edge more quickly than lower turnover models. Yet, I can’t help but to realize that in many empirical tests, high turnover brokerage accounts do worse than low turnover ones [references available upon request].

How can I be sure that churning my account is equivalent to making more independent and identically distributed bets? How can I ensure that churning is not simply just churning (i.e., making the same bet over and over again)?

13

p:

Typically systems do not make iid bets.

Bill

14

Bill, David (Primus) and All,

Can you expand on your knowledge of this a little? Is it okay that much of our data can be made stationary? Is it ergodic and if so does that help?

While we are on the topic of geometric compounding (Kelly criterion) it might be noted that the returns are path independent. Does this relate to ergodicity?

Bill, good point. Our stock bets may not be (probably are not) i.i.d., I think. And I would like to learn more.

My 2¢: I find it difficult to find texts that specifically address these issues for investing. Although, ergodicity is frequently discussed in “The Kelly Capital Growth Criterion” collection of articles. Making the data stationary seems to be a minimum requirement in the literature but seldom do they say whether this is always adequate or what limitation may still exist. It does seem that even the most compulsive on this issue just do an Augmented Dicky-Fuller test and act as if the issue is settled if the data passes this test. FWIW, daily logarithmic returns that are detrended by an appropriate benchmark have alway passed this test when I have checked. This does include all of my active ports.

Thanks in advance for any information, links, opinions etc.

David,

My experience seems to confirm this.

I have looked at this extensively with my models. One way to resolve this is with bootstrapping. Specifically with the BCa method. Comparing a simple t-test, bootstrapping and even Robust Bayesian methods always give very similar confidence intervals (or credible intervals for Bayesian statistics).

Interestingly, even bootstrapping the median gives the same (or very similar) confidence intervals as long as I use the preferred BCa method in my experience. Percentile and other methods: not so much.

Best points estimates of the mean can diverge from other estimators of the central tendency for a long time, however. There tends to be a good amount of skew and fat tails in the small data sets that impact the data for a long time. The data seem to eventually become symmetrical with larger data sets, however. But, even with samples greater than 150 there can be a significant difference in the point estimates for mean, median and Bayesian statistics estimating the mean using the t-distribution (all due to skew in the sample and fat tails).

But point estimates are meaningless anyway, IMHO. Welch’s t-test with the CLT assumption gives as good of a confidence interval as any of the tests in my experience.

-Jim

15

Jim,

This post, on the stylized facts of security returns, may be relevant to the conversation: https://quant.stackexchange.com/questions/34597/have-any-new-stylized-facts-of-asset-returns-been-discovered-since-2001

Bill,

Can you explain why any number of discrete trades (or bets) are not i.i.d.? Why is it not warranted to view every discrete trade as an independent sample drawn from an underlying distribution?

16

David,

I would still like to get Bill’s take on this.

But even if our data is not i.i.d. we probably can still do some statistics. Specifically, there are theorems that extend the central limit theorem beyond the assumption of i.i.d. They all assume stationary data. Then there is always some addition of “mixing,” “Martingale” assumption or “ergodic” assumption for the many different proofs (they are legion). These theorems are not easy to read. And even when understandable it is difficult to “prove” that it is okay to do certain statistics with P123 data.

Here is a mostly readable link: https://www.stat.tamu.edu/~suhasini/teaching673/time_series.pdf

There is a lot there and I will not try to distill the math.

But my practical take is this: if you take the log returns of a fairly short period (days, week, maybe months) and detrend it you are probably good. To check one can do an augmented Dickey-Fuller test to test for some of the assumptions of stationarity.

New to me is the idea (from the link) that you can do an acf test in R (tests for autocorrelation over a range of time periods). This suggests that you might be okay on the ergodic (or weaker) assumption. Specifically, you are checking for long-term correlations that could prevent the data from being ergodic. The image shows an example for some of my data. But the link has multiple examples. And as I said above, I am not sure that I have proven it is okay to use this data in a t-test or a regression. For example, one could easily argue that the acf shows little linear correlation of the means and that any correlation is short-lived (if present at all) but there are other correlations (linear and non-linear) that are at least possible and are not ruled out (see link).

My data show only small amounts of correlation with little pattern that if present is quickly extinguished. All within the error bars. And it passes the augmented Dickey-Fuller test.

This may seem a little boring (okay it is) but this goes beyond statistics into the field of “prediction.” Paul Samuelson and others have developed much of this in an attempt to answer the question of whether we can even talk about predicting the stock market. Which I have been doing largely on faith up until now. Although, I note that the developers of many of the techniques used by P123 probably were aware of all of this. For example, the way the rank performance tests are done do not violate any statistical assumptions (and is widely used). People are free to draw their own conclusions as to whether this non-statistic is ergodic or not.

If there is anyone who thinks they can just look at the equity curves without doing all this, I have this to say: I think you are probably right about that.

-Jim

Screenshot 2017-06-18 13.00.22.png
Screenshot 2017-06-18 13.00.22.png1136×666 20.1 KB

17

p:

One way to see it is along the line of what J’s doing. One test I run is to plot a scatter plot of the return of month N + 1 vs. month N, over 15+ years of data, and draw a least squares line through it. For good systems the line typically has a distinctly positive slope (as well as intercept) with strong returns in month N predictive of strong returns in month N + 1. No big news here, the market trends at least some of the time. But the returns are definitely NOT iid, which implies the underlying trades are not iid.

Bill