The 4 sources of excess returns.

[yuvaltaylor]

Notice no one is claiming that good statistical methods are a source of excess returns.

I’ve never met a statistical test that told me something I already didn’t suspect to be true or false. In this sense, they are simply plot devices we can manipulate to intrigue ourselves with truisms and/or shape reality to fit our own narratives.

As Jim rightly points out, good statistical methods are indeed a source of excess returns. The entire idea of behavioral economics is that without a sound grounding in statistics, investors (among others) make all sorts of mistakes.

Statistics is the science of empirical data, of how to analyze and interpret it. It is, of course, easy to misapply statistical methods to a set of data, and I see your point. But just because some econometrics is based on flawed reasoning, we shouldn’t dismiss all statistics as irrelevant. It can be helpful for investors to know, for example, what mathematical operations can be sensibly applied to annualized returns; how to interpret regression to the mean; when linear regression is appropriate and when it isn’t; sensible measures of volatility, variance, and correlation in order to deal with risk and uncertainty; probabilities when faced with investment decisions; sufficient sample sizes, in order not to make false inferences from data; testing procedures that are robust rather than subject to outliers; and so on. All of these can lead to excess returns.

Here at Portfolio123, most of our work is based on statistics. When we use the LoopAvg or the Aggregate function, we’re using statistics. When we provide you with alpha, Sharpe ratios, or R-squared numbers, that’s statistics. Bucket rankings are statistical, as are the average returns of rolling backtests. We give you histograms to help you interpret simulated returns. We use point-in-time data and are free of survivorship bias because of statistical principles.

Statistics has been misused in a lot of the “soft” sciences; psychology, policy studies, and economics are especially rife with problems, and statisticians have been complaining about this quite loudly lately. I enjoy the discussions in this forum about when statistics have been used properly and improperly, though I plead guilty to being unfairly dismissive of certain statistical practices (having been properly chastised, I’m going to be much more tolerant now). We shouldn’t view statistics as a “plot device” or a way to make data say what you think it should say, even if it is sometimes used that way. Compared to geometry and astronomy, statistics is a relatively new science. But a science it is. Without it, P123 wouldn’t exist.

[Jrinne]

Yuval,

Well said.

For my part I think I should keep in mind that, in school, they really do teach the most important stuff first.

I should keep in mind that sometimes a histogram says more than a t-test.

In any case, I (and perhaps those reading my posts) would probably be best served keeping to Statistics 101 in the forum (for the most part) and things that have already found wide acceptance in Finance at that. Things that could be looked up in Wikipedia or Investopedia during any discussions.

I would not add much to what P123 is already doing.

If I had a recommendation it would actually be what would be taught in a Machine Learning 101 class in the first week: Cross-Validation.

This is something that P123 already does and it is essentially paper trading. P123 does this by forcing us to run our designer models out-of-sample for a period before we can sub the model.

I am still learning about this. I am still struggling with which metric to use, for example, which you can see in some of my recent posts. Looking at the top bucket will serve me well until I find the perfect metric (which probably does not exist).

But I think, done well, one can exclude SOME models without paper trading them for long periods. I do not think this replaces paper trading but just gets rid of some of the bad systems early and allows one to paper trade the systems with the most potential.

P123 is probably already doing everything it can to help us with cross-validation through the requirements of the designer models. My interest is with my own private models and I have no interest in lecturing anyone on this.

I do need to remember that they do teach the most important things first and that there are reasons the field of Finance has selected certain statistics already.

-Jim