"Curve Fitting For Fun . . . " Revisiting that Designer Model

Hi all,

It’s been nearly two years since I launched the “Curve Fitting for Fun and Maybe Profit” Designer Model. Let’s see where we are.

THE IDEA

Curve fitting on Portfolio123 has been occurring, and I’ve come to recognize certain factors as being key components to the curve fitters’ palette – and I created a ranking system named just that, “Curve Fitters Palette,” based on those factors. Most of these are used because, although they make no sense at all from a financial standpoint, they have often showed well in simulations of the past. Others are legit for those who really want the styles they represent (mainly the price and size-based factors) and were included because they tend to be favored by curve fitters given the way conditions post 1999 allowed them to flourish (meaning their presence tended to boost simulations that started in 1999).

Saying the factors are no good, etc. is all well and good but the reality is that we live, invest and trade in a world ruled by supply and demand so financially speaking, it can be argued that “might makes right.” If enough people think a nonsensical ratio like Pr2SalesQ is worth using, does might-make-right turn it into something that ought to be considered? Put another way, using the language of the p123 virtual strategy design course, can such items be used to harness and potentially work with the ever-present and more-important-than-many-realize “noisy” component of stock prices? (Besides P123, who knows how many elsewhere are working with similar factors.)

I tested it by applying this ranking system (which is now publicly visible: [url=https://www.portfolio123.com/app/ranking-system/272364]https://www.portfolio123.com/app/ranking-system/272364[/url]), not to an entire universe but to a subset that has been screened in a modest way for basic liquidity and some simple rules using sentiment and quality (the latter not being an effort to grab the best but instead to weed out the worst).

THE OUTCOME

As you can see on the Designer Models page, this strategy closely tracked the Russell 2000 ETF since it went live on 9/11/15. That’s actually not bad given the handicap of an equal weighted portfolio versus a market cap weighted benchmark at a time when big caps have been in favor. So based on this back-of-the-envelope ranking system (I didn’t even bother to think about weights; I just made everything equal), it looks like there may be something to the notion that constructive work can be done using these factors as a way to harness market noise.

A simulation that replicates the Designer model is also publicly visible: https://www.portfolio123.com/port_summary.jsp?portid=1498864

I’ve long believed and argued that screening is critical to strategy building, that when we rank, we push the statistics too far when we ask the top 10 stocks to be better than the next 10, the next 10 to be better than the 10 after that, etc. I see screening (buy rules) as the equivalent of a pre-qualified prospect list purchased by a salesperson as an alternative to prospecting from a telephone directory. When serious screening rules are used, you limit application of a ranking system only to a subset of the universe that you believe has some potential to blend well with the ranking logic.

I decide to establish a second sim in which I took out the buy and sell rules that didn’t related to liquidity. I had to have something for selling, so I added Rank<90. That sim (which matches the live dates of the Designer Model version) is here: https://www.portfolio123.com/port_summary.jsp?portid=1498866

As you can see, it’s bad.

Finally, I set up a third version in which I even eliminated the liquidity rules. There are now no Buy rules at all and only one rank-based sell rule (Rank<90):
https://www.portfolio123.com/port_summary.jsp?portid=1498874

The third one is also pretty bad.

MY TAKEAWAY

I’ve seen what I wanted from the Curve Fitter Designer model and believe it’s time to put it into retirement, which I’ll do in the very near future. (I opened everything up here in case the few who follow it want to replicate it in their own accounts).

As to curve fitting, if we recognize these factors for what they are, it looks like there is a sound basis for their use. The key is to go in with eyes open and recognize how they might be helpful; as momentum-oriented factors that help one ride the N part of the P = V + N (price = value + noise) equation. And as momentum goes, these can be particularly interesting since most who work in this style stick to prices only. Harnessing something we might call “fundamental momentum” might lead to less crowded momentum trades.

One caveat though: Please take the Buy-rules seriously. That can help you a lot. Think about what kinds of stocks might be most amenable to momentum plays like this and apply the ranking system to those. Admittedly, the “good” version of the model was nothing to write home about, but it accomplished a lot using rules I spend less than a minute thinking about. It would be interesting to see what might result if more effort were dedicated to creating good sets of Buy and Sell rules.

Thanks Marc, interesting food for thought.

I am getting access restricted when I follow the link https://www.portfolio123.com/app/ranking-system/272364

Marc, I created a curve fitting Designer Model before yours when the Picklehead models started to lure all subscribers due to the insanely ridiculous annual returns achieved in his backtests. Nearly three years later, mine is surprisingly not doing so bad… In fact it is even beating the S&P 500 TR! In conclusion, this proves even a long incubation period of time is useless in identifying curve-fitted models from good ones. See for yourselves:

https://www.portfolio123.com/app/r2g/summary?id=1202693

This goes back to one of my recent posts, where I spoke of focusing on Mid and Large caps as opposed to small and micro. I feel that curve-fitting and/or model building mistakes are a lot more forgiving in the large/mid space, seemingly giving a market rate of return if poor, as opposed to hugh losses in a poorly constructed small/micro model.

I think it depends on the type of curve-fitting employed. If you read the description of mine, I combined a strong ranking system with a curve-fitted market timing model. The result of both together is that the market timing avoids all market corrections since 1999 but becomes blind post-launch of my designer model. Therefore, its only purpose is to create cosmetic / artificially inflated annual return numbers and pristine metrics such as incredibly high sharpe ratios, low drawdowns, etc. Now that it is out of sample, it only relies on the ranking system, which is legit. I believe if I did the same thing with a small cap universe, maybe I would have still outperformed the benchmark.

Anybody else experiencing this? I checked and the system is set for public visibility and I tried getting access using a different account and succeeded.

Me too, it doesn’t work.

I remeber.

You were going after a different kind of problem; market timing rules created with perfect hindsight. I don’t really know anything we can do to make that sort of thing work – absent praying to Goddess also known as Lady Luck.

Works for me.

It is working for me.

Marc, Maybe I’m misunderstanding intents of all the elements of your model, but I was wondering why you consider items like Q/PYQ change in something like Sales or Earnings to be likely curve-fitting? I’m curious because I am using them, granted in combination with longer term similar measures, but still the most recent quarter to me is often where a stock with good longer term values can sometimes crack up and provide warning signs. Is the concern mainly about using these in isolation? thanks,

I don’t know about Marc, but I generally caution that the quarterly income statements and cash flow statements don’t take into account seasonality. If you’re looking for great income growth, you’ll jump into a bunch of retail stocks after they report 4Q earnings and then leave when they report 1Q earnings.

It’s not a problem with sequential TTM or annual comparisons.

It seems there may be some caching of some sort going on and/or it may have something to do with when you logged in relative to when I changed the setting to public. So if you’re having a problem, you may want to log out and then log back in.

The link is now working for me. Thank you

Hi Marc, I’m new here, but I’m interested in reading more of your perspective on integrating the screening/ranking process if you can point me in the right direction. Particularly on process. I’m assuming you start with a restricted universe (for example: companies with declining sales) and then proceed to tailor a ranking system particular to that universe (that likely can end up looking quite different from an all stocks ranking system)? Do you think restricted industry specific ranks make sense, or should good ranking systems be more broadly applicable? thanks,

btw, I got your book on screening in the mail just a few days ago but haven’t gotten too far in yet.

Hi Marc,
Thanks for sharing.
Do you mind explaining why you think EPS%ChgPYQ and Sales%ChgPYQ makes no sense at all from a financial standpoint? EPS%ChgPYQ is in the rank you shared in this thread.

gs3

Marc,

Thank you for sharing this.

I think I agree with everything you said in this recent post. Furthermore, I agree wholeheartely with all of your posts regarding a cautious approach when using factors that may not make sense.

To supplement your comments I thought I might look at what “significance” one can attach to your out-of-sample results for this port.

Of course, “significance” can be defined in many ways. More importantly, there are other ports and other things to consider in determining what factors might lead to excess returns.

Below is a Bayesian test (BESTmcmc in R) assigning different credibilities for excess returns compared to the benchmark you used in your port. This is the mean of the daily log returns for the port minus the daily log returns for your benchmark.

The CompVal was set at 5% excess annual return—in excess of the benchmark. In other words Exp(252*.000198) = 1.05116 which is approximately an excess return of 5% annually. So this is where the 0.000198 comes from in the attached image.

About a 42% credibility (the 41.7% in the attachment) that the parameter for this port’s returns is 5% or more in excess of your benchmark’s returns by this test and based only on this data.

FWIW

BTW, I might have guessed you would say something like: “This port suggests you guys are not hurting yourselves too much by using this stuff.” Anyway, if you had said this I would have to say the data supports it.

-Jim


In your opinion, do “price to sales quarterly” and “price to free cash flow quarterly” make no sense? Or are they legit to represent the value style?

How about “price to sales TTM” and “price to free cash flow TTM”?

Thanks.

Paul has it right. Seasonality is a huge problem for this sort of thing. For more discussion on this sort of thing, see this, from the virtual strategy design class: https://www.portfolio123.com/doc/side_help_item.jsp?id=221

See this from the virtual strategy-design class.
https://www.portfolio123.com/doc/side_help_item.jsp?id=221