I’ve built ranking systems to mimic Barra risk factors and would like to create an aggregate series that I can regress over 36 months of returns to calculate the respective risk factor betas.
I have a couple of questions:
How do I align the monthly aggregate series returns with individual stock returns? Should I use LinReg for this?
What’s the best approach to create the aggregate series? Would it be better to use the optimizer feature to generate rolling 1-month ranking system returns (e.g., using the Hi/Low spread or slope) and then upload them? Or is there a way to do this directly with aggregate series?
I’m aware that a risk model is in development, but I’m unsure if I can wait that long. In the meantime, I believe this method could provide something better than nothing and relates to what many are working on in terms of factor momentum.
The image represents the value factor's monthly return as an imported data series. I believe I used Barra's methodology to calculate monthly cross-sectional returns, utilizing the slope of decile returns as a proxy for a monthly cross-sectional regression.
Now, I need to regress this value factor against monthly stock returns over the past 36 months. I attempted to create a formula, but I can’t find a way to test how it behaves.
Does the necessary syntax exist to achieve this?
Once I calculate the slope of the regression, I plan to use it as the value factor risk for each stock.
Ah, I didn't realize the imported data series was monthly. You certainly don't want to use 21*CTR on it (that's to simulate monthly values using daily returns). The only ways around this I can think of are a) to convert your imported series to weekly and use Close_W or b) to somehow convert your returns to a monthly series using the Aggregate Series tool.
Or you could export everything and do it in Python or Excel . . .
To export stock returns, it seems I’d need to first export all the returns to Excel and then re-import each one as a stock factor. This process feels error-prone with alot of work.
Am I understanding correctly that I should copy the monthly data into the weeks between the months? If so, how do I ensure the monthly data aligns correctly?
I also recall a tool that allowed us to access stock returns to debug formulas, but I can’t seem to find it on the menu bar anymore. Does that tool still exist?
Thank you for your help! I’m happy to share the results with the community if it’s of interest.
I wanted to share some insights regarding the key point I’ve been emphasizing. Below, you'll find the screener using both an AI-based and traditional system to construct a long/short portfolio.
According to P123, the system demonstrates an annualized alpha of just under 10%, with an R² of approximately 0.19 and a beta of 0.3.
However, this portrayal is misleading.
When evaluated against common, well-known, and readily available risk factors using PortfolioVisualizer, the alpha turns out to be negative. The observed returns are primarily derived from a combination of betas, rather than genuine alpha.
This highlights the critical need for P123 to adjust its approach to stock returns, focusing on true alpha rather than relative or total returns. The current methodology risks leading us in the wrong direction.
Conclusion: Yes this factor gives some alpha (0.215200831 or 0.2% for weekly returns) with a beta of 1.041265085. High statistical significance for alpha (the intercept in the graph) and pretty high in magnitude for a weekly analysis..
Note: The hight R^2 is probably just correlation due to market movements and may not be too important. The alpha and beta probably being more pertiinent, I think.
Does Korr have a point about the usefulness of this method and can it be extended to commonly used P123 factors? Does this show the usefulness of this commonly used P123 factor as it generates some alpha? Fama and French factors not discussed here and no opinion expressed here on the usefulness of those factors.
So I decided to think about my own question above ("Does Korr have a point about the usefulness of this method and can it be extended to commonly used P123 factors?") Being relatively early in the morning and not wanting to express any controversial views myself I just asked Claude 3 what it would recommend to P123 users regarding any practical application of this:
"Ah, for practical application, I would suggest:
Use this regression analysis as one tool for evaluating potential factors for their strategies - if a factor shows statistically significant alpha with a reasonable beta (like EBITDAQ/EV does here), it could be worth considering.
However, they should:
Run the analysis over different time periods to check robustness
Consider how the factor performs in different market conditions
Look at the actual distribution of returns, not just the regression statistics
Consider what economic logic underlies the factor's performance
Think about whether the factor's performance might be related to other known factors
Test transaction costs and implementation feasibility
Think about how to combine multiple factors that each show alpha - individual factor analysis is just one piece of building a complete strategy.
The key is to use this as one piece of evidence in factor selection, not the only criterion. A factor showing statistical significance in this type of analysis is promising, but should be just the starting point for deeper investigation."
Can you please unpack this for me? Why would a combination of factor betas be worse than alpha? After all, your portfolio is built on exactly the factors that create factor betas. Are you searching for a source of alpha that lies well outside of or is uncorrelated to the factors that have been shown to work by many researchers?
I have long been a huge advocate of using alpha against the market as the best measurement of portfolio returns when they're long only. However, I am not convinced that using alpha against factor-loaded long-short portfolios is the best measure of a long-short return. Why? Because alpha is entirely unsuited to measuring anything with a short or hedge component. When used against a long-only benchmark, using alpha favors maximizing the hedge or short component no matter what. I have not tested using long-short portfolio alpha against long-short benchmarks, but because portfolio returns = alpha plus beta times benchmark returns, the lower the beta, the higher the alpha. By using alpha as your measure, you're then aiming for a portfolio that has a negative correlation to the long-short benchmark that you're using.
Personally, I want my portfolio to be correlated with a long-short value portfolio, especially if the value measures are good ones (not price-to-book), and I'd be very happy with a very high beta on that measure alone. I'd also be very happy with a very high beta on a long-short low-volatility portfolio. High betas in those cases means that my portfolio is working the way I want it to. High alphas in those cases would mean the opposite.
Thank you for pointing this out. That would be one of my concerns about using a Fama and French regression model. Price-to-book is really a noise factor at tis point and I wonder if it tells me anything at all. I would regress to something else if I were to use this. What exactly to regress to would be a difficult topic but was part of what prompted the above post.
I agree essentially. I get that I am not doing a Fama and French regression with a different factor above, but you could and what I have done is related (and maybe more useful even). At least as long as Fama and French models keep with price-to-book is my point of agreement.
BTW, if I were to pursue this one could use modern P123 features with factor analysis and regress to those "latent factors." It would not be hard to do this with factors one believes in (whomever is doing the factor analysis). You could decompose exposure of an ETF as Korr suggests with any set of features.
Here is one reason to discuss factor analysis above. As Claude 3 puts it: Yes, that's a fascinating aspect of factor analysis! Even if Yuval and you use different observable factors, you might be capturing the same underlying latent factors because….
In other words, there would be considerable agreement about the "latent factors" even if we used different features in independent studies
.
Jim
On beta vs. alpha:
If the goal is to mimic beta exposures, that’s completely valid, and measuring factor loadings confirms the portfolio is achieving those exposures. So, we'd want to measure these regardless. It’s similar to what Vanguard did for the mutual fund industry: offering low-cost, efficient exposure to market returns revolutionized how investors allocate their portfolios. If following betas is the goal, this is the right way to invest—focus on well-established factor exposures through cheap, tax-efficient products like ETFs.
That said, if our strategy ultimately replicates those same factor exposures, it raises a fundamental question: why go through the complexity and cost of crafting these models when proven, efficient solutions already exist? If I’ve missed your point here, I’d appreciate your clarification. Higher beta just means you can replicate the benchmark with leverage to equate the betas.
On the role of unique strategies:
To be clear, I’m not arguing that our strategies need to avoid factor-based approaches entirely. However, they also shouldn’t simply replicate what’s already “common.” P123’s value lies in helping uncover differentiated alpha or unique approaches—not duplicating what’s easily and cheaply available.
On alpha in long-short portfolios:
Alpha can represent both spread returns (returns derived from long-short differentials) and non-spread returns (returns uncorrelated with traditional factor exposures). These can be measured against either a long-only or long-short benchmark, depending on the portfolio's structure. My concern is ensuring we use the appropriate benchmark to measure alpha meaningfully. If I’ve misunderstood your point, could you clarify? I’d like to ensure I fully grasp your perspective here.
I hope this addresses your points, but if I’ve missed anything, please let me know—I’d value further discussion.
Try this. Overweight your short positions. Instead of 50/50, try 25 long and 75 short. I believe your alphas will go up across the board. In my own experience, if I compare a long-short strategy to a long-short benchmark, I can maximize alpha by maximizing my short positions, and I minimize my alpha by maximizing my long positions.
If I'm wrong, then I'm wrong, and you can just ignore me about this. But if I'm right, then it seems to me that alpha is really the wrong measure here. If I'm right, then the better your short strategy is, and the worse your long strategy is, then the higher your alpha will be. And that's really not what you want to be aiming for.
Thanks for the discussion—it seems my results align with your experience. When I adjusted the portfolio to 100% short and 30% long, PortfolioVisualizer showed higher alpha with greater weight on shorts, though the alpha (8.2%) was smaller than what P123 reported (21.86%). The beta shifted to -0.81 (PortfolioVisualizer) and -0.99 (P123).
So you are correct with that observation and intuition!
However, to clarify, shorting doesn’t mathematically increase alpha—it mathematically reduces beta. The increase in alpha comes from shorting individual stocks that drive idiosyncratic returns. For example, if you short the index (beta of 1) while going long an equal amount, the portfolio’s beta becomes zero, but no alpha is added because the index doesn’t generate any stock-specific inefficiencies. By contrast, shorting poorly performing individual stocks reduces beta and increases alpha when those stocks under-perform expectations, driving stronger spread returns.
These results suggest there’s more edge to be made shorting stocks than going long. If this aligns with the broader patterns seen in P123 designer models and strategies, it supports the need for a robust risk model to balance long and short exposures effectively, constructing beta-neutral, alpha-driven research. Ideally, we’d have a way to systematically design beta-neutral portfolios using a risk model to maximize alpha while controlling exposures. This would be the best way to properly research winning ways.
Thanks again for your insight—it’s been very helpful in refining this perspective. Do I understand your remark?
Reductions in beta correlate with increases in alpha and vice-versa. A few years ago I created a rather elaborate proof that as long as the market portfolio has a positive tendency (market returns are more likely to be positive than negative), then beta and alpha will be inversely correlated. You can read it here: https://backland.typepad.com/investigations/2018/06/why-low-beta-outperforms.html. So as you emphasize the short side of your portfolio, your alpha goes up in part because your beta is going down, not independently. This wouldn't be the case if the long-short portfolio you're measuring yours against were basically flat. But because it isn't, alpha and beta will have an inverse correlation.
So having a higher alpha due to an emphasis on shorts does not imply that shorting has a greater edge. Instead it implies that alpha is an inappropriate measure for a long-short portfolio.
Let's say you used two ranking systems for your portfolio, one for the long side and one for the short side. The ranking system for the long side has an excellent design and an excellent out of sample track record. The ranking system for the short side is half-assed and untried and basically consists of three factors; it really doesn't perform well at all. This combined long-short portfolio will still likely show a greater and greater alpha as you increase the emphasis on the short side because you're still going to be lowering your beta by doing so.
I love alpha. I use it every day and it's one of the most important tools in my toolbox. But I don't use it for hedged portfolios. I'd advise you to look for other tools for long-short portfolios too.
Wow—that’s impressive! I’ll admit I’m not yet in a position to fully verify the math or conclusions, but I do agree with your point that other measures like the Sharpe ratio or information ratio become far more relevant if you are hedged to the market. And questions of choosing the appropriate benchmark are pertinent.
That said, I have a few follow-up questions that I hope you can help me reconcile:
In my experiment, when I reduced beta relative to the benchmark, my alpha actually went down, not up, as did my beta. Why would that happen?
You seem to conclude that in down markets, high-beta stocks will show alpha. Have you empirically tested this? While it could be true, it feels counterintuitive to me—perhaps I’m misunderstanding your point?
What do you consider alpha in practical terms, and what tools or metrics would you recommend I (or others) use in our toolbox?
Thanks again for the insights and for sharing your work—it’s given me a lot to think about.
While alpha is negatively correlated with beta, that doesn't mean that every time one goes down the other goes up. It just means that it's more likely to work that way.
I just ran a screen on the Easy-to-Trade universe with one rule: Frank("Beta3Y") > 75. Backtested on the dates 10/1/2007 - 3/1/2009, the portfolio has an alpha of 16.46%. Running the same screen with Frank("Beta3Y") < 25, the portfolio has an alpha of -1.09%.
Mathematically, alpha is the return of the portfolio when the market return is 0. In practical terms, it measures outperformance compared to the market. But I would say that it's really only useful as a measure in a mixed or upward market and when you're going long only. I prefer to use a combination of the following measures when measuring the results of a hedged portfolio: CAGR, median quarterly or yearly returns, and CVAR at 2%, 5%, 10%, and 15%. CVAR is the average of all monthly or quarterly returns below the Xth percentile and is a purely risk-based measure. The Sharpe ratio presents a lot of problems for me, so I don't use it.
