Industry factors rollback

Announcements
1

I just wanted to start a separate, focused thread on this topic since it has caused a lot of grief.

It’s a long post, so here’s the conclusion: we’re rolling back the Industry (Ind for short) factors to what they were before (with all their flaws) because what we did introduced new flaws. We will create what we think are correct Ind factors later. The current Ind factors will continue to exist even after we release new ones (but will be flagged as deprecated).

Also, both the new version of P123 and the “alt” site will have the same Ind factors. Therefore the only difference in the “alt” site will be the exclusion of holidays in the time series.

We will update you tomorrow, Saturday, when it’s all done.

Now the gory details.

A little background first about the original version. We provide Ind factors which are convenient to use and fast since they are pre-computed. It’s not a complete list by any means but they cover the major ones like Price2SalesTTMInd, Pr4W%ChgInd, etc, about 90 I think.

For each Ind factor there are two values one for USA and one for Canada. You can recreate the values that are pre-computed using the Aggregate function, for example:

For: PEExclXorTTMInd
Use: Aggregate(“PEExclXorTTM”,#Industry, #Avg)

Ind factors are only pre-computed for one universe. For the US it’s the All fundamentals Incl F-Shares, For CAN it’s All fundamentals incl. dual listed.
Also the Aggregate function by default trims 16.5% of the outliers and excludes ADRs.

There are many “flaws” in the way Ind factors are being calculated:
[list=1]
[]Only ADRs were being excluded, but many foreign stocks (500+) are still used that are not ADRs, like Ferrari, UBS group, Deutche Bank. So it’s neither here, nor there.
[]The USA Ind factors include thousands of Canadian “F” shares, but many Canadian stocks are not included . So it’s more like a North America Ind factor; but not quite.
[]Many of the Canadian industries are completely taken over by one or two stocks.
[]The Ind factors project was never completed. Where are the Sec factors, or SubInd factors?
[/list=1]
We wanted to correct the Ind factors in the new version but it was rushed. To do it the “right” way, the way we intended, we need full access to global data which we do not yet have. This would mean that the Ind factors would go through yet another revision once we get global data.

It is pointless to go into more details of what we wanted to do. It was announced in the original post, and there will likely be tweaks once we do a proper new Ind factor release. So for now we’ll rollback to the original Ind factors.

Let me know your thoughts. Sorry again for the inconvenience.

2

Thank you, Marco, for explaining the motivations for updating the *Ind factors. It’s much easier to accept this change once the shortcomings of the original design is understood. I wish this information was included in the original change document.

I’m still a bit confused on the Taxonomy series price/liquidity rules. Will the new rules still be deployed? If not, what are the legacy rules? The legacy *Ind rules you listed are only scope related.

Walter

PS Since the *Ind factors are pre-computed, they’re determined independently of the current Universe, right? That’s something I hadn’t considered before. I think I may be using Aggregate() more in the future.

3

Thanks, I guess this is a good idea, though I really preferred the new solution. This is probably way too much work, but I have to ask: Is there any way you could keep the new and old solution side by side? (especially the new Pr4W%ChgInd + 13/26/52 versions).

4

I also think each upgrading/change should keep 100% backward compatibility and indicate to the existing user those old one is going to be deprecated in the future, even for some important commonly used factors we can keep them simultaneously so to reduce the impact to existing users.

5

Walter -

I want to apologize for the fact that I did not communicate the shortcomings of the original design. I also want to apologize for the confusion around the taxonomy series.

Despite what I wrote earlier, the taxonomy series are not being used to determine any values that we use in factors or functions. They are only used in charting and on the Industries & Sector pages (https://www.portfolio123.com/app/taxonomy).

Yes, the -Ind factors are pre-computed, as Marco explained, based on specific universes, not on the universe that you’re using. So using your own Aggregate() will give you somewhat different results than using the -Ind factors, depending on how different your universe is from All Fundamentals.

6

Thank you, Yuval. I very much appreciate the clarifications. Funny, I use the taxonomy (aka Industries & Sectors) page a lot and yet didn’t associate it w/ the price/liquidity changes. My bad.

I’ll have more questions about the pre-computed *Ind factors once I understand how multi-country will work.

Walter