I am still unable to run sims on the new engine - I keep getting Invalid uid in PriceHistory. Anyone else experiencing this?
Daniel, we’ll investigate
philjoe, the engine is our ratio & backtest engine which runs your screens, ranks, backtests, etc.
Nicoletta, what are the settings for you r two runs ? is it Factset vs Compustat ?
dnevin123, issue with CompleteStmt should be ok now
Thanks
Hugh,
WeeksIntoQ, WeeksToQ, and WeeksToY now use additional dates (i.e. news date, filing date, and final date), taking the earliest date available to compute the result. Some issues were recently addressed causing some unwarranted differences beyond that. Apologies for the delayed response here.
I’ll have to check into MScoreSGAI tomorrow. Thank you for your diligence with these differences.
My simulations are not working on either a daily or weekly rebalance basis under the “current” system, but do work in “legacy” mode
Hi Aaron and Marco,
Aaron’s explanation about the new WeeksIntoQ, WeeksToQ, and WeeksToY is helpful to understanding the issues involved: The new WeeksIntoQ, WeeksToQ, and WeeksToY variables measure something different than the old WeeksIntoQ, WeeksToQ, and WeeksToY.
It’s possible to make a case for both. For example, I use the legacy WeeksIntoQ, WeeksToQ, and WeeksToY at times to try to avoid holding certain positions through earnings reports. It’s not perfect in that regard but the best proxy I could find. Losing the legacy WeeksIntoQ, WeeksToQ, and WeeksToY has had severe effect on some of my models.
As described, the new WeeksIntoQ, WeeksToQ, and WeeksToY may be useful at judging the freshness of a company’s financial data. Now that I know what these variables mean, I’m going to explore their usefulness. In the meantime they are causing problems.
BOTH versions (legacy and current) of WeeksIntoQ, WeeksToQ, and WeeksToY could be useful. I STRONGLY recommend that p123 make them BOTH available in the new engine.
(Please forgive the capitalization that follows but we have been through this a number of times…) WHEN P123 MAKES BEHIND-THE-SCENES CHANGES TO DATA POINTS, VARIABLES, CALCULATIONS AND FORMULAS WITHOUT INFORMING SUBSCRIBERS AHEAD OF TIME ABOUT WHAT IT IS DOING OR WHAT IT HAS DONE, IT CAUSES A GREAT WASTE OF TIME. Many subscribers either are already or aspire to be serious traders. Real money is on the line. PLEASE DON’T MAKE THESE CHANGES, HOWEVER WELL-INTENTIONED OR CORRECT, WITHOUT LETTING US KNOW!!!
Along these lines… WHAT OTHER CHANGES TO DATA, VARIABLES, CALCULATIONS, FORMULAS AND/OR WHATEVER ELSE MIGHT BE OF CONSEQUENCE TO SUBSCRIBERS HAS P123 MADE DURING THE TRANSITION FROM THE COMPUSTAT LEGACY TO THE NEW COMPUSTAT (CURRENT/BETA) ENGINE?
I can’t see how any subscriber can begin to measure the consequences of transitioning to FactSet data without working through these engine-related issues first. Once the transition to FactSet has been finalized it is going to be near impossible to trace these behind-the-scene changes that may have nothing to do with FactSet itself or its data but will still effect everything involved with FactSet and its data.
Hugh
p.s. I believe the new engine is having issues handling “Ind” variables such as Inst%OwnInd I mentioned this before but don’t believe a trouble ticket has been created. Before I go looking for more examples, has p123 made any engine or data-related changes in this regard?
Thank you.
Hugh
Hey guys, I took a couple of minutes this evening to compare the Compustat Legacy and Compustat Current engine behavior with respect to the Core Combination Ranking System.
When you run the ranks for today on the P123 Large Cap Universe at first everything seems basically the same, but when you dig in you start to see some rather large differences which appear to driven by the Final Stmt flag. First thing to notice is that the Final Stmt Flag is set to “Y” for all tickers on the Compustat Current engine, while 65 of the 388 tickers have a “N” value for Final Stmt on the Compustat Legacy engine.
This difference is not merely cosmetic as the tickers which have a “N” value on the Compustat Legacy engine have significantly different Rank Positions (RankPos) on the Current engine. On average the absolute RankPos difference between tickers on the two engines is 1.8, but if the ticker has a “N” on the Legacy engine then its average absolute RankPos difference is 4.5.
Is the lack of Preliminary Stmts on the Compustat Current engine by design? From my limited perspective, it looks like the Current engine is failing to incorporate Preliminary Stmt data like the Legacy engine did. Could this potentially explain some of the drop-off in performance many of us are seeing in our Strategies when we run on the Compustat Current engine vs the Compustat Legacy engine?
Thanks,
Daniel
Daniel , CompleteStmt will be fixed later today. Should not affect RankPos though, it’s purely a display issue. THere are other fixes that will be released that like fix the differences.
Hugh, lets discuss WeeksTo/Into in a separate thread. Your use case is why they were added: to avoid ( or focus ) on companies about to announce. It was purposefully made fuzzier by using weeks instead of days since we do not have exact announcement dates in the future. But now they are more precise in the past (albeit still +/- week). You could add a separate factor DaysTo that is more precise and leave WeeksTo as before. Lastly, I’m assuming your past results are being affected with the greater precision. That could be a sign of trouble for the strategy itself too.
C_Ibex, still investigating daily sims
The 2 graphs shown are just a re-run done yesterday against the saved graph I had saved on 02/16/2020, So is Compusat Lagacy against current. I here attach the same Sim with the four possibility. Unbelievable differencies…
Fabio (Nicoletta)
Marco,
Thanks for the response. I am curious about the comment that the FinalStmt flag difference is purely cosmetic. If that is the case, why I am seeing such a difference in RankPos for the companies which have an “N” for FinalStmt in the Legacy engine. Note that this discrepancy is even more pronounced if we look at universes other than the P123 Large Cap Universe. For the P123 Micro Cap universe the average difference of RankPos across all securities is 9.9, but if I focus on just the securities flagged with an “N” on the Legacy engine the average difference in RankPos is 28.4.
This definitely seems more than cosmetic. There appears to be a fundamental difference in the way the “non-Final” Stmt data is being handled between the two Compustat engines.
b - Added the stuff below[/b]
In the interest of comity, I went ahead and attached the spreadsheet I was using to do this analysis. Might be a bit difficult to follow at first, but if you focus on Rows 2-5 of the “P123LC - Compustat Delta” tab I think it might be helpful. Note that Row 5 calculates the Absolute Average difference in the Rank assigned to a particular Stock in both engines for each of the Factors used in the Core: Combination Ranking System. The higher the number the worse the difference. (I also color scaled the cells, so the redder the cell the worse the difference.) Row 4 calculates the same Average Absolute difference in Rank for just the Stocks flagged with an “N”. Rows 2&3 should be easy to figure out from there. Based on this quick analysis it looks like the major differences between the Current and Legacy Compustat engines come from the following Core:Combination Factors in descending order of severity.
Unlevered Free Cash Flow to EV
EBITDA/EV
Free Cash Flow to Price
EPS Acceleration - recent
Gross Profit to EV
EV to Sales
Pr2SalesTTM
EPS%ChgTTM
Surprise%Q1
IntCovTTM
Surprise%Q2
Excess Gross Margin
PR52W%ChgInd
EPS5YCGr%
EPS Acceleration - longer term
Interest Coverage (5-year)
Pr2TanBkQ
You can also see from comparing Row 4 to Row 5 that the differences in Absolute Rank for these Factors are significantly worse for the Stocks with the “N” FinalStmt flag.
I’m sure you guys have seen all this yourselves, but I figured I would send it across in case it might help.
Thanks,
Daniel
I haven’t been on in a couple of weeks. But i notice that a lot of my sims are showing different results now. Why may that be?
Is it because of the switch to different provider? Or some other change? Please update
Hugh -
Here is a list of all the changes that we have made between the “Legacy” and the “Current” versions of the database. This list does not include the bugs listed on the Trello board. These are deliberate changes.
Shares now returns NA when “Shares Outstanding” is unavailable instead of returning a value from SharesBasic or SharesFD.
Skip preliminary fallback if numerator is present and denominator is invalid (0 or negative) in preliminary: DbtS2NIQ, DbtS2NITTM, GMgn%, NPMgn%, PMgn5YCGr%, TxRate%.
AnnounceDaysPYQ, AnnounceDaysQ, WeeksIntoQ, WeeksToQ, WeeksToY use additional dates.
Div%ChgA, Div3YCGr%, and Div5YCGr% are -100 instead of NA or 0 when going from positive to zero.
Preliminary fallbacks have been revised for these factors: EV2EBITDA, ValROETTM, Pr2BookPQ.
Anything which is still different and is not mentioned above should be reported to us and considered an issue that we need to address. Once again, see the Trello board for issues that we are currently addressing: Trello
Now there are 3 threads about the change so I’m not sure where to post this I was looking at RY:CN and I noticed some big differences in Items. I see on Trello that P/CashFl and EV are having issues … is this the same reason for these differences. There also seems to be a issue with cash reported? Here are a couple screen captures between Compustat and Factset from the snapshot screen. I know that it’s a bank and the whole relevance about EV but none the less I would expect these values to be similar.
I am getting super slow responses from the server, anyone else?
It’s a bank. That’s the long and short of it. There are currently huge differences between how Compustat and FactSet deal with banks. We’re working on how best to deal with it and will keep you updated.
- Yuval
Hi Yuval,
Thank you very much for your detailed reply concerning changes made between the Legacy and Beta versions of the database. I appreciate it.
With the possible exception of the change regarding “Shares Outstanding,” which I think deserves further investigation, it seems clear that the greatest consequence of the recent changes you list involves WeeksIntoQ WeeksToQ, WeeksToY, AnnounceDaysPYQ and AnnounceDaysQ. Changes to the other values you mentioned do appear to be corrections.
Regarding WeeksIntoQ & Company, I would like to avoid arguing that the new means of calculating these variables is incorrect because I think it’s so clear that the new means measure something different than the old means, and both have value.
To assume that starting the clock in all cases from a preliminary announcement does make sense from a certain perspective but probably overlooks how little information is often communicated in preliminary announcements vs. the far more complete and standardized quarterly reports which can be studied and relied upon, in part, because there are so many more cases (4 times/year for the life of the database for thousands of companies). That’s why I don’t agree that the new variables are more precise. They are more precise to the new definition of what they are measuring. But the old variables are more precise to the old definition of what they were measuring. Believe me, I look forward to testing and hopefully integrating the new variables. I just don’t want to lose the old.
I would therefore strongly encourage p123 to keep the old variables available as is and include the new variables using different names. The more potentially useful data the better!
FORTY PERCENT of the values of WeeksIntoQ in a small sample I studied in order to compare Legacy vs. Beta versions of this variable were different. FORTY PERCENT!!
it’s a lot, I believe, to ask subscribers to assume their simulations, screens, formulas, and designer models, built over a long period of time, should unilaterally move to the new method.
Thank you.
Hugh
Hugh -
Try this experiment. Open a screen and put in as your three rules WeeksIntoQ, LatestActualDays, and ShowVar(@Weeks,Trunc(LatestActualDays/7)). Run it on a date in the past–say five years ago.
If you use the Legacy version of our website, there will be lots of differences between WeeksIntoQ and @Weeks. Every one of those differences is due to an announcement date that is different from the latest actual date.
Now run it on the current version of our website with Compustat using prelims. You’ll still see a few differences, though not many.
Now run it on the current version of our website with FactSet using prelims or not. You’ll see ABSOLUTELY NO DIFFERENCES. Why?
Because the data is not there. FactSet does not give us these dates.
That is why we had to change our engine, and that is why the old way of doing things is not going to be available.
Hi Yuval,
Thank you very much for your message.
I attempted your experiment with confusing results
First, I could not get “trunc” to work. That is a relatively minor point because it is so easy to compensate for this but could you please confirm that your language above is correct?
Second, OF GREAT CONCERN, is that running the screen on Legacy, Current with CompuStat w/prelim and FactSet w/prelim using today’s date produced three different stock lists.
Among the first 25 stocks listed, Current CompuStat w/prelim added tickers ABG and ACCO to what Legacy was showing.
Among the first 25 stocks listed, FactSet w/prelim failed to include ABR, ACAZ, ACB, ACEL, ACGL and ACHC, all of which were included in the Legacy list.
I don’t believe there should be any differences between Legacy and Current CompuStat w/prelim. Certainly we would not expect as many differences between Legacy and FactSet w/prelimn.
- Unless - and this is very possible - I am misunderstanding your instructions, I was not able to replicate the “Absolutely No Differences” you suggest.
Help!
Thanks.
Hugh
Hugh,
I’m not sure why this didn’t work for you. I was trying to demonstrate that we don’t have certain data points in FactSet and that the data that we offered on the legacy server was incorrectly handled. Let’s try a different approach.
We receive four dates from Compustat: the filing date, the press release date, the effective preliminary date, and the effective final date. The effective dates reflect when Compustat processed the data. They are data-vendor specific. We are not receiving effective dates from FactSet.
WeeksIntoQ used to be based on the effective preliminary date. The documentation, however, for WeeksIntoQ says: “Number of weeks into the most recent quarter. A value of 0 indicates the last earnings report was less than a week ago. When it reaches values > 11 an earnings report should be announced soon.” That’s not what WeeksIntoQ was giving you. WeeksIntoQ was actually giving you the number of weeks since Compustat processed the announcement. So the Legacy server was giving you a number that was unsupported by the documentation.
The new server gives you the actual weeks since the announcement.
There’s another entirely different database based on estimates, which is where LatestActualDays comes from. LatestActualDays is “Calendar days since analysts actuals were published for the most recent quarter.” That should match the press release date because analysts always update their EPS actuals on the date of the press release. So using LatestActualDays is a good way to check that WeeksIntoQ is being reported correctly. LatestActualDays is based on Compustat estimate information because we haven’t yet finished loading the FactSet estimate data.
To sum it up in the fewest possible words: WeeksIntoQ used to be wrongly calculated and reflected when Compustat was giving us the information rather than when the information was released. In the current version it has been fixed. And we are not getting any “effective” dates from FactSet prior to March 2020.
Thank you very much, Yuval.
First, please do run your experiment on each of the Legacy, Current w/prelim and on FactSet w/prelim engines to make sure you are generating the same list of stocks. If not, we may have stumbled on an issue, yes?
Second, just to be clear… If stock ZZZZZ announces preliminary information on February 15th, 2020, files on May 1st, 2020, and S&P took a week in each case to process, its dates would be…
Press Release Date: February 15th, 2020
Effective Preliminary Date: February 22nd, 2020
Filing Date: May 1st, 2020
Effective Filing Date: May 8th, 2020
…correct?
You wrote above that WeeksIntoQ used to be based on the Effective Preliminary Date. I don’t think that is so. I think it was based on the Effective Filing Date. Are you sure it was based on the Effective Preliminary Date?
Third, I would discourage any subscriber from relying on LatestActualDays unless it turns out that FactSet is doing a much better job with this data point. Based on previous investigations, I’d say that this variable is theoretically perfect, but far from being so in practice. There are two issues, I believe. The first - and I admit I’m speculating based on my exploration - is that prior year values were not really point in time. The second is that analysts do NOT always update their EPS actuals on the date of the press release. They usually update their EPS actuals AS OF the date of the press release, but often at a later date. And sometimes they screw up and don’t update or their updates are not captured. This is especially so when analysts drop coverage.
Fourth, regarding FactSet, if the data we will have to work with is a) the press release date and b) the filing date, I strongly encourage p123 to create separate WeeksIntoQ variables based on each. WeeksIntoQ based on the press release date will allow subscribers to judge most accurately how early a company first presented information about a quarter. WeeksIntoQ based on the filing date will allow subscribers to judge where the company currently stands in relation to its last quarterly filing. DaysIntoQ would probably be better than WeeksIntoQ, by the way, since WeeksIntoQ can be derived from DaysIntoQ but not the other way around. If companies always released the same information on the press release and filing dates, perhaps the two data points would be redundant. But companies don’t. As mentioned before, I think it’s an unnecessary argument to debate which of the two dates is more useful. They each provide potential incremental value.
Thank you very much for getting into the weeds with this. The CompuStat/FactSet transition period is very valuable, I think, however painful.
Hugh