In today's release, we changed the default setting for "PIT Method - Prelim" from "Use" to the more conservative "Exclude" setting. The settings in your existing work will not be changed, but the Exclude setting will be the default when you create new Live and Simulated Strategies, Ranking system (performance and ranks) and Screens. This also applies to any API endpoints that accept pitMethod as a parameter and any DataMiner scripts that use those API endpoints.
3 Likes
Dan,
Thanks for the update! Can you provide some background on why this change was made? Have you seen problems with the Prelim Data?
Cheers,
Daniel
6 Likes
Interesting that question that did @dnevin123 . Any update?
1 Like
Sorry for the delay. It's all about our new philosophy to help users create more realistic backtests with less chances to curve-fit or use forward looking values.
Using prelim data with Factset can expose certain factors from filings that were not published in the prelim/press release. In technical terms: the Factset historical filing database we have has exactly one version the "as-reported" filing , and one version of a restatement (if any). What Factset does is fill in the as-reported version with as much data from the prelim, then fill/update when final is filed by the company.
By surfacing data only on the filing date you are not using items that might have not been published on the press release date. Compustat is better in how they handle preliminary, but we still think it's wiser, as a default, to avoid preliminary data. You also avoid "frankestein ranks" where you mix factors that fallback and factors that come from press releases.
In addition, both vendors suffer from an artificial delay they introduce in processing the filing, which further muddies the concept of Point in Time since it's all relative. Compustat data is PIT as far as when they deliver it. Anyone that is downloading SEC filings instantly when they appear, and has this amazing AI powered analyst that standardizes the data, will always be ahead of all data vendors.
For the reasons above we fell that being conservating in the right choice for us. In fact we want to introduce more options like delaying the data 15 days or more. Most academic papers for example delay the Annual data 6 months, and do not use interim data.
NOTE: for what it's worth, while developing our Risk Model we initially used prelim data by mistake. Then we recalculated it using final data only and the differences were statistically insignificant.
3 Likes
Thanks for the insight. I really appreciate it
Hello,
I am not sure that I understand this well. Could you share an example for a company release?
Thanks in advance
What about live trading ?
Using prelim or not ?
Live trading is different. With "Use Prelim" enabled you will get all the latest data, and incomplete prelim statements. Typically the missing factors will be the ones less likely to be reported in a press release, like cashflow items and balance sheet items.
This means that some of your factors will fallback, others will not. If you use one of our built-in complex factors (made up of multiple line items) we try to keep things consistent as much as possible. But if you write your own, long factor formulas some of the components might fallback, others not.
It all probably works out in the end for live trading. What we could perhaps add is a statistic like "MissingItemPct" that tells you the percentage of items that are NA. With this you could , for example, have a buy rule that rejects a stock like this:
CompleteStmt=FALSE and MissingItemPct > 30
This would reject a top ranked stocks if it has 30% or more NA's in the latest filing which are flagged as incomplete. I suppose this new factor could also be valuable for complete statements.
4 Likes