Am I using future information?

[vagrantlee]

If I am using TTM sale growth rate to rank stocks, is the performance result using future information? In reality, TTM sale growth rate is only known after a few months when company release earning, but stocks can be purchased based on this information in backtest.

How can I prevent a ranking from using future information?

Thanks!

[test_user]

TTM calculation are based only on already-reported data, so there should be no future information involved. In general the functions on Portfolio123 are set up so that they are "safe" to use in ranking. There's only a few exceptions I can think of, like Future%Chg or using negative time in FHist().

[vagrantlee]

Thanks for your response. I am a new user in Portfolio123, please bear with me.

Just try to make my concern clear, I will ilustrate using an example. In my example, there is a company A which always reports its earning one month after a financial quater ends. Dec 31 is the end of a financial year, company A gives its anual report on Feb 1 after market next year. One specific year, company A earning beat a lot, and stock price jumped a lot on Feb 2.

In reality, I can only know the good news on Feb 1 after market, may decide to purchase the stock on Feb 2 after stock price has risen a lot. In back testing, if this report time is not considered carefully, my strategy may allow me to see the earning TTM rose, purchase the stock at much cheaper price from Jan 1 to Feb 1 time period. A single day error could make a huge difference.

The result could give me false hope that following earning result to buy will give me great result, but the backtesting could accidentally use future info that we didn't have.

If Portfolio123 already consider this detail, it would be great. Just try to confirm that is the case.

Thanks a lot!

[test_user]

No problem - It's an important issue! This data quality issue is called "Point in Time" (PIT).
The short answer is that the data is more than good enough to build strategies on (IMO). The longer answer is complicated. Start by reading:

Our Financial Data Providers and Approach to Historical Data Accuracy - Help...

The topic has been discussed at length on the forum, here are a few relevant threads:

Fundemental Data II

No, FactSet does not provide point-in-time data. Compustat kept effective dates for all changes that they made. So if a number changed from 135 to 160 on November 3, 2015, we knew that prior to November 3, 2015 it was 135 and after that it was 160. FactSet does not give us effective dates for changes and they do not inform us of those changes. Moreover, they do not give us the dates on which they made their data available to subscribers. So if a company announced on November 3, 2015 but FactSet…

And this:

Does the PIT method account for FactSet delay in updating for earnings reports?

Generally, the news date we give for dates prior to July 2020 is one day after the date FactSet gives for the earnings report, though there are a few exceptions. If FactSet doesn’t give us a date for the earnings report (which isn’t very often), we use a complex checklist to determine a reasonable news date for PIT data. In July 2020, when we switched from Compustat to FactSet, we started keeping track of when FactSet actually gave us the data, and we use those dates. I’ve been observing thi…

And (in the same thread):

Does the PIT method account for FactSet delay in updating for earnings reports?

Regarding look-ahead bias, do we throw the baby out with the bathwater? As I argued earlier, some look-ahead bias is absolutely inevitable simply because FactSet didn’t have the data it now has and Portfolio123 wasn’t always in existence. But we have gone to tremendous lengths to eliminate it as much as possible. Show me another backtesting engine that has less look-ahead bias than we have. Information about the FactSet estimate data that we’re using is here: https://www.factset.com/marketplace…

There are a few ways to stress test strategies if you're worried about this. For example forcing a simulation to only buy stocks that are N weeks after earnings release, or requiring stocks to have had a high ranking system score for multiple weeks in a row before they are bought.

[ZGWZ]

Based on the out-of-sample performance of many designer models, the foresight bias does not appear to be very severe. More important to note is overfitting.

In fact, it would be extremely easy to have extremely high returns in backtesting if there was a serious such forward-looking bias, but that didn't actually happen.

[ZGWZ]

@vagrantlee

How can I prevent a ranking from using future information?

Don't put any minus number in the offset parts of functions.

[yuvaltaylor]

Another way is to use a rule in your backtesting universe like latestactualdays > 5 or dayssince(latestnewsdate) > 5. This weeds out any occasions prior to 2020 when the date that the data appears on Portfolio123 would have been earlier than the date that FactSet released the data.

At any rate, to keep things short, the effective date of all data is always post-release. That's why there's no serious lookahead bias on Portfolio123. So in the hypothetical scenario you worry about, the effective date of the earnings release will be February 2. No simulation or screen with an as-of-date prior to February 2 will show those results. Instead they'll show the previous TTM results.