In Ranking no matter which day of the week the specified start date is, the ranking is performed on the previous Mon. with a rank calculation from the previous Sat. so the are really only up to 52 distinct ranks done each year. That does not match up with the claim of a point-in-time database. It also seems
that biases are introduced/ignored such as Jan effect and possibly Day of Week. If there are
huge 2-day swings in the market such as occurred in 2008 they are picked up if they happen Fri. - Mon., but not otherwise. I can specify a start date in the first week of Jan and if I don’t know
which day of the week it is, the ranking may reach back into Dec. up to 4 trading days which plays into end of year selling biases and Jan. buying biases. A multi-year test will certainly have this problem.
I am researching Bob Haugen’s expected return factor model. I know I can’t replicate the model because P123 does not have multiple regression, but I can use his top 10 list of salient factors.
He starts out with a simple linear regression performed each month over the training period so I
hoped I could perform rankings from month start to month end.
I was hoping I could additionally generate many, many random start - end date pairs and subject a set of ranking systems to the pairs.
I don’t think it works like you think it does. For simulated backtests, ranking is preformed on the weekend after the market closes. It already knows what stocks it wants to recommend on Monday. I don’t see how any bias is introduced here. There is no look-ahead bias.
In live trading, the ranking systems are updated daily.
Can you please explain what bias you think is introduced? I don’t follow your logic.
Keep in mind that fundamental variables are updated weekly in simulations. So assume that on Wednesday April 12th earnings are announced. If you are running a backtest, this won’t be available for ranking until the following weekend of 15th or 16th and you can trade this on the following Monday the 17th. So it is the opposite of look-ahead bias. It is lagged if running simulations (although in live trading you can rank daily).
Also, if you run daily backtests…the price data is updated daily. Just not the fundamental of the equation. If you use daily backtests (which this isn’t really the right platform if you need to hold positions for a couple days as it is end of day data), put your price rules as rules and not in the ranking system.
But again, if you are trading 1 or 2 day mean reversion - you need some intra-day platform. This is not the right fit for what you are looking for.
there are really only up to 52 distinct ranks done each year. That does not match up with the claim of a point-in-time database.
Point in time is for Monday rebalances only (such as 1, 2, 3, 4, 6 8, 13, or 52 weeks, etc.); not daily, monthly, or annually.
He starts out with a simple linear regression performed each month over the training period so I
hoped I could perform rankings from month start to month end.
Yes, it’s a shame that it’s extremely difficult to implement a factor momentum based model.
Partially that’s because Yuval and other people who tried to do it didn’t have as much success with it as they had hoped.
I was hoping I could additionally generate many, many random start - end date pairs and subject a set of ranking systems to the pairs.
You wouldn’t achieve your goal with this. That’s because for most factors, the rankings remain relatively stable from one week to the next. Those many start and end dates would not improve the sample size as much as you might think.
You mentioned monthly backtests. You can definitely do a four week rebalance from four consecutive weeks start dates ad get 4x as much data. It will overlapping however.
I think your expectations for PIT are too stringent.
PIT databases are designed to prevent future events from affected a simulation. So if a company is forced to restate earnings, that event would not over-write the prior recorded earnings statement. Done properly, a simulation would not be privy to any future events.
PIT has no guarantees on data latency. If a Monday ranking uses data from the prior week, the DB is still PIT.
P123 should provide a whitepaper on PIT, ranking and when DB updates occur.