There is often a question in the forum as to whether a live port and a sim--over the same period--will perform the same or not. The possibility of FactSet's revisions making the data not perfectly PIT being one concern.
Port on Automatic Rebalance first:
Sim with same settings:
Conclusion: For my ranking system the port and sim--over the same period--are nearly identical. In fact, the sim outperformed the port by a trivial amount (literally a rounding error for CAGR).
More generally, I am beginning to think that with the new advanced slippage method and a holdout test sample, it might be possible to get a good idea of what the out-of-sample results will be—within a relatively small margin of error. Here is Marco's post on the advanced slippage option for sims (not used in the above sim): Custom slippage feature - #2 by marco