We’ve been over this in a few other threads, but essentially P123’s data is not completely stationary. We do our best to make it point-in-time, but tiny changes do creep in. The two primary reasons are: a) we’re always fixing bugs; b) FactSet makes some data corrections/changes, both to analyst and fundamental data. Other sources of differences could be user-caused: a change in universe rules, a change in a custom formula, a change in a ranking system. That’s clearly not the case here, but I did want to point it out. When you’re ranking a few stocks out of several thousand, these changes can have a “butterfly effect,” which tends to be rather random in nature. The fact that the performance differences are worse for the live strategy may be coincidental. Again, this was the subject of an earlier thread, and there the performance differences were positive in some cases and negative in others.
If you really want to get at the heart of what’s causing these differences, I suggest the following. On the rebalance date, run the “ranks” function of your ranking system on your universe. It should match the simulation’s recommendations, adjusting for buy and sell rules. Make sure to include composite and factor ranks. Download the CSV file and save it.To be really careful, also run your universe and download and save that. A month later, run the simulation and note down differences. Then run the ranking system and universe again with the as-of-date that matches the rebalancing in the simulation. Download the CSV files and see how they differ from the ones that you downloaded a month ago. That will pinpoint exactly what changed for each stock in question.