AI Factor - Recreating Linear Model Predictions

[Jrinne]

This this topic was discussed previously and I wondered if it answered everyone's questions or not: No need for raw scores - #4 by marco

@marco

The way around it at the moment is to use the same “ML Training End” date. This option appears when you choose “Entire Dataset”.

The problem with our current solution is that you will need to download more and more data for predictions. For example if your model was trained using 10 years of data and 4 years of validation, when you do predictions a week later you have to download 14 years + 1 week and use the exact same “ML Training End”. From this download you only use the new, single date, that has been normalized using the same stats as the original training.

It’s very inefficient and costly. We planned to store the mean and std for re-use but wanted to hear some feedback first.

I don't see a post giving the final decision on some of the questions.

I do think it is clear that the ML predictions for the target will be z-score predictions as long as the target normalization is set to z-score (whether normalized for data or entire dataset).The ML is only able to predict a target that it is trained on. When using entire dataset normalization, those predictions could be transformed into raw returns (based on the mean and standard deviation of the training data) but P123 does not do that automatically now. And it cannot do it for some target data as P123 is not allowed to give us raw data for much of FactSet's data. P123 could do it for returns (and some other pricing data).

The only real question, that I can see, is whether for live ports the predictions are online. Meaning whether they are restrained on new data that comes in after each rebalance. Marco gives a clear discussion of this option in his post—including the cost involved.

I am pretty sure that if there were retraining of the models going on after each rebalance we would see it in our AI/ML charges. We might be able to make a feature request for automatic retraining after a set period. Say every X number of weeks, months or quarters (retaining would probably have a charge).

Maybe that could be considered as a feature request after some discussion. Feature request: option of automatic online training of AI/ML models.

Maybe P123 can clarify, expand upon or verify some of this for me and other members. And correct me where I am wrong.

P123 is working on integrating an advanced LLM that may have full access to the code. Even members with a rich understanding of machine learning can miss things in the menus (like the option of normalizing features by date), if nothing else. There is also the potential that the LLM could calculate things like predicted raw returns when asked (whether there is a menu option for that or not).