One of the main motivations was because it was impossible to recreate the results of an AI Factor validation Long/Short portfolios with a screen backtest. The reason is that there was no support for placing NA's in the middle, or rank=50, in the settings. If any stock had an N/A prediction the results would differ (we return an N/A prediction when more than 30% of the features are N/A no matter what)
Below is an example of the new interface with the proper settings for Quick Rank when using an AI factor
Ever since this update (which may be unrelated) my backtested screen performances have dropped dramatically - CAGRs going down by 10-20%. All of my screens rely heavily on FRank. Could the default way FRank works have changed? Any other possible changes that maybe causing this?
I thought maybe I'd done something to a screen without noticing, but I'm seeing the performance decline on all screens.
Edit: Just to confirm, I'm not referring to screens using quick rank, nor AI screens. Feel free to take a look at "SUE Screen" "AR Screen" or
"Small Cap SY Alternative Ranking" as three examples of screens that have had dramatic performance drops since this update.
Looks like the new engine is ignoring the setting in Method to "Use Ranking System Default" which for your ranking system is set to "Percentile NA Neutral". It uses the default "percentile NAs Negative" instead.
As a workaround for now set the Method to "Percentile NA Neutral" to force it, and save the screen settings.
I think the fact that the NA Negative to NA Neutral gap is so large suggests that there may be an overfitting risk. Because generally it doesn't vary that much.
The book feature is what allows for long/short in simulations.
The one thing that is hopefully coming soon is the ability to set exposure on an absolute level rather than relative. Right now, if you mix a long and a short strategy and set it 50/50, and you have $100K, it will put $50K long and $50K short. Typically you have $100K long and $100K short in this scenario.
If we can apply our own leverage then you can have 150/50 or 100/100 or whatever mix you want.
I realize AI Factor is the main thing to iron out right now. But this update would really help the long/short guys to adopt P123. Fingers crossed...
Hi, saw this and was checking and wondering has this already been fixed? I use "Percentile NA Neutral" on most things and checking a few screens produces results very similar to stock lists I was getting and using last week.
