I have recently traded 5%, 10% or more of daily turnover on some low liquidity stocks all over the world and I don't seem to have observed any significant price impact. Many of the low liquidity stocks have good market makers so it seems easy to get a lot of liquidity at low cost around the midpoint price. I even tried trading 2/3 of the average daily turnover on a Mexican stock with less than 1% price impact because I just took the liquidity by market order and the spread was less than 2%. For me, the commissions seem like a bigger problem than the market impact.
It seems that, even through IB, I can easily access liquidity equivalent to 5-10% of average daily turnover on AIM shares without incurring a price impact cost of more than 1%. The situation seems not so serious as you described before.
Re: UK - You must be buying stuff that isn't on the SETSqx exchange. Usually the SETSqx stuff is less liquid and you can only buy/sell them during short auction intervals throughout the day, unless you use a more UK centric broker with better MM access.
Unfortunately I don't know of a way to filter out SETSqx stocks from P123 models so it's tough to isolate them when you're trying to run backtests and whatnot.
Of course, if you're indeed able to work with the limited IBKR trading times of the setsqx stocks, and find that the price impact isn't great, then that's excellent.
I now realise that for companies that only have half-yearly reports, the Q stands for half a year rather than a quarter. But somehow my unadjusted accordingly, selected factor list based on the backtesting results of the US microcap universe (i.e., dan's factor list) can still work better in the European stock market than in the US stock market.
During the AI training phase, what PIT method is used?
Also the AIValidationFactor function uses stored data so the result of a simulation using whatever PIT method should be the same but it is not the case, any explanation?
We have PIT Method Prelim = Use hardcoded for dataset loads.
I believe it should not be set to Prelim = Use as this is also no longer the default option when creating a simulation.
I have some strategies that are performing badly live but when I backtest the same period, I get much better results. When I do not use Prelim in the backtest, the performance matches the live strategies. Using Prelim for some factors is really problematic as the data is no longer PIT.
2 Likes
Agreed. I'll submit a job ticket to add the PIT Method option to AI Factors.
2 Likes
I have run a simulation using no buy rule, one rank sell rule and a ranking system that uses "AIFactorValidation" only.
Given that AIFactorValidation gets stored prediction data, the simulation should not change wehther I choose PIT method or not. But it does and by a big margin.
Can you please explain how it is possible?
I ran an AI Factor validation backtest using both the Screener and Strategy Simulation+Optimizer.
What I'm seeing is that the results are completely unaffected by either the ranking method setting or the PIT setting.
Specifically, both ranking methodsâNeutral and Negativeâand both PIT optionsâUse and Excludeâproduce exactly the same results. Across all four combinations, the annualized return, Sharpe ratio, standard deviation, active return, and other performance metrics are identical.
If you'd like, I can take a look at your setup.
1 Like
Thank you for looking into this, but I think that cannot be generally true. Rather, it is highly specific to your particular factors and model.
As an example, a 2-factor random forest model might split at rank 25 in an early node of a tree. If the ranking method is set to Negative, a ticker might split to the left; if it is Neutral, it might split to the rightâending up in completely different branches and final leaves. Because AIFactorValidation outputs prediction data that gets ranked, the downstream handling of those ranks absolutely matters.
More generally: a lack of variance in one specific test case is not proof that variance doesn't exist in every case.
BTW, while I am using P123 classic in my funded models now, my backtests perform better with exclude (possible due to incomplete and conflicting data when PIT is set to use) and my ports use exclude. So PIT use not only introduced look ahead bias it isnât even always the best performing model. So I think this is not purely academic.
One would think access to future information would always help. But this suggests CEOs maybe only be release what they want you to know with news releases. Things that make their company look good and burying the problems in their release. Just as I might show just one of my whole cards in poker suggesting the possibility of a full house if and only if the unshown card is a ace.
BTW, Andrew Left (Citron) just lost a criminal court case (25 year minimum sentence), in part for just telling part of the story about his portfolio management on CNBC and other places. So it does happen with retail investors being burned: Office of Public Affairs | Activist Short Seller Convicted for $21M Stock Market Manipulation Scheme | United States Department of Justice
1 Like
Good call. My tests remain indifferent to PIT Method. However, Rank NA handling can affect simulation results.
1 Like
Still showing different results for me. Take a look at ranking system 546922
I have compared ranking for specific dates (for both PIT and Not PIT) and they are a bit different
For your Universe, the PIT setting affects the universe composition, not the underlying predictor calculation. When Method PIT = Exclude, the universe is smaller because more securities fail the stalestmt = 0 test.
As a result, the absolute rank values will differ between the two universes, since ranks are calculated relative to the securities that remain. However, for stocks that exist in both universes and have a distinct ranking score, the relative ordering is identical.
The only meaningful differences occur when the AI Factor Validation returns NAs. Those securities are grouped together with the same score, and their internal ordering is effectively arbitrary. Because of this, the listed ordering within the NA buckets can vary between the Use and Exclude PIT settings.
In practice, both Method PIT = Use and Method PIT = Exclude produce the same ranking order for all common securities with valid, distinct scores. Any observed differences are limited to the handling and ordering of securities with NA values.
Interesting, didnât know there were permissive PIT requirements and that (in my universe case) 7.5% of the stocks would be ignored. We obviously donât want that during live. Does my universe shrink as well when using PIT âdo not useâ in Live portfolios?
The question is do you want to consider Preliminary data in your strategy?
Because you use StaleStmt=0 in your Universe;
If a ticker has preliminary data, PIT Method = Use makes that data available. Because it covers the latest publicly available period, StaleStmt returns False. That ticker would be in your Universe.
For the same ticker, PIT Method = Exclude ignores the preliminary data and falls back to exposes the older complete data. Because that data covers an earlier period than what is publicly available, StaleStmt returns True. That ticker would NOT be in your Universe.
1 Like
Ok I see now how StaleStmt works with PIT method. Thank you.