Folks - I’ve been trying to develop a new ranking system. I’ve been using the ranking system performance simulation tool to judge the effectiveness of the ranking system and the results were in line with my expectations. However, any portfolio based on that ranking system gets really poor results. I suspect that there is something basically wrong with my ranking system that the ranking system performance simulation tool isn’t showing me. That tool is too much of a “black box” to allow you to find the problems in your approach.
So I had a thought, could we develop a portfolio that exactly matches the buy/sell logic of the ranking system performance simulation tool? Then we could run the portfolio through the portfolio simulator. That would help us see the actual stock buy and sells to prove or disprove a ranking system’s performance. It must be possible. The logic can’t be that complex. I tried but couldn’t get the same results. I think we have to do it if only because the scientific method would demand it.
I think it shold be possible to do this approximately by doing the following things:
setting the “rebalance frequency” to whatever you have in the ranking system,
and remembering not to put in any liquditiy rules, but a rule like price> 3
put in a sell rule that forces selling of everything such as rank >101
zero slippage
5)timescale the same
now the hard part, the size of the “bucket” has to be the same, so if you want the rank 99.5-100 bucket that could be a bit tricky. One option is to simply set the number of stocks to something large like 500, then put in rank>99.5 as a buy rule. This buys as many stocks as it can, but leaves some in cash distorting the results. However, you can still use the data from the profit/stock/holding period to reconstruct the annualised return of the invested capital.
You can also test other buckets by putting in rules such as rank < 99.5 AND rank > 99
The alternateive is to guesstimate how many stocks are in each bucket by assuming that each bucket contains a fixed percent of the stock universe, though thte number of stocks in the universe does vary over time so it is imperfect.
Another possibility is that your ranking system is not very robust. For example, try changing the rebalance requency and if it dramatically changes the results, you may want to bin the ranking system (pardon the pun).
But it would be most helpful if you could post a link to your ranking system and simulation then we might be able to figure out whats going on!