http://stockmarketstudent.com/stock-market-student-blog/creating-the-final-ranking-system
Thanks!
Works and had a surprising benefit for me: Robustness!!!
It did indeed make the top bucket significantly taller for one of my ranking systems (7 factors or functions). But did it increase the returns of my 5 stock system?
Annualized return before: 77.11%
After optimization: 77.29%. Hmmmm.
O.K. But what if I test robustness by increasing the number of positions?
25 positions before optimization: 35.26%
25 positions after: 48.19% Optimization increases robustness!!!
Jim
Jrinne - did you prune any factors? Something you can try is to develop a second ranking system with the pruned factors and then add in some new factors. If you can get good results then combine the first optimized RS with the second, 50% weight for each, creating a more complex but probably safer RS.
Steve
Good point. One factor went down to less than 4% but refused to disappear. You make a good argument for pruning it.
Thanks.
Jim
Jrinne - you should optimize with either 100 or 200 buckets in order to improve a five stock port. But just remember you may be optimizing on noise or rare events.
Steve
Steve,
I do use 200 buckets. Very true about noise. Understanding the statistics is a challenge and also port of the fun for me.
I was also thinking that perhaps slippage should be added if one has a good idea of the value. For example a momentum factor could affect the turnover rate and this would not be reflected in the optimization unless slippage is added.
Thanks,
Jim
J - whatever works for you. It sounds like you are starting with a very optimized RS.
Another experiment I am going to try is to optimze two or three ranking systems, one with 20 buckets, one with 200 buckets and one somewhere in the middle ~70 buckets. Then at the end combine the three into one with equal weightings 33%, 33%, 34%. This might give overall superior results.
Steve
Thought it was optimized by trial and error but weights changed a lot, top bucket went from about 65 to 82 and as mentioned robustness changed a lot.
So maybe optimized but, I think, much better after using your technique.
My other ranking system for next week seems strange. Trial and error made little difference over wide ranges. Just 5 factors in 2 groups with similar factors (2 nodes). Have assumed this is a sign of robustness. Will be interesting.
BTW, I think your technique gives some insight into robustness. If values are randomized without a big drop in the top bucket could indicate robustness.
Thanks,
Jim
These are with a universe ADT > $1,200,000
Also delta is better and first bucket is the lowest bucket. I think this optimized ranking system is just a better system.
Once you are finished optimizing and construct a port/sim then you should test your port with some noisy RSs, generated using the tool as I explained. The objective is to make sure the port degrades gracefully with “slightly off” weights, i.e. doesn’t fall off a cliff.
Steve
Jim:
Steve’s approach has merit.
Here’s another way to estimate if the optimized settings are robust. Divide the data history into 5 timer periods. (eg, 1999-2001, 2002-2004, 2005-2007, 2008-2010, and 2011-2013. Then compare your before and after optimization results for each timer period. An optimization is likely more robust if it improves all 5 than if it just improves 2 even if the improvements for the 2 are huge.
Combining Steve’s approach with the multi time period approach would be ideal eventhough it takes a fair amount of time.
Another way to look for robustness is to create two custom universes (one of “odd” stocks and the other of “even”). If an optimization on one of the universes does not improve results for the other universe, then the optimization is suspect.
Brian
Brian,
Thanks! Will try your ideas.
Jim
I have achieved my objective by highlighting the use of the RS Optimizer and its limitations in a tutorial so I will be stopping here. Users don’t have to use my spreadsheet with the weight randomizer if they are not comfortable with it. I’m sure there are other ways of generating weight permutations. In any case, I think I have found a way to overcome the issues that Brian has mentioned and I don’t want to reveal the technique may prove to be very good…
Thanks for following along. Feel free to send me an EMAIL with any questions you have.
I would like to request that anyone using the RS optimizer as I have described above try to do so in off-peak hours if possible. I’m sure this function takes a lot of processing power.
Thanks
Steve
I use Rank in my buy/sell rules. I tried putting different Rank>xx in the optimizer and I kept getting the same results. Are you supposed to put in:
Rank>98
Rank>97
Rank>90
or is there a different syntax? It does not break; it just gives same results for all.
Thank you.
David - It is quite natural to get a little confused at first. The buy/sell rules exist in your stock model (port/sim), but the ranking system doesn’t care about buy/sell rules. It’s only function is to rank stocks from 100 down to 0. The optimizer tries to give you the best ranking system it can. This is before you start using Rank > xx. If you still have problems you can EMAIL me if you want.
Steve
David,
I’m guessing that your sim is probably already buying the highest ranked stocks and they are already ranked above 99 (Rank > 99). So anything less than 99 (e.g. Rank > 98) should give the same result because it is below your cutoff.
I find it useful to optimize the sell rules e.g. Rank < 99 vs Rank < 98. This changes the turnover, total slippage, average return per trade, etc. I let the sim buy the highest ranked stocks without putting a cutoff in the buy rules.
OK, it looks like I am not breaking any syntax rules for using the optimizer but I may not have the right strategy to begin with for using Rank in the buy/sell rules. I’ll look at that again.
I have gotten good results from the optimizer with some of the other parameters. Cool tool.
Steve, I looked at your tutorial. Helpful. I am glad you have taken this on.
Thank you both.
Steve, your instruction manual for optimizing ranking systems is very informative and P123 should pay you royalties for doing their work.
The purpose of the optimization is to get as many high ranked stocks as possible into the top bucket. However, this does not necessarily produce the best returns for a sim, as I have noticed.
Best,
Georg
It seems that better performance may not be the optimizer’s strength. There are other things it can do well however, including generation of a set of “noisy” weights for sensitivity testing. And steering the RS performance for specific economic conditions such as rising or falling interest rates. I hope to touch on these in the near future.
As for compensation, I’m happy with the way things are. I’m making some money from the R2G models. I just wish we could offer more of them.
Steve
Jim/Steve, I got it to work. Needed a wider range. Your suggestion of looking at both the Rank sell and not just the buy rules was dead on. Much bigger impact on the sell side. This experiment has helped me understand a little better (I think) how ranking strategy works with buy/sell rules (be fairly open in considering what you are buying but be careful about your sell rules as well). Good exercise.