I’ve come looking for opinions on evaluating ranking data.
In the past I have almost exclusively been using Annualized Returns to eval ranking weights and factors. However I recently dove back into looking more closely at Performance because I came across an interesting scenario. When I use my standard weighting for bucket evaluation which takes into account things like correlation weight, slope, R-squared, and of course bucket return. I get a total score that shows there is some level of correlation with the buckets and ranks. Nothing new here and I’m sure lots of us do it this way.
However this time I was checking the performance of two scores that were very close but the factor weights were very different and what I saw was the following:
Weight A, scored hi because the top bucket performed very well
Weight B, scored hi because the overall performance looked like there was some level of graduated progression with rank.
Normally in these cases I go with Weight B because many times I’ve ran into situations were a top bucket only score is due to a few lucky holdings and that’s apparent in the Performance graphs with poor R^2 values.
In this case R^2 is better and the return value is much greater (~4x).
So what weighting would you choose A or B and why?