[quote]
But if backtesting is now evil why does P123 exist?
[/quote]P123 backtesting is just a tool. It can be used to torture the data to tell you what you want. Or it can be used to test ideas before committing real money (and we are the best at that).
I have to already kow the authers name. Yesterday all the models authors were listed . I guess less is more.
Can you elaborate? The names and usernames are listed. You can also type them in the search. Few letters suffice
Thanks
Hi Marco,
Huge fan of you, and the site as proof by being a long term user, and a subscriber of many R2G’s over time. I recommend that you put back the simulated backtest graph. We can judge for ourselves how curve-fitted the data is by looking at turn-over rate, number of holdings, and liquidity data. It looks like now, if I want to see a backtest graph, I have to run a book simulation on the R2g system. That’s too painful. I will not subscribe to any R2Gs if I can only view the returns over a short timeframe.
I feel that not showing the backtest chart is counter to the whole concept of running simulations used to create the R2G model in the first place. I want to know that if a desirable chart since launch just came out of a huge drawdown, and has historically actually been extremely volatile. Now, how do I find this information with a simple glance at the R2G model list? In the end, you have to provide as much data as possible, and trust that your customers will be able to evaluate, and compile the data, and then quantify the risks. Buyer beware, we understand this, let us be the judge of risk/reward, the data will speak for itself.
If you want to add a feature to highlight, and quantify the risk of curve-fitting, you might consider creating a score based on a simple algorithm. Variables may include liquidity, turnover rate, etc. For example a model that will buy and hold large index ETF’s may receive a curve-fitting score of 10 (or an A) etc.
Also, you may consider including some standardized robustness test results for each R2G model. This will help customers evaluate risk.
Regards,
Stu
Hi, I must have missed the listing of authors were are they? Also can I get and annual return on each 2 year performance , . Thanks
Marc, Usually I agree we almost all of your thinking but this time I must differ. Sims come from data which is content. Federal regulators? If it were not for the sims when P123 started R2G then you would have almost no paying subsribers. Maybe the Federal regulators were different 3 years ago. What the sims did show was precious few models did OSS what they did IS. Now a new subscriber cant see that because all that is shown is the excess return. So if a model in the past that is say a microcap model of low volume returned annual return of 50% but in sample returned 18% per year we would not see that . All we see now is that it might have a positive excess return even though it performed badly OSS. I also hoped people don’t trust the sims too, but I hope they don’t now trust p123 to make their decisions for them based on less than 2 years of OSS data. I believe you are the best model designer in the community, but in this case your way off target. I would never invest real money on the last 2 years data and neither should an unsavvy investor. Glad you did the “right” thing here since for the last 3 years you were doing the wrong thing and P123 portfoilios are still doing the wrong thing. You do not need to save us from ourselves.
Hi there,
Please bring back the backtest results! It gave an idea of the model behavior at least… I don’t understand why it has been removed. By the way, It would have been nice to have some warnings (emails) about this change. It’s not cool to treat your subscriber like that. Your communication towards your customers is really bad…
As a subscriber, I totally agree with what tpk and jbarnh wrote.
Having reviewed the recent change that I’m sure the P123 team spent a lot of time on, I find myself disappointed.
I am a consumer of developed models, using 7 fee based models and 1 ‘free’ model. I also have 3 prototype live models I developed that are running OOS so I can see how they perform before committing money to them. When I first joined, I appreciated the P123 site and the R2G concept as it allowed me to ‘outsource’ the model development to talented people who may approach the market from a perspective which may not be my strength. This has allowed me to diversify my portfolio in style (value, momentum, etc.), and concepts (hedging, mkt timing, etc.).
As a consumer, there was no doubt in my mind that curve fitting models existed in R2G. However, it’s not always evident given the relatively short time span of R2G and overall market conditions that may be affecting model performance. Getting to know the model designer through a willingness to share their development process and their promptness in answering specific questions helped alleviate some concerns, but the OOS performance vs IS performance will eventually shed light on curve fitting.
So while I applaud the P123 effort to discern and refocus designers away from any curve fitting benefit, the approach to this change and end result has left me as a consumer disappointed.
Through the community board, I got the sense the only input sought by P123 was from designers and how they may react to the change given how their models were about to be re-represented on the site. Perhaps I missed it, but did P123 ever reach out to the consumers (i.e.; customers) of these model products and ask them what was important?
I personally used a lot of the chart features such as date range input so I could focus in on specific timeframes and see how the model handled that period of time… IS or OOS. The model analytical data such as win%, average return win, average return loss were also important to me to calculate system expectancy. I would like to have this type of data broken out by IS and OOS but it didn’t exist in the previous version. Yearly return and DD data was important so I could calculate Asymmetric ratios… again IS or OOS.
While I understand not everyone may use these features or data analytics, it is important to me the customer as it is the method I used to help discern IS vs. OOS performance. P123 and others may not even agree with the method I use, but that’s not the point. The point being that it seems P123 arbitrarily took away features and data without seeking what their current R2G consumers wanted in this change.
I am fine with what ever decision is made by p123, there are arguments for and against showing the in sample data.
P123 owns the product, has the responsibilty and decides, basta!
I trade some of the r2gs I offer by my self and I have to say, that the ISS Data and capital curve is very valuable for me
in Drawdown situations. I look at the capital curve or drawdonw curve in sample capital curve and ask my self, is this “normal behaiviour”, if yes
I know everything is okay and Keep on trading (that happens a 100 times to me a year, so p123 really supports me to have diszipline,
I rationalize via p123 from week to week so to say).
I also love for example my big cap momentum System, because it behaves very “regular” e.g. squeezing out Alpha step by step, month by month
and not going flat for 12 months and then having a Monster month.
This for me is a sign of a good System, but you can only see that, if you can use the in sample behaiviour.
But hey, as I said, there are a lot of arguments for and against it…
And once again, smart Alpha is for me only 5% of the value of p123, it is nice to build up Reputation and for helping me: “see it works”, you
have not tweaked a System for 2 years and the Performance is fine (if I had no r2g I would think, you might have tweaked it, so
you might ly to your self, then it would be harder to follow it).
I also think, that there is no way of protecting Investors / Traders, p123 is simply not a Money printing mashine, something like that simply does not
exist, you do the mistakes: dessinging Systems (and lying to your self without knowing it) or if you choose r2gs, there is no
short cut for the most of us (at least not for me)
Regards
Andreas
On the Smart Alpha Zoom Features, like the ability to put a From and To date, has disappeared. Not sure if this was intentional, but can you put it back? I use it a lot.
Tom C
Where is “Max Profit Contribution Single Stock”? I find this statistic very useful and hope it is still there. thanks, debbie
Still trying to get an answer as to where the authors list is under the new Smart Alpha and can I get a annual return column on all the exsisting r2g models out there. ot excess return but annual return that was available before.
Answers:
Max profit contribution was meant to flag simulations that possibly relied on single transaction for the bulk of the performance. Not very relevant we expose the simulated results.
If you are looking for a drop-down for authors, that is not available anymore. Just type a few letters of a designer name or username in the seach box. It’s much more flexible this way. And dropdown lists become unusable when number of names grows.
Annual return since launch is not included because it’s a very misleading stat to compare models with different starting dates.
“Change is the law of life. And those who look only to the past or present are certain to miss the future.” - JFK
P123,
I applaud the changes you have made. For me, it proves you have your customers best interest as priority #1.
My only suggestion would be to release these types of revolutionary changes to the general public on a beta site for comment and review. To your credit, the release to the designers was a step in the right direction. Hopefully that can be made available to the general user population next time as well. I think this will engage your power users (almost all of whom are on this thread) earlier in the process and give you ample opportunity for you and the community to answer any concerns that arise. I’m not suggesting you have to adopt any of the feedback, but it would certainly give everyone an opportunity to familiarize themselves with the modified tools.
Andreas,
As always, you are absolutely right.
Konstantin,
I am a designer. I have two large cap R2G models published on this site based on the same ranking system. One version is designed to run in a tax deferred account and uses market timing for hedging, the other is designed to run in a taxable account with no hedging or market timing. I don’t ask my subscriber(s) to test with their own money - although they are welcome to do so by subscribing - instead I’m testing it (the tax deferred model) with MY own money - about $40k.
The designers you choose should be investing in their own models. They should be willing and able to tell you what the model does and why. They should be willing to share details on what the ranking system considers and how the buy and sell rules operate. If the model utilize market time you should understand and be brought into the fold by the designer on the strategy employed. I would estimate that most model designers would say the same thing.
Very happy with the new SmartAlpha and focus on OOS performance. One suggestion: sector allocation is replaced with “themes” which is too general for me. prefer sectors. thanks!!
Marco, If the drop down box is eliminated where does a newcomer go to find all the different R2G model designers. He doesnt know their names or usernames. Better yet where does an 11 year member like me go in order to remember them all. When you say it became unusable. I do not know what you mean I used it all the time. Now all we have is what is on the highest multisort ranking. AS far as misleading goes , what could be more misleading than just giving excess returns, This means that an 8% 16 year backtested return model gets the same ranking as an 50% backtest return as long as they have the same excess return. Again this is less information and data to make an educated decision. One final thought .In 2008 when all the P123 portfolios took big hits, new ones were created and put in their place. The old ports were put on a separate page. All of the new P123 ports where in sample then Why did you not leave the old ones and why are the new ones not treated the same, as you are now criticizing IS backtested sims returns as misleading . Under the same standard shouldn’t all the P123 port now only go back only to 2008-2009 and not 1999? When will that change come?
On the Smart Alpha Zoom Features, like the ability to put a From and To date, has disappeared. Not sure if this was intentional, but can you put it back? I use it a lot.
Tom C
bump
Did you try click and drag to zoom ? To zoom back out click MAX
Marco,
The click and drag to zoom is okay, but there are times when I want to be specific on a date range. For example, I may subscribe to a model earlier than I invest funds. So I want to type in the actual date I committed captial to the model to compare the model return vs my return. It allows me to see the variance between execution performance relative to the model. I don’t expect to be spot on, but I should be within a tolerance zone. With the click and drag method, it takes more user clicks to zero in on the date.