Estimize is interested in redistributing their crowdsources earnings and macroeconomic estimates to P123 users (as an add-on, perhaps $75/mo? t.b.d.). History from 2011 I think.
Do you know if it’s any good?
There are several other alternative datasets that redistribute to quant platforms like Brain, Algoseek, Quiver, Kavout, etc.
We can relatively easily add datasets, but would still take away resources from other things we are doing.
We would love to hear your thoughts.
Thanks
I’d be interested for sure.
There’s been several papers that have measured their consensus estimates to be more accurate than sell-side analyst estimates. But, their coverage I think skews towards larger cap stocks, and larger cap stocks tend to have more user estimates and hence more accurate consensus estimates. I get the sense that more users here tend to focus on the small and micro cap segments.
I already use several estimate and surprise factors, so it’s compelling to have another angle of attack.
I know that the company that owns Estimize (Extract Alpha) has several other datasets too. For example, on QuantConnect you can subscribe to their True Beats, Cross Asset Model, or Tactical Model, each for $75/mo. Would those potentially be available too?
If a dataset is on QuantConnect, it’s safe to assume that we could also make it available to our subscribers.
I have no experience w/ Estimize but I would be open to trying. However, given the expected charge, it better provide some amazing utility. And how likely is that?
I would personally subscribe to this if it is available. Tons of potential new factors can be created if we can combine these with existing P123 factors.
Marco,
I agree with Azouz (and others) that I would to look at this. But also, I could easily make a decision as to a factor’s usefulness in about an hour (less really), IF THE DATA IS PIT.
So I guess that would be a question for me: Is the data PIT?
Assuming that it might be, I note that perhaps with a trial-period (or short subscription) your AI specialist could make the same determination as I might make. They are trying to sell this to you? Will they let you look at some of the data?
You would think that they could make some past data available. If they are hyper-paranoid about their past data they could encrypt it (i.e., mask the ticker names).
Many might be able to make this determination in short order using a variety of methods. Methods might vary by preference for P123 members. BUT HOW LONG WOULD IT TAKE YUVAL OR DAN TO MAKE AN ASSESSMENT OF THE FACTORS? Dan, for example, has a spreadsheet of a huge number of factors and seems to make an assessment of the usefulness of each of these factors in a timely manner. Timely enough to be useful for his investing is all I know for certain.
P123 staff’s assessment (including you, Dan, Yuval and the AI specialist) might be different than the one I would make or Azouz or SteveA or Feldy. BUT ALSO MAYBE NOT TOO DIFFERENT.
If we are to make the choice independently, could you (somehow) give us a single-factor rank performance test for some of the factors? Maybe compare the different offerings that you mention? As a start.
If I have to wait until P123 offers this as a subscription can I get a one month trail offer with some comment on whether the data is PIT after any manipulations by P123 (e.g., lag if warranted)?
TL;DR: You could consider a detailed look at the data rather than relying on random, relatively uniformed, comments from the forum. I count my comments as being uninformed at the time. I cannot predict what I would do once I became more informed.
Jim
It would be the wholy grail (and therefore I am sceptical!) if we can get very good estimates as a data set. Jim is right, it needs to be pit.
All my eps estimate strategies outperformed 20 and 21 but had a real bad streak in 22 and so far in 23.
I think what would be needed is, the community can test it and then let the strategies run for some time before we go live. If the data is bad (non pit or other reasons) strats can fail fast and we see this before we put capital in.
If the test phase is well (could be tested via public R2G portfolios) I am in.
Andreas
From their FAQ:
Is Estimize data point-in-time?
Yes, we keep point-in-time data for all estimates contributed to the platform. We never delete estimates or change estimates. The sanctity of this data is extremely important to us because our clients need to be able to trust that the historical data they are backtesting is what they would have seen at the time in production.
To be clear, it’s not just about estimize data. What we need to decide is where to spend our resources, and adding datasets to our simulator requires some retooling. We have something rudimentary for Imported Data Series and Imported Stock Factors, but it’s not scalable.
Estimize would be the first one. Many others seem interesting, like twitter sentiment and many more that use natural language AI, etc.
In that context I think it is definitely worth while for our next project.
Marco,
That sounds great!!! I have mentioned natural language AI myself. Link to ChatGPT language processing ChatGPT
I do not mean this to imply that ChatGPT is the best natural language AI for possible consideration by P123.
My only addition, really, is some confidence that the forum AS WELL AS you, Dan, Yuval and the AI specialist and anyone you may consult with can make a well- reasoned decision.
Jim
Sounds interesting.
But to be honest, I would much rather have Asian or Pacific stock data available or be able to backtest further back in time.
The impact of testing strategies over long term data in multiple regions makes a much larger difference to me at the moment then adding another dataset (even though that would be valueable as well of course).
In my opinion an equity strategy backtest is truely solid if it is (1) univeral, (2) timeless and (3) makes sense from a financial or business perspective.
Adding new data from for example Estimize might give a few new effective factors, but adding a region or extending the backtest period would potentially give information about the effectiveness of way more factors (the ones that P123 already offers).
I give higher priority to extending the ways to test the robustness of current factors, than to add new factors.
Best,
Victor
We will decide about Asia soon. The European ROI is about breakeven now (but a lot of upfront work is out of the way so Asia will be easier)
Might be best to determine the desirability of this proposal by conducting a user poll. The quantity and intensity of votes could help determine whether this is a productive use of resources. My $.02 anyway.
Clearly that will be a huge addition in my opinion. Any single alternative data factor could be translated into so many new factors (using formulas, regression, interaction with other factors…).
The European ROI is about breakeven now
You laid the groundwork with no cost. Now, you will be ready. When international stocks outperform, there should be a surge in subscribers and a surge in ROI.
Why would this be any better than what one can get out of the P123 Core: Sentiment ranking system?
It is different data. Another way to measure sentiment and complements what we already have.
Reason I like alternative datasets (in a broad sense: top-down, bottom-up, extraneous) is that they would be ideal with AI/ML model. Plug them in and let the training factor them in or not.
So maybe after the AI launch it’s a good next project.
Here’s a site that lists them Database - AlternativeData with interesting industry stats (from 2018 so a bit stale).
Marco,
Let me stipulate that you can do that. That you could input a bunch of factors into—say XGBoost—and not overfit. That the program will “learn” (in the machine LEARNING sense) which factors are most important and weigh them properly. How well that works could be a topic for another post. Again, just accept that for now.
Again using XGBoost as an example (it also works with random forests) you can see how valuable a factor is by using “feature importance.” This is provided in the XGBoost program.
Just one link: Feature Importance and Feature Selection With XGBoost in Python
Okay another: The Multiple faces of ‘Feature importance’ in XGBoost
And documentation from XGBoost documents (a few paragraphs down): Feature importance
My simple point is this: There are some opinions above as to how valuable Estimize data and/or other data might be above. You can get a quantitative answer to this question.
TL;DR: Pretty simple to get a quantitative answer in XGBoost on which factors will work best and ultimately to calculate which one will give you the most “Bang for your Buck” in a business sense if you plan on doing much machine learning. Once you see if a factor is worth the price then let the machine do its thing. But see how much you are willing to pay first—especially if you are going to then use it in XGBoost.
Jim
Hm…
I am wondering if this is not a very sophisticated form of overoptimizing the past, with mostly dismal performance in the future ??