- setting a ticker to 1 adds it to the stock factor on the specified date
- setting a ticker to 0 removes it from the stock factor on the specified date
- a ticker setting remain in force until over-ridden by a new value/date
- when there’s an update to a list, set new entries to 1, set removed entries to 0, leave unchanged entries alone
- if you run OSX/Linux you can use comm and awk to make the process somewhat easier
- plan on spending much, much more time on dealing with delisted/renamed stocks
- hopefully, p123 will automatically handle delisted/renamed stocks in currently active stock factors
Wouldn’t it make sense(and be less onerous) that with any new update, previous tickers are automatically zero unless updated with a new “1”?
The idea behind importing factors was not to create lists of tickers but to assign values to tickers. So let’s say you are able to get a source for executive compensation and you want to use that to buy and sell stocks. Companies normally report this annually. So your CSV file will look something like this: 01-05-2019,XYZ,3200000 / 02-04-2019,ABC,425000 / 02-17-2019,LMNOP,1200000 etc.
So if a company comes out with a new report listing their executive compensation, you want to quickly be able to add that to your custom factor, without having to repeat all the old ones. You definitely don’t want a fallback that assigns all the old data zeros.
Now some users want to use factor importing in order to assign ones and zeros to companies depending on whether they are in a list of companies that someone else has created, which is great. But we already have custom lists for that. If you wanted to create 20 lists based on the holdings of a certain ETF at different rebalancing intervals, it might be easier than using custom factors. Let’s say you’re trying to replicate the holdings of an imaginary ETF named WOWY. You create 40 custom lists of its holdings on 40 different dates. You then call up those lists in your backtest by using the DaysSince command. If DaysSince(date) is greater than X, use this list; if it’s greater than Y, use this list instead. You can nest Eval commands in a custom formula to do this. I think this would probably be easier than using the factor import, and you could have done this for years before now. The power of custom factor importing is much greater than just 1s and 0s.
Yuval, what you are missing is that the ranking system is applied to the universe, for example “All Fundamentals”. That is a lot different than applying it to only the stocks in your point in time lists.
Well you definitely want to do this in a universe. DaysSince can work there. I’m guessing that you would eventually run into a rule count limit.
Walter
Not just that, but also that handling the stock factors in my manner is exponentially easier than nesting eval commands and dealing with dozens and dozens of different lists…
I didn’t even think of this feature and I find it great, but the way its set up now seems less useful than what I immediately imagined it to work like.
I’m not sure nested evals are needed. This seems to work in a universe. It has its own data management issues, but hey.
Walter,
That is an awful lot of lines of code for a sim rebalanced weekly.
About 1,300 lines for a 20 year sim rebalanced weekly? Is that in the ballpark?
I do not know what Marko (Nisser) wants to use this for but it seems like there might be situations where one might have to go the stock factor route.
I get that it can all get done the way you and Georg have developed. But I haven’t used it with the outdated ticker problem being the worst problem for me.
Anyway I am not sure where I would disagree with anything Marko has said about ease of use.
Some sort of automatic zeroing would not be a bad thing.
Looks like we might have a handful of power users who use it mainly on ETFs now.
I guess I count myself among those who could spend a few weeks and maybe get something with only a few errors from incorrect zero dates —like holidays–that may not affect things too much.
Not a feature request. I do not know if P123 wants to move to other things or not and I am probably indifferent.
But I do not see where Marko is wrong.
I do think that features seem to be getting harder and harder to use .Getting harder and longer to program into the platform and are useful to fewer and fewer people when the features arrive. That is not going to get better as we move past the low-hanging-fruit. Maybe we are beyond the low-hanging-fruit already.
Not that Python would be a piece of cake for everyone. But as easy as this and it opens a huge number of features all at once.
Not a feature request. Just an observation.
And honestly I wouldn’t even know what to recommend if I were making a feature suggestion. But Marko has a point, I think.
-Jim
I have done this with lists already for USMV, and it is very cumbersome. This is for a sim starting on 1/1/2013 and ending on 6/30/2019. The starting date is important because Portbars is being used, it cannot be changed, otherwise you get nonsense.
First you need to set up your historic holdings so your sim knows which list to use and when, this is done with Portbars.
Step 1: you need a custom formula as a buy rule in the sim such as: $USMV_starts_01_01_2013
Eval(PortBars<4,InList(“USMV1212”),Eval(PortBars<61,InList(“USMV0313”),Eval(PortBars<125,InList(“USMV0613”),Eval(PortBars<193,InList(“USMV0913”),Eval(PortBars<255,InList(“USMV1213”),Eval(PortBars<318,InList(“USMV0314”),Eval(PortBars<380,InList(“USMV0614”),$USMV_Jan13P1)))))))
Then you need another formula for $USMV_Jan13P1
Eval(PortBars<444,InList(“USMV0614”),Eval(PortBars<506,InList(“USMV0914”),Eval(PortBars<568,InList(“USMV1214”),Eval(PortBars<631,InList(“USMV0315”),Eval(PortBars<695,InList(“USMV0615”),Eval(PortBars<757,InList(“USMV0915”),$USMV_Jan13P2))))))
And another formula $USMV_Jan13P2
Eval(PortBars<819,InList(“USMV1215”),Eval(PortBars<883,InList(“USMV0316”),Eval(PortBars<946,InList(“USMV0616”),Eval(PortBars<1009,InList(“USMV0916”),Eval(PortBars<1071,InList(“USMV1216”),Eval(PortBars<1134,InList(“USMV0317”),Eval(PortBars<1197,InList(“USMV0617”),$USMV_Jan13P3)))))))
You are not done yet. And another formula $USMV_Jan13P3
Eval(PortBars<1260,InList(“USMV0917”),Eval(PortBars<1321,InList(“USMV1217”),Eval(PortBars<1385,InList(“USMV0318”),Eval(PortBars<1448,InList(“USMV0618”),Eval(PortBars<1514,InList(“USMV0918”),Eval(PortBars<1572,InList(“USMV1218”),InList(“USMV0319”)))))))
So now you have the holdings data from 1/1/2013 to 3/31/2019 in your buy rules which is good until 6/30/2019.
Step 2: You have to set up the universe USMV-ALL-since-Jan-13, which is InList(“USMV1212to0319”) and are all the stocks that USMV had held over the period 1/1/2013 to 3/31/2019. This does not overcome the problem of ranking the stocks that you have in the current list at a specific time, but it is good enough, better than ranking All Fundamentals.
AAP AAPL … ZBH ZTS
So if you think this is easier to do than uploading a stock factor $$USMV into the universe, and no buy rules in your sim, then please go ahead with the above.
Jim, Agreed. Especially problematic if it runs up against a rule count limit. I plan to stick to stock factors for this use case. However, as Yuval pointed out, stock factors are meant to capture continuous data. Say you had AAII’s Sentiment Survey data, you wouldn’t want to clear older entries when there’s an update.
Walter
EDIT: I was going to suggest adding commands to stock factors, but that has issues, too. Maybe.
EDIT 2: I think my use case example is wrong. Ignore that bit.
Walter,
Agreed. I like this feature and it is even possible that I will use it someday.
My only real point, that is new, is that InList is a non-starter for many things unless I am missing something.
A bit of a “red-herring” I think. But again I could be missing something.
Marko has already made an articulate point about automatic zeroing and I will not belabor that point.
-Jim
Jim,
I used InList because it was there for a prior use. I don’t think it’s required. Ticker() may work too.
Walter
EDIT: auto-zero has problems, too. I don’t think that’s a solution.
Walter,
Definitely agree that auto zero only works if it works as an option.
As far as InList it is great to have as another option. I have only used it for live ports to date.
I will have to look at its potential use in sims before commenting much more. I think there are some thing it I do not know about it.
Thank you.
-Jim
Jim,
Let’s not overlook the lack of documentation. That’s the real problem.
Walter
Normally I don’t like to talk about what we’re planning very much because even the best laid plans etc. But in this case, since there’s so much discussion about it, I’ll tell you. We are planning to subscribe to a service that will give us the historical holdings of ETFs. We will then enable any ETF to be named using the Universe command in universes, screeners, and simulations. That will be point-in-time. From there we will introduce more functions, one by one.
As for when this will be implemented, I don’t know. We have a lot of improvements in the works, and in the meantime the FactSet transition requires a great deal of attention.
Obviously, once this is implemented, you won’t have to use custom factors to define an ETF-based universe.
Wunderbar, Yuval!
There has been a lot of discussion and it has me interested.
It will be nice to thoroughly test how piggybacking off of ETFs works.
Thank you.
-Jim
Yuval that will be great having all those ETF holdings in P123.
You can even charge a few $ extra for this add-on.
As you may know I have had this idea already almost 6 years ago when I periodically downloaded the holdings of USMV and VDIGX. At that time it was difficult to decide what ranking system and rules to use, for obvious reasons.
Now, I have all the stocks for USMV in a stock factor and can check on how a port would have performed with various number of stocks from USMV , and different ranking systems and rules.
I can confirm that I can always outperform the ETF easily. Attached is a screen of the Top25 from USMV since approx. date of inception of USMV.
We can also contemplate running shadow ETFs this way, that would be very annoying to the ETF providers, but there is nothing they can do about it because they have to publish their holdings every day.
I thought this was the correct syntax for stock factors series. But i am getting some errors
2020-01-25 INTC 1
2020-01-25 CRM 1
2020-01-25 NOW 1
2020-01-25 BLK 1
2020-01-25 BMY 1
can someone help me out?
I did the same thing before for a different series and that worked fine
Your date syntax is wrong.
Should be
2020/01/25 INTC 1