Trading at the open

Manage
21

Okay, so here are the two possibilities we are exploring:

  1. Buy on the open and suffer whatever slippage you may get which is the old school way most of us do it, and

  2. Put in a limit order today at yesterday’s close and if it goes unfilled then buy at today’s close.

Slippage is the issue here since both methods will get you in 100% of the stocks we intend to buy that day. When comparing methods I will reference slippage from a simulated execution from today’s open. As I’m sure you all know, in probability theory the expected value (or mathematical expectation) is equivalent to the probability-weighted sum of the possible values.

If we calculated EV of buying at the open it would look like this:

EV = 100% * -0.25% = -0.25%

100% percent probability of getting whatever slippage you care to assume - I’ll use 0.25%. A more conservative number might be along the lines of 0.5%. I’ll also give it a negative value because it has a negative to your account.

The expected value of using entry method 2 would look like this:

EV = (52.7% * -0.25%) + (35.8% * 1.13%) + (11.5% * -2.07%) = 0.035%

52.7% probability of the open being below the previous close and we get in at the open while suffering from the same -0.25% slippage as in Method 1,
plus a 35.8% probability of the open being above the previous close and then falling to hit our limit order intraday times 1.13% positive slippage (a figure I just calculated as the average previous close to today’s open change for stocks that gap up on the open but the low is less than or equal to the previous close, over the last 5 years ),
plus an 11.5% chance of opening above the previous close and never hitting the limit and buying at the close times -2.07% slippage (1.82% open to close movement of these stocks intraday plus 0.25% slippage on that transaction since it would be a market order).

According to my calculations Method 2 produces positive slippage on average. The big weakness in all of this is my programming skills, so if anybody cares to independently calculate the probabilities and returns it would be greatly appreciated.

22

Does this take into account that on a re-balance we are usually both selling and buying a position?

23

No, we are only talking about buying here.

24

I reviewed my programming and found an error - this is the actual expected value equation.

EV = (52.7% * -0.25%) + (35.8% * 0.95%) + (11.5% * -2.07%) = 0.035%

The actual value really didn’t change much. Sorry.

25

My turn to try to add some clarity. I believe that “the ones that got away” are indeed critical and without them the performance is affected badly - even buying at next day close doesn’t help. A simple screen test can do it.

I ran a screen using the Russel3000 as universe and a Random ranking system and looked at one year performance. All used Daily rebalancing.

With no buy rules (ie the full universe) the AR is 59% (0.19%/day) which seems about right knowing how the market has performed.

If I then exclude those whose low is less than prior close (ie include “the ones that got away” only) you get a spectacular 21,000% (2.17% / day) return. About 12% of the universe were in this group.

Furthermore, the ones we were able to buy at yesterday’s close (reverse the buy rule) showed a loss of about 21% (-0.09% / day) - about 88% of the total universe.

Clearly the difference is huge. Are these consistent ? If you calc. a weighted average daily return from these two screens you get about 0.18% - ie basically the same as the whole universe (ie no buy rules)

Buying the “ones that got away” at the close doesn’t help - they showed about the same return (0.2%/day) as the “no rules” case. Whether or not a longer holding period will overcome the initial 2% “handicap” of not buying at yesterday’s close depends on other factors (ie the specifics of your system)

To me its not surprising that bulk of the total return for the universe comes from a small number of stocks that perform really well.

Seems that if you want to buy at closing prices then you buy at the close - why not ?

Cheers

Lindsay

26

This is basically what I’d kind of suspected, just lacked the skills to do the analysis :slight_smile: thanks, and likewise for various others’ excellent contributions so far…

Not sure about that though - surely in that case (if you rebalance before the close and use those recs), you’re using data that doesn’t include today’s intraday changes, so you’re in the same position as having rebalanced before today’s open and failing to get the open price for whatever reason. So say one of your recs is a stock that gapped up and hasn’t come down to or below the open, do you buy it? Surely it’s one of the ‘ones that got away’ already by this point, and buying it at the close doesn’t really do you any good?

In any case, your analysis seems to be indicating that the proposed limit order at close(0) + buy-on-close if not triggered strategy may not be the way to go. So rebalance-Monday (in my case :wink: rapidly looms, and what are we left with? Perhaps; 1) a limit on open with a limit set a little above the previous close (perhaps at the threshold over which you’d be allowed to bust the trade anyway) - this part is to try to get the open where possible but with a sort of added gate to filter out these weird (and costly) fluctuations we’ve been seeing, and 2) any that don’t go through, action via manually submitted market orders (or limit orders set at the ask - you’d trigger any orders hidden inside the spread anyway) shortly after the open?

27

All:

Here is some data on sells, or exits from my sample. To the extent my sample matches Rob’s data for the market, which it did for buys, or entries, this may be valid. The sim I used to get the data is here ; a fairly standard public sim.

Below is the data which suggests that the same trading strategy works. Using a limit order set at yesterdays close, I get 59% of the sells executed at yesterday’s close, 28% at the intraday high and would sell the rest at today’s close. This is similar to tests I have done on a lot of different strategies. I general, the sells at the open are around 50% and intraday, at the high, about 40%. So, I think this sim is representative of others.

Glenn

ExitTest 5=10.jpg
ExitTest 5=10.jpg764×380 51 KB

1 Like
28

Hi,
Regarding measuring the slippage on the open. If the prior close is 100 and you set a limit order for 100, and are filled at 100, do you incur slippage? I measure my slippage from the prior close and so I would say no.

I have seen that the ones that got away can outperform, though to some extent I think this can vary with the strategy. But there is also value in achieving a better entry price (again, this may depend on the strategy). I could see in the sims that if I used a rule like open(-1) < close(0) * x (where x < 1), I could improve my gain/trade (with less trades). So the question was, if I set a limit order to close(0) * x what is the impact. A while ago I set up a spreadsheet with several thousand trades so I could test this. I found that if I set the limit order above the prior close, I caught more of the ones that got away and achieved some improvement in gain/trade. I also found that if I lowered my limit order below the prior close, I lost some trades but because of a better entry price with the same exit, I now also achieved a higher gain/trade. Which way to go with the limit order I think depends very much on your strategy.

The tests are one thing and real life can be quite another. The backtests generally assume that you will be filled if the stock trades at your limit price, and that if the open is less, then you will be filled at the open. This is very often not the way it works in real life. On a lower open, sometimes I am filled at the open but other times I am filled at my limit order. In this case perhaps I don’t incur slippage relative to my target entry, but certainly I’m doing worse than what I would have seen on this in the backtest. I have also seen stocks trade below my limit price with no fill at all, or sometimes only a partial fill.

For this reason I track my real time trades. I place my orders prior to the open (mostly). I track my slippage vs. the prior close. YTD, for my daily buys and sells (longs and shorts) and for my weekly sells I have positive slippage, sometimes significantly positive. Especially for short term trading, to have positive slippage relative to the target entry and exit prices can be a significant edge. The only strategy for which I have negative slippage is my weekly buys - because these are weekly I allow more leeway on the entry.

Don

29

Lindsay: I ran the screen and got the results you mentioned. I assume you used rules like low(-1) >= close(0) to get the 21,000% and low(-1) <= close(0) to get the -21%. I am not sure the screen works very well for me.

On the first rule, I agree that you want to include the rockets. Unfortunately, you don’t know until after the close of today’s market which ones they really are. If I don’t buy at the close the rockets open the next day up 3.94% on average (I never tested this and find it surprising; supports my gut feelings). I think you are better off buying in at the close.

On the reverse of the buy rule, or low(-1) <= close(0), This rule would not include the stocks that opened below the limit order. Using a rule like Open(-1) <= Close(0) OR low(-1) <= close(0) will include these stocks, but it still shows a negative AR of about 20% (-.06% per day). Why? First, the screen is measuring from the open so it would not include the amount the close(0) limit buy was below the open (1.15% from my test above) or the amount the intraday limit buy at the low was below the open (1.13%, interesting I never tested this). This suggests that the real screener gain per day would be higher. How much? Maybe in the 1% gain per day range? If so, I get AR close to 1500%. Hard to say what it really is but I don’t think it is -23%.

One other thing that I think is significant. I can’t tell how the screener will buy - buy first if the open is < close(0) and then move on to the next stock, or the reverse. The former is how a limit order would actually work during the day, the latter is look ahead bias. I just don’t know.

I am curious how you figured out “about 12% of the total universe” from the screen data. Did you use the average positions divided by 3000 for Pr3000?

Daniel: On “In any case, your analysis seems to be indicating that the proposed limit order at close(0) + buy-on-close if not triggered strategy may not be the way to go.”

I don’t think so. Here are some options using the test sim. These may be relevant if the sample sim is representative of the market or your port(s).

  1. Buy and Sell with a market order to get the open - AR = 48% - this simply uses open prices. The problem for me is at what price do I really get filled. It is also the worst option.
  2. Buy/Sell with a limit order at the P123 recommendation; don’t buy if not filled - AR = 61% - here you miss the rockets, but it is not a loser.
  3. Buy/Sell with a limit order at the P123 recommendation; buy at close if not filled - AR = 83% - here you buy everything, including rockets at the close.

How good can it get? The best I can do is AR of 95%. For buys, a limit of close(0) * 1.005 at the open and switch to close(0) * 1.00 intraday. Buy the rest at the close. For sells, close(0) * .995 for the open and close(0) * 1.00 intraday. Sell the rest at the close. I think there is probably too much “science” here and this would be difficult to keep straight in actual trading, especially with multiple ports. I do think it suggests that a little flexibility is good. I tend to use 2 to 3 cents.

Glenn

30

Lindsay,

These numbers are similar to the ones I quoted in my third post in this thread, and I agree that stocks whose low is greater than the previous close tend to perform better that day than those who don’t. But the bottom line that I was trying to demonstrate with my Expected Value calculation is that even though your are missing the open to close gains on a minority of stocks (the “high flyers”) on the day of entry, you are more than making up for it with price improvement (entry at yesterday’s close price instead of the today’s higher open) on the entry of a larger number of other stocks. Once the first day is over you still own the “high flyers” so if they keep flying you will still realized their gains.

However, your Expected Value would be very different if your port does not select stocks whose behavior is similar to the entire universe as a whole. For example, if you had a port that theoretically selected only those “high flyers” (and if you have one that does, could you share it with me?) then your Expected Value would be -2.07% and I’d be the last one to argue for using the previous close limit entry method.

Rob

31

ok this is very interesting indeed - any chance you could explain how I could do a similar experiment but specific to a given portfolio I’m interested in? (I expect the information’s already there in the discussion, but well, sorry, guess I need the remedial maths version :wink: I will of course post results for the group…)

Agreed/I’ll almost certainly do the same, but would feel a lot better knowing what theoretically optimal %-age I’m aiming for :slight_smile: (but perhaps more importantly, think it’s important to sanity-check that a given system doesn’t behave differently wrt. ‘high flyers on open’)

32

All:

Given the above discussion, I decided to analyze my 10 re-balance trades today and share the results. Only one day and a rather unusual one as well. To me the results look surprisingly like the analysis in this post.

I had 8 trades filled using a limit order set at the P123 recommendations. All were automatic. There were 4 sells & 3 buys at the open and I got favorable slippage of .42%. One sell was filled at an intraday low at my limit (0% slip). Overall I think I got favorable slippage of .36% average on these 8 trades.

I had 2 limit orders that were not filled, both were buys and both gapped up at the open, above the prior close (+3.2% and +1.3%). They looked like rockets that would “get away” so I bought them. They closed higher today (+8.0% and +4.3% from the P123 rec.) and I ended up with +3.9% and +1.2% gains. Neither hit a low intraday that approached the limit. Time will tell if they were good long term trades, but I got in for the ride as my model tells me to do, and did it better than the models testing results, which would be today’s close price.

Daniel: Regarding “any chance you could explain how I could do a similar experiment”, I download the realized transactions to Excel. I then use the XLQ add-in for Excel. I have the buy and sell dates from P123 and I have the prior close, open, high, low and close prices in the spreadsheet using XLQ. Then it is just a matter of comparing the prior close (P123 rec.) to the open to see if you would get filled at the open, intraday at the high/low, or the close for each trade. It is not simple but I would not say its that difficult. Don Peters described XLQ well here . I am not associated with XLQ or have any reason to suggest them other than the insights, understanding and things learned by being able to look beyond the P123 data at the actual, and possible, trades.

Glenn

33

I have been following this thread with a lot of interest. Now after reviewing all the comments again it has reinforced my reasoning for the way I trade.

I have mentioned in other threads that I set up market order trades before the market open using the Port’s recommendations based on the previous close. I then place the individual trades as soon as the market opens and the bid/ask values are close, unless a stock is trending down. That gets me a price close to the open price with insignificant slippage (relative to the price at the time I place the order). If the stock trends down immediately after the open I wait until it trends up over about a 5 minute window and then place the market order. If the stock has not trended up within 20 minutes (when I have to leave for work) I place the order at that time. That gives me positive slippage on those stocks (relative to yesterday’s close). These are the only stocks that I think we may have an chance of doing better that the Sim. Here are my reasons for trading this way:

First, and I think the most important reason, is the fact our Sims/Ports recommendations select winners 60 to 75 % of the time. I won’t trade a Sim that has winners less than 65% of the time. So it is more likely that more of the stocks will go up instead of down on the first day. My data has shown that to be true. I don’t want any of those to get away.

Second, as Rob showed in his analysis on page 2 above, you don’t want to miss buying the stocks that open up and never revert to yesterdays close during the day (the stocks that got away if you use a limit order at yesterday’s close). According to Rob’s analysis of the total market those stocks make an average gain of 1.82% the first day, open to close. I feel that the stocks that are recommended by our Ports probably exceed that. Glenn’s results of buying those stocks before the close show that, on average, they have a gain the next day. So on average, these stocks have the biggest gain over the first 2 days of all the other recommendations. Why would we take the chance of missing out on that?

Third, I need to make my trades before I leave for work at 8:50 Central Time, and I have few opportunities or desire to check the market during the day. My stops cover me for that.

Denny :sunglasses:

34

Denny,

I am curious. How many total positions are you trading (daily and weekly). How much time is this real time monitoring taking daily?

Best,
Tom

35

I have to agree with much of what Denny says. I think he is right in that most of the Ports we are trading are selecting stocks that do much better than the average and may have a higher occurance of the high flyer effect discussed earlier. I myself am able to watch the market and enter trades in a similar manner as Denny does - at least on most days. I have tracked my executions relative to the open (which is the price I use in testing my sims hence my “benchmark” for execution) for all my transactions since December, which is over 200 one-way trades. For reference my sell executions on average are -0.18% worse than the open, and buys are 0.01% better for an average of -0.1% worse execution than the opening price. These executions are from two different account, a retail brokerage account and a direct access (Interactive Brokers) account. My batting average is actually better for the direct access account as I can see the quotes and react instantly to them. -0.25% for sells and 0.23% for buys for an almost scratch average of -0.02%.

This is very satisfactory to me but I wish I could find a way that was as effective for those days when I can be in front of the screen in the morning.

36

Tom,

I typically hold between 20 and 30 stocks and 5 ETFs at a time. I trade 2 daily ports (total 5 stocks) and the rest are weekly.
So on monday, on average, I am selling and replacing 3 to 5 stocks (so 6 to 10 trades) and the daily ports typically trade 1 or 2 stocks on the other days.
the weekly trades are set up Sunday night, and the other days of the week I set up the 1 or 2 trades ~ 30 minutes before the open.

So on monday, more that half of the trades are made in the first 5 minutes, and most of the others in the next 5 minutes. Generally there are only a few that are trending down the full 20 minutes I am watching the market that I then trade just before I leave for work. the rest of the week there may be only 1 or 2 stocks that are not trades in the first 5 minutes. I watch the market less than 1 hour each morning, but normally I am also doing something else on the computer at the same time.

Denny :sunglasses:

37

Dears,

The following buy rule is working with simulations:
Hi(-1) != Open(-1)

But if a port using the above rule it gives the following error:
“Negative offset for Hi() is only allowed for backtesting”

My question, how to consider the above-mentioned buy rule in real life trading since it isn’t considered in the port ?

Regards,
Isam

38

Essam,

Hi(-1) != Open(-1) is not too hard to trade as long as you can watch the market during the day. This buy rule means that the Sim only buys stocks that increase in price above the open price sometime during the day.

In the Port turn off this buy rule. When you rebalance you will get a list of stocks that meet all of the other buy rules. Make a copy of that list. Watch the price of each stock on the list after the market opens. As soon as any one of the stocks trades at a price above the open price place a market order. Most of the stocks will have a price higher than the open shortly after the open.

I have no idea how you can trade this rule if you can’t watch the market during the day.

Denny :sunglasses:

39

Essam

I think the rule you are using doesn’t make much sense. It checks if the high of the day is not equal to the open, so it will be true in all cases except when the hi is not exactly equal to the open. You probably want to use a rule like Hi(-1)>Open(-1) to check if the price has risen above open.

In terms of executing these trades, you could monitor the prices at open and after you know the opening price, then use a Buy Stop order to purchase if price rises above the open.

-Bruce

40

Essam:

Be careful of the look ahead affect with this rule when back testing. The rule will find stocks that increase in value AFTER the open, but if your sim uses the OPEN price for its BUY price then the only way to trade this in “real” life is to use a “crystal” ball that can see into the future.

Regards,
Brian