"Next open" always the best

[SpacemanJones]

Hi all, I wanted to share some work inspired by conversations here. I got interested in implications of delay mentioned here because my framework has tended to be “moving faster is better than moving slower” when something new shows up in my ranks or when something falls out. In theory at least. In practice it’s harder, because I almost never just place “market on open” orders. But in truth, I can’t recall measuring the idea of “what is the cost of taking a week or two to get in and out of positions?”

I ran some studies using RankPrev(n) on a couple of screens I use. These are fairly low turnover screens (maybe 15-20% turnover every 4 wks) and much of the change in rankings - both new arrivals and falloffs - seem to happen around earnings events. For the most part liquidity is good and slippage is low. (Probably 90% of companies are over $1B mkt cap and currently 30-40% are over $40B mkt cap - so lots of very large companies), although occassionally I get some that require patience.

Essentially for the screens I found that transacting using rankings of RankPrev(1) instead of Rank caused an annual return performance to drop 0.8 to 1.5 pp (vs Rank), and RankPrev(2) had a drop of 2.5 to 3.8pp (vs Rank). RankPrev(3) through RankPrev(10) mostly stayed in the range of dropoff of 2.0 to 3.2pp (vs. Rank). Beyond RankPrev(10) the degredation mostly continued trending downward with it generally being in the 4.x pp range out to RankPrev(24) and being 5-8pp decline from there out a year to RankPrev(52).

I was surprised that the delayed performance held up so well, but in hindsight these are fairly low turnover screens with quite a few longer term fundamental quality requirements.

But my experience is that even though the screens are low turnover, the new arrivals and departures can move in violent ways - regularly 10-20% moves in both directions. I’m realizing I’d like to study the precise arrival and departure events for these screens in closer detail because and entire year of gains/losses often happens in day (or even in zero time overnight) with them, but not sure how to. My intuition after watching is to move faster on the arrivals (they seem to gather steam over time) even though I’ll miss most of the bounce, and be more patient with the departures that are big drops (maybe more often than not initial over-reaction to downside) - but that’s all anecdotal and I have low confidence in what I’ve seen as things behave differently in different market conditions and industries.

[WalterW]

Using the Easy to Trade Universe and Small and Micro Cap Focus ranking system, I ran weekly sims across 01/02/06-08/23/23. Attached is a chart of the annualized returns. Next Open performs better on Monday and Tuesday. After that it doesn’t really matter.

Thankfully, Monday’s close is near Tuesday open. There’s continuity there. Same for Tuesday/Wednesday. So maybe it is efficiency - or momentum.

I’m going to try redoing the chart to offset the closes since they’re only separated from the next day open by the overnight session.

[sglinski]

Buy the close and sell the open (and vice versa) would be nice features for P123 sims…

[yuvaltaylor]

@Whycliffes

There is no way of testing this in p123, so I would be happy to hear some input from more experienced investors. How difficult is it to trade during the opening hours and to get into or close to the opening price?

You can always place a market order before the open if you’re willing to pay the spread and risk the market impact of your order. It all depends on what kinds of stocks you’re trading.

Because I place really huge orders for somewhat illiquid stocks, I always use VWAP orders where I can. When I can’t, I try to break them up and spread them through the day. But for small orders for stocks with low spreads, market on open shouldn’t be a problem. Another thing I often do is place a limit order before open a little below the last close for a buy and above the last close for a sell. When it gets filled, that’s great.

But remember this: it is impossible to get better fills than most market participants by cleverly placing orders. If it were possible to do so, then you could profitably do pair trading (sell a stock in one account and buy it in another on the same day) with liquid stocks and there would be no need to be a stock-picker.

[rwbattyaz]

It depends upon what your priorities are. I know I may be giving up some of the spread to get a fill. The stocks I trade are from the NOOTC universe with a $200K median and less than 1% bid/ask spread. I place limit orders using the prior close and the limit being a half percent against me. So on a $20.00 prior close this would be $20.10 for a buy and $19.90 for a sell.

I’m in Arizona which means the market opens at 6:30am local time. When I login about 10:00am local about 85-90% have been filled. This week 19/20 sells were executed. This is an IRA which I need cash to buy and 9/10 orders were filled completely. The remaining one was partially filled and I completed it 0.3% above my limit – the stock went up another 0.5% to close.

I very much try to catch the open with focus on the less liquid stocks as my conclusion without deep analysis is that it helps. You can also tell when you are crossing with index funds with all the small quantities.

Cheers,
Rich

[Whycliffes]

@SpacemanJones

I’m realizing I’d like to study the precise arrival and departure events for these screens in closer detail because an entire year of gains/losses often happens in day (or even in zero time overnight) with them, but not sure how to

It seems that most of us find that selling and buying at the open, or at the previous close gives better results. But what we don’t know is if it’s both sides that contribute to the better performance or just the stocks you sell (or buy) at the open.

Is there any way of testing whether to sell at the beginning of the day, and buy at the close and vice versa?

Maybe yuvaltaylor knows?

[Jrinne]

Whycliffe’s,

Discussion: Strong evidence for signal(s) (both the buy and sell signals) affecting the price action the day of the trade for this model over this limited time-period? This just a test of a method and not much data for any broad conclusions.

I hate data wrangling or munging (whatever you call it). But would it be that hard in Python (you know the answer better than I do)?

The transactions for next open and next close should be exactly the same shouldn’t it (for the sims)? So you should be able to sort the buys and the sells for each and maybe they will still line up in a DataFrame or spreadsheet.

Or maybe you are advanced enough that you just to an external merge or something.

If not lined up, maybe a date sort and/or ticker by alphabetical sort (double sort) might fix most of it… Not being a great programmer I would probably use a dowloaded spreadsheet from there and compare the percentage difference in the opening and closing trade prices for the buys and the sell.

Or just upload a sample of the transactions and into ChatGPT code interpreter and have it tell you the answer.

TL;DR: The good and bad news is we will use ChatGPT and Python more in the future. But we can answer any question we may be interested in if we have the proper downloads.

Yea, that worked (sample of only 100 trades to make sure I did not get into any issues for upload size):

Q:

A;

Code ChatGPT used.I agree you should check it but you could and correct any errors you find (I did not):

Extract the buy and sell rows from both dataframes

open_buy = open_df[open_df[‘Type’] == ‘BUY’]
open_sell = open_df[open_df[‘Type’] == ‘SELL’]
close_buy = close_df[close_df[‘Type’] == ‘BUY’]
close_sell = close_df[close_df[‘Type’] == ‘SELL’]

Reset the index

open_buy = open_buy.reset_index(drop=True)
open_sell = open_sell.reset_index(drop=True)
close_buy = close_buy.reset_index(drop=True)
close_sell = close_sell.reset_index(drop=True)

Compute the percentage difference in buy prices and sell prices

buy_price_diff = ((close_buy[‘Price’] - open_buy[‘Price’]) / open_buy[‘Price’]) * 100
sell_price_diff = ((close_sell[‘Price’] - open_sell[‘Price’]) / open_sell[‘Price’]) * 100

buy_price_diff.mean(), sell_price_diff.mean()

Jim

[yuvaltaylor]

@Whycliffes

Is there any way of testing whether to sell at the beginning of the day, and buy at the close and vice versa?

Maybe yuvaltaylor knows?

See Returns in trading versus non-trading hours: The difference is day and night | Journal of Asset Management

If most of a stock’s rise happens overnight, it makes more sense to sell at the open and buy at the close.

[Whycliffes]

I don’t know how many of you have developed a European RS, but for some reason, the use of open vs. close doesn’t work the same way in Europe.

Can anyone confirm this?

[yuvaltaylor]

For me it works the same: next open is best, next close is worst.

[AlgoMan]

I made a spreadsheet to compare sell and buy transactions - If you make a copy of the spreadsheet you can copy/past your own transactions in to it.

Compare Next Open/Close prices P123

[Whycliffes]

Thank you! I will test it!

Do I understand your findings correctly that the best idea is to close the sellers at the open, and buy the buys at the close?

[AlgoMan]

In my sample data it looks like the best buy is on the previous close, best sell in next open (with a small margin to previous close).

When I click on Rebalance now for a live strategy, it says “Recommendations will be based on fundamental data as of 2023-08-26.”, that’s two days ago. In a simulation, do we simulate on fresh data from today, or two day old data (as in live strategies)?

[Whycliffes]

Im getting some weird results!

Im trying to compare the same strategy, but buy close vs. buy open.

I downloaded the whole transaction history, which should be identical. But it isn’t. As early as line 28, the difference starts. Why?

OpenvsClose

[Jrinne]

TL;DR: I hate munging data a little less since ChatGPT came along. And do not let this fool you. I suck as programmer. I did not get inner and outer merge right. I called it “external merge” and did not have the right use, for example.

Here are the results for 20 years of data for the sim above. ChatGPT allowed me to upload the entire file:

Difference of open compared to close (buy then sell). I.e. buy open and sell open just as Yuval says above (for my model too); Or for buys the close price is higher than the open and for sells the close is lower than the open (on average over 20 years). We have the same result FOR A SIM RUN ON MONDAY:

@yuvaltaylor

For me it works the same: next open is best, next close is worst.

So I do not know why the columns do not line up but I was pretty sure they would not. Maybe this is important and Yuval can tell us why if Whycliffes wants to know.

Above I mentioned an “external merge” the correct term would have been ‘outer merge’ and I really needed an inner merge. Above was not correct.

One needs to do an’ inner merge.’ ChatGPT walked me through that. And BTW, you can upload the full 20 years of transactions.

Code:

// Perform an inner merge on the ‘Date’ and ‘Symbol’ columns
merged_df = pd.merge(tran_df[[‘Date_open’, ‘Symbol_open’, ‘Open’, ‘Price_open’]], tran_df[[‘Date_close’, ‘Symbol_close’, ‘Close’, ‘Price_close’]],
left_on=[‘Date_open’, ‘Symbol_open’], right_on=[‘Date_close’, ‘Symbol_close’], how=‘inner’)

// Extract the buy and sell rows from the merged dataframe
merged_buy = merged_df[merged_df[‘Open’] == ‘BUY’]
merged_sell = merged_df[merged_df[‘Open’] == ‘SELL’]

// Reset the index
merged_buy = merged_buy.reset_index(drop=True)
merged_sell = merged_sell.reset_index(drop=True)

// Compute the percentage difference in buy prices and sell prices
buy_price_diff_merged = ((merged_buy[‘Price_close’] - merged_buy[‘Price_open’]) / merged_buy[‘Price_open’]) * 100
sell_price_diff_merged = ((merged_sell[‘Price_close’] - merged_sell[‘Price_open’]) / merged_sell[‘Price_open’]) * 100

buy_price_diff_merged.mean(), sell_price_diff_merged.mean()

Jim

[Jrinne]

BTW, same sim run on Friday (WeekDay = 6). People can draw their own inferences which may or may not apply to their models:

But maybe consistent with what Yuval has said, that as a general rule there is not much change in the prices during the day.

Jim

[AlgoMan]

You will not get the exact same transactions if the account size differs over time, that’s why I made the spreadsheet, to only compare the transactions that matches.

[geov]

@AlgoMan

I made a spreadsheet to compare sell and buy transactions

I have used your spreadsheet on a model with 100 positions of the S&P 500. The model has over 21,000 transactions. Your sheet returns that it is best to buy at the next open and sell at the next close.

BUY transaction difference
0.07%	If positive, better to buy in the open	If negative, better to buy at Next Close
SELL transaction difference
0.04%	If negative, better to sell in the open	If Positive, better sell at next close

[AlgoMan]

Is that trades on Mondays or any day?

[geov]

All trades on 1st trading day of the week.