Charts not displayed when mouse over the [5D][1Y] next to tickers

Hi Georg,

Using some coding language (VBA in my case), I create a link that accesses directly the txt file version of the historical data I need. Then I can download the txt file and do whatever I have to do with it

Example: http://www.google.com/finance/historical?q=NASDAQ%3AMERC&startdate=May+19+2017&enddate=May+19+2017&output=csv

Couple of further helpful hints:

  • Note the startdate and enddate in the link → you can choose to download the data only between specific dates
  • Note the Ticker structure “exchange%3Aticker” (e.g. NASDAQ%3AMERC for the ticker MERC on the Nasdaq)
  • You can often get away with just MERC i.e. sth like http://www.google.com/finance/historical?q=MERC&startdate=May+19+2017&enddate=May+19+2017&output=csv
    -… but in this case (and it is the first time this happened to me this week in 2.5 years of using this) → Google will bring up a MERC ticker on the LONDON stock exchange (nothing to do with the Nasdaq one). So it is safer to use the exchange if possible

NB: for a txt file with all the tickers on US exchanges → http://www.finra.org/industry/oats/oats-reportable-securities-list
There are 2 files to choose from at the bottom of that page

NB: Yahoo had a similar http structure for direct txt file access but it stopped working this week.

I hope this helps,

Jerome

Georg,

There was an article that I found via Quantocracy that explains what the problem is and how to fix it:

http://tradingwithpython.blogspot.com/2017/05/yahoo-is-dead-long-live-yahoo.html

This references an article that is on GitHub:

https://github.com/sjev/trading-with-python/blob/master/scratch/get_yahoo_data.ipynb

Whatever Verizon has in store for Yahoo (+ AOL), I am not sure that it will look much like it did before, or have the features that people have used for years. The “features” they have recently added over the past year have so degraded the performance of the finance portal that I have given up using it. I guess this is just one more nail in the yahoo coffin.

Simon

Jerome and Simon,
Thank you for your help in this. I don’t know why Yahoo Finance is stopping the automatic downloading of data. Why don’t they just charge a few dollars per month to make this available.

Best,
Georg

I gave up on Google long ago. Their data is spotty and error-prone. Plus, the quotes are unadjusted for dividends. I had been looking at them because I was trying to decide if I wanted to do some specific functions for them using my EXCEL add-in.

I’ve looked at them several times over the years. Last time I looked, I found a number of days with 0 volume, where O/H/L/C was set to the closing price of the day before.

Just checked SPY. The more recent data has no zero volume days, but 2012-01-13, 2011-12-30, and 2011-12-11 all do. Going back to 2001, there are 19 days with zero volume. So it looks like they may have improved a bit.

And, just going back to the start of 2014, they show a closing price of $182.92. The adjusted closing price should be about $171.35.

Their option data is even worse – missing expiration dates and contracts left and right. They don’t even have an “Option chain” link for SPY, and MMM only shows a single expiration date, while there are actually 12 of them.

For 2016 and 2017, I see three times where closing price differs from Yahoo and 99 times where volume differs. For example, 2/29/16 shows $193.35 vs $193.56 and 122448542 vs 125918100. Adjusted close should be $188.67. MarketWatch shows the Yahoo values, but it could just be that MarketWatch uses the same historical quotes provider.

BTW, for my add-in processing, I am creating the adjusted closing prices myself, using the dividend stream shown on Yahoo. They’re supposed to be fixing the data for dividend adjustments, but I won’t hold my breath. I needed to find an alternate way to grab prices from Yahoo after they obsoleted their old CSV API. If they ever fix the dividend adjustment, I’ll just unplug my dividend adjustment calculation.

I should mention Yahoo data is spotty right now as well. Indexes and ETFs appear to have days where they have zero or null values. But Yahoo does seem to be scrubbing the data. I also found a case, RNN, where they applied a 1-for-10 split as if it were a 10-for-1 split.

But I’m not sure we should even call it “Yahoo” or “Google” data. They must have a provider for their data. I know at one time, Yahoo charts did not agree with their historical quotes, because the data came from different providers – the charts were based on a provider that didn’t adjust for dividends. Well, some adjustments were made, but not all. :frowning:

For those of you that claim dividend adjustments are unnecessary, do you really want a dividend payment to trigger a technical buy or sell signal? Sure, dividends are usually small, but ignoring them is like ignoring stock splits. After all, stock splits don’t happen very often, right? I recall in 2008 when a lot of ETFs and funds paid out large distributions at year end, some more than 10%, there were a lot of complaints about big shifts in something as simple as a moving average.

For example, SDS had a $46 distribution in December of 2008 when the price was $350, about 13%. They also paid out about $16 three months earlier.

umh…

So are we saying that to get a feed with reliable EOD close prices, we have only as of today:

  • scrubbing P123 pages (but I thought the data was from Yahoo?)
  • the feed from our respective brokers (IB in my case via the API). But even there, I remember having a couple of occurrences in the last 2 years where the EOD close price was incorrect (still can’t get my head around how a broker can have an incorrect close price)

Nothing else?

Thx

Let me ask a more basic question. If P123 is paying Yahoo to provide those charts, why hasn’t Yahoo given them new URLs for the charts?

How about stockcharts.com as a provider for the charts? They have a number of smaller “Candle Glance” charts, with a variety of parameters. The parameters could probably even be customized by individual.

http://stockcharts.com/c-sc/sc?chart=MMM,uu[305,a]dacayaci[pb20!b50][da]
http://stockcharts.com/c-sc/sc?chart=MMM,uu[305,a]dacayaci[pb20!b50][db]
http://stockcharts.com/c-sc/sc?chart=MMM,uu[305,a]dacayaci[pb20!b50][dg]

[quote]
But I’m not sure we should even call it “Yahoo” or “Google” data. They must have a provider for their data.
[/quote]They disclose their data sources.

Yahoo gets US stock quotes direct from the exchanges. Historical data is from CSI. See here .

Google gets end of day prices from SIX Financial Information. Intra-day data may be provided by Interactive Data Real-Time Services, Inc. See here .

It looks like Finviz has now replaced the Yahoo thumbnail charts. Looks great for US stocks but does not work for Canadian stocks. Can the Canadian stocks be linked to stockcharts.com instead of Finviz?

Stockchart Terms of Use are against it. Yahoo wasn’t as long as clicking on chart took you to Yahoo.

Bottom line we need to do our own. We can do it for the 1Y for now, then once we start collecting delayed data we can do the short term one with intra-day.

I don’t think we need tick data like we had with Yahoo. Delayed price is enough to quickly check for red-flags during rebalance days, just like it is now with finviz. Agree ?

Agreed. Thanks Marco.

Just for perspective.

I aggregate my holdings then click Yahoo Quotes: this continues to work well with intraday quotes on all of my holdings (in a separate tab).

Obviously, this will not work seamlessly for a recommendation that I have not bought yet during the day.

But Yahoo is not completely broken and is adequate for my present needs.

-Jim

Thanks Marco!

Mine now works with Finviz charts :slight_smile:

I use XLQ to retrieve data from Yahoo, Google and IB. For those of you interested in retrieving data from Yahoo, this is what XLQ said in an email today regarding a patch they released to fix the issues with Yahoo data → “This version re-enables Yahoo historic downloads using their new method (requiring https cookie and crumb). Xlq will only connect for historic data if a valid cookie is returned, avoiding connection errors if any further changes occur. With the upcoming merger of Yahoo with AOL under the name Oath further breaking changes could be expected, and the site should be checked for updates.”

agree Marco!

Yahoo disabled the ability to download historical stock prices into Excel in May 2017.
Here is a link to a VBA compatible downloader for Yahoo Finance data from SignalSolver. They even re-arranged the columns to match with the old arrangement so it will neatly fit into your old excel data sheet layout.

http://www.signalsolver.com/download-historic-stock-price-data-excel/#SimpleSpreadsheet

The Yahoo data is still not very good. See this:

http://www.amibroker.com/kb/2017/06/30/wrong-close-price-in-yahoo-data/

Also, I had someone complain today about the new function I wrote for my EXCEL add-in, because some stocks weren’t passing through data in July yet (e.g. MTL and GFI, as well as 38 others). But when I look at the data now, 7/1 and 7/3 are there, but the 6/30 data fields all have null values. So they must be having trouble getting data loaded on those 40 (and probably other) tickers. At one point, I know a lot of ETFs had null values in their history.

With my add-in, I decided to pull the data out of the web page instead of doing the cookie and crumb method. It’s all in the source code of the web page, but in JSON format. It makes the web page huge. Up to several megabytes. The cookie and crumb methods also usually are set up to try up to 5 times to get the data, because they can get errors. I implemented one into my add-in and still have gotten errors with 5 tries.

When I run into days with all null values, I have my add-in carry forward the last close for that day’s O/H/L/C and set the volume to zero.

Also, last time I looked, the dividend adjustments on weekly and monthly data were done correct. They were accumulating the data weekly and monthly and then applying the dividend adjustment at the end of the period. That means a month-end close could be adjusted for a dividend that occurred earlier in the month. Yuck!

And take a look at the 1-for-10 stock split they applied to RNN in early May. LOL.

I have no doubt more changes with me coming…

Just a quick FYI:

Someone posed the question on SE on whether Yahoo! Finance data was still any good given the changes.

The response is basically “no”. It’s somewhat saddening to see such a great resource fall into disrepair.

Hi All,

What are people who want historical data doing?

I guess I am not “doing” anything that is a permanent solution—or even a complete solution. Recently, I signed up for XLQ for a trial period: as Dan and others have recommended (see above post by danparquette).

Using this yesterday for a small amount of data–to get the historical opening price–I had 3 data points that seemed to be in error from Yahoo and a couple additional data points that were technically correct but did not manage a split in my preferred manner for this data.

Comparing Google data to the Yahoo with XLQ and looking into the discrepancies helped. And Excel could, of course, be made to highlight the discrepancies. I am not sure that I will sign up for XLQ after the trial period.

Better ideas? Thanks.

-Jim

Quandl has lots of data and a standardized interface with a API for Excel and Python. Most of their data is free, but they make money by selling premium data.

Randy Harmelink (rharmelink) maintains a free Excel add-in that scrapes data from many sites.

Jim,

I’ve compiled a pretty exhaustive list of tools here: http://the-world-is.com/blog/resources/general-investor-resources/

Personally, I have historically used XLQ and the SMF Excel Add-in to good effect. I would probably search Quandl first now – it has expanded its offerings to numerous premium data sets which are ostensibly higher quality than free ones.

As an aside, XLQ used to default to Yahoo! Finance as a data source. If that’s what you want to avoid, then you might want to check the program’s preferences. It might have changed though.

//dpa