Hi all, this post is going to be a relatively short and to the point run through of creating an annotated heatmap for the Dow 30 stock returns using the Python Seaborn package.
Let’s start with what is a heatmap actually is; it’s defined as “a representation of data in the form of a map or diagram in which data values are represented as colours.”
This makes it a great tool to quickly visualise the magnitude of stock returns over time in a matrix/grid format, using a colour map/scale to represent the size and direction of each stock’s percentage change over that period of time.
Creating a heatmap without stock ticker labels annotated, i.e. a heatmap annotated with just the numerical value of the relevant cell is a very easy process, thanks to the power and ease of use of Seaborn.
It can be achieved as follows:
#import relevant modules
import pandas as pd
import numpy as np
import pandas_datareader as data
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline
#create a small function that we can feed a list of stock tickers to to download pricing data
def get_prices(tickers,start):
prices = data.DataReader(tickers,'yahoo',start=start)['Adj Close']
return prices
#set the url that we will scrape the Dow 30 ticker information from
dow = 'https://www.cnbc.com/dow-components/'
#read in the url and scrape ticker data
data_table = pd.read_html(dow)
#convert ticker column to list
tickers = data_table[:][0]['Symbol'].tolist()
#download daily pricing data for each ticker in the Dow 30
prices = get_prices(tickers,'01/01/2017')
#Calculate percentage return over download period
returns = (((prices.iloc[-1] / prices.iloc[0]) - 1) * 100).round(2)
The above code gets the data we actually need to populate the heatmap, with the actual heatmap creating being as easy as folows:
fig, ax = plt.subplots(figsize=(14,9))
plt.title('Dow 30 Heat Map',fontsize=18)
ax.title.set_position([0.5,1.05])
ax.set_xticks([])
sns.heatmap(per_change, annot=True, fmt="", cmap='RdYlGn', ax=ax)
plt.show()
This creates the following:
The above is all well and good, but we can’t actually see which cell relates to which individual stock! Not ideal.
The (slightly) tricky part is going to be creating the label arrays to actually annotate the heatmap with the relevant cell’s stock ticker information.
This can be added as follows:
#create a reshaped array of ticker symbols that matches the desired shape of the heatmap
symbol = ((np.asarray(returns.index)).reshape(6,5))
#create a reshaped array of percent returns that matches the desired shape of the heatmap
per_change = ((np.asarray(returns)).reshape(6,5))
#create a new array of the same shape as desired, combining the relevant ticker symbol
#and percentage return data
labels = (np.asarray(["{0} \n {1:.3f}".format(symbol, per_change)
for symbol, per_change in zip(symbol.flatten(),
per_change.flatten())])).reshape(6,5)
Create the new heatmap, this time using the “annot” call to use our newly created “labels” list to annotate it.
fig, ax = plt.subplots(figsize=(14,9))
plt.title('Dow 30 Heat Map',fontsize=18)
ax.title.set_position([0.5,1.05])
ax.set_xticks([])
sns.heatmap(per_change, annot=labels, fmt="", cmap='RdYlGn', ax=ax)
plt.show()
We now have the following:
A nicely annotated heatmap showing both the returns and stock ticker relevant to each of the Dow 30 stocks.
I’ll leave this post here, and as always – any questions or comments just leave them below.
Cheers!
8 comments
Hi Stuart!
Hope all is well. I am trying to replicate this code but using a CSV instead since yahoo and google no longer work. I was wondering if you can check why my code may not be running? The output is as follows:
“`
import pandas as pd
import numpy as np
import monthly_returns_heatmap as mrh
import matplotlib.pyplot as plt
import seaborn as sns
df = pd.read_csv(‘C:\\Users\\sam\\PycharmProjects\\PycharmCodes\\Fiverr Time Series Returns AAPL NFLX INTC\\aapl_nflx_intc_close_prices.csv’)
# print first 5 elements in our data frame (which is the first 5 close prices for AAPl, NFLX, and INTC)
print(df.head())
# calculate monthly returns
# compute aapl monthly returns
df[‘aapl_monthly_returns’] = np.log(df[‘AAPL’] / df[‘AAPL’].shift(20))
# compute nflx monthly returns
df[‘nflx_monthly_returns’] = np.log(df[‘NFLX’] / df[‘NFLX’].shift(20))
# compute intc monthly returns
df[‘intc_monthly_returns’] = np.log(df[‘INTC’] / df[‘INTC’].shift(20))
# HEATMAP TO COMPARE MONTHLY RESULTS
# extract columns monthly returns columns from dataframe, which relate to apple, nflx, and intc monthly returns
symbol = ((np.asarray(df.loc[:,’aapl_monthly_returns’:’intc_monthly_returns’].index)))
# create a reshaped array of percent returns that matches the desired shape of the heatmap
per_change = ((np.asarray(df.loc[:,’aapl_monthly_returns’:’intc_monthly_returns’])))
# create a new array of the same shape as desired, combining the relevant ticker symbol
# and percentage return data
labels = (np.asarray([“{0} \n {1:.3f}”.format(symbol, per_change)
for symbol, per_change in zip(symbol.flatten(),
per_change.flatten())])).reshape(2630)
fig, ax = plt.subplots(figsize=(14,9))
plt.title(‘Dow 30 Heat Map’,fontsize=18)
ax.title.set_position([0.5,1.05])
ax.set_xticks([])
sns.heatmap(per_change, annot=labels, fmt=””, cmap=’RdYlGn’, ax=ax)
plt.show()
“`
Thanks in advance!
Sam
It’s hard to investigate without the input files and without seeing your error message. Maybe contact me via email as previously.
Stuart, For reference, the 3 monthly returns columns for aapl_monthly_returns, intc_monthly_returns, and nfxl_monthly returns are all 2630 rows in length
And the first column in the the dataframe that they are all in is ‘Date’ in format: ‘1/2/2008’ until ‘6/13/2018’
Hey mate, There is something missing in your code.. “per_change” is not defined?
Sorry – its supposed to be “pct_change”
Hi Sir,
I was trying to replicate but the function doesnt work i guess. It gives the error
ValueError Traceback (most recent call last)
Cell In[3], line 4
2 dow = ‘https://www.cnbc.com/dow-components/’
3 #read in the url and scrape ticker data
—-> 4 data_table = pd.read_html(dow)
5 #convert ticker column to list
6 tickers = data_table[:][0][‘Symbol’].tolist()
File ~/anaconda3/lib/python3.10/site-packages/pandas/io/html.py:1212, in read_html(io, match, flavor, header, index_col, skiprows, attrs, parse_dates, thousands, encoding, decimal, converters, na_values, keep_default_na, displayed_only, extract_links, dtype_backend)
1208 check_dtype_backend(dtype_backend)
1210 io = stringify_path(io)
-> 1212 return _parse(
1213 flavor=flavor,
1214 io=io,
1215 match=match,
1216 header=header,
1217 index_col=index_col,
1218 skiprows=skiprows,
1219 parse_dates=parse_dates,
1220 thousands=thousands,
1221 attrs=attrs,
1222 encoding=encoding,
1223 decimal=decimal,
1224 converters=converters,
1225 na_values=na_values,
1226 keep_default_na=keep_default_na,
1227 displayed_only=displayed_only,
1228 extract_links=extract_links,
1229 dtype_backend=dtype_backend,
1230 )
File ~/anaconda3/lib/python3.10/site-packages/pandas/io/html.py:1001, in _parse(flavor, io, match, attrs, encoding, displayed_only, extract_links, **kwargs)
999 else:
1000 assert retained is not None # for mypy
-> 1001 raise retained
1003 ret = []
1004 for table in tables:
File ~/anaconda3/lib/python3.10/site-packages/pandas/io/html.py:981, in _parse(flavor, io, match, attrs, encoding, displayed_only, extract_links, **kwargs)
978 p = parser(io, compiled_match, attrs, encoding, displayed_only, extract_links)
980 try:
–> 981 tables = p.parse_tables()
982 except ValueError as caught:
983 # if
iois an io-like object, check if it’s seekable984 # and try to rewind it before trying the next parser
985 if hasattr(io, “seekable”) and io.seekable():
File ~/anaconda3/lib/python3.10/site-packages/pandas/io/html.py:257, in _HtmlFrameParser.parse_tables(self)
249 def parse_tables(self):
250 “””
251 Parse and return all tables from the DOM.
252
(…)
255 list of parsed (header, body, footer) tuples from tables.
256 “””
–> 257 tables = self._parse_tables(self._build_doc(), self.match, self.attrs)
258 return (self._parse_thead_tbody_tfoot(table) for table in tables)
File ~/anaconda3/lib/python3.10/site-packages/pandas/io/html.py:629, in _BeautifulSoupHtml5LibFrameParser._parse_tables(self, doc, match, attrs)
626 unique_tables.add(table)
628 if not result:
–> 629 raise ValueError(f”No tables found matching pattern {repr(match.pattern)}”)
630 return result
ValueError: No tables found matching pattern ‘.+’
Hi there – thanks for your comment. Please find a full working version of the code below:
#import relevant modules
import pandas as pd
import numpy as np
from pandas_datareader import data as pdr
import yfinance as yf
yf.pdr_override()
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline
#create a small function that we can feed a list of stock tickers to to download pricing data
def get_prices(tickers,start):
prices = pdr.get_data_yahoo(tickers,start=start)['Adj Close']
return prices
#set the url that we will scrape the Dow 30 ticker information from
dow = "https://www.dividendmax.com/market-index-constituents/dow-jones-30"
#read in the url and scrape ticker data
data_table = pd.read_html(dow,displayed_only=False)
#convert ticker column to list
tickers = data_table[0]['Ticker'].tolist()
#download daily pricing data for each ticker in the Dow 30
prices = get_prices(tickers,start='2017-01-01')
prices.dropna(inplace=True)
#Calculate percentage return over download period
returns = (((prices.iloc[-1] / prices.iloc[0]) - 1) * 100).round(2)
#create a reshaped array of ticker symbols that matches the desired shape of the heatmap
symbol = ((np.asarray(returns.index)).reshape(6,5))
#create a reshaped array of percent returns that matches the desired shape of the heatmap
per_change = ((np.asarray(returns)).reshape(6,5))
#create a new array of the same shape as desired, combining the relevant ticker symbol
#and percentage return data
labels = (np.asarray(["{0} \n {1:.3f}".format(symbol, per_change)
for symbol, per_change in zip(symbol.flatten(),
per_change.flatten())])).reshape(6,5)
fig, ax = plt.subplots(figsize=(14,9))
plt.title('Dow 30 Heat Map',fontsize=18)
ax.title.set_position([0.5,1.05])
ax.set_xticks([])
sns.heatmap(per_change, annot=labels, fmt="", cmap='RdYlGn', ax=ax)
plt.show()