Notebook

Hypothesis:

SPY and SHY z-score-normalized dollar volume difference plotted as a heat map can be used to detect COVID-19 global pandemic related shifts in market regimes

Discussion:

Extreme relative dollar volume movements in SPY and SHY, both intraday and near end-of-day, indicate a COVID-19 pandemic related shift in the market regime, relative to the earlier period of 2020, when the economic impact of the pandemic had not yet been factored into the market. The data also suggest that U.S. government market interventions may play a role in driving changes in market regimes (further investigation is required to reveal explicit correlations, e.g. passage of a major economic stimulus package by Congress, and Federal Reserve actions).

In [1]:
from pytz import timezone
import matplotlib.pyplot as plt
import pandas as pd

In [2]:
# get minute bar price & volume data for SPY & SHY & compute dollar volume
# assumption is that empty bars can be filled with zeros, corresponding to no trading volume
price = get_pricing(['SPY','SHY'],start_date='2018-12-18',end_date='2020-04-15',fields='price',frequency='minute')
volume = get_pricing(['SPY','SHY'],start_date='2018-12-18',end_date='2020-04-15',fields='volume',frequency='minute').fillna(0)
data = price*volume
data.tail(3)

Out[2]:
Equity(8554 [SPY]) Equity(23911 [SHY])
2020-04-15 19:58:00+00:00 9.646300e+07 2043647.365
2020-04-15 19:59:00+00:00 2.011489e+08 2425462.662
2020-04-15 20:00:00+00:00 3.356134e+08 2800677.880
In [3]:
# add time & date columns to the data, for pivot table
data['time'] = data.index.tz_convert(timezone('US/Eastern')).time
data['date'] = data.index.date
data.tail(3)

Out[3]:
Equity(8554 [SPY]) Equity(23911 [SHY]) time date
2020-04-15 19:58:00+00:00 9.646300e+07 2043647.365 15:58:00 2020-04-15
2020-04-15 19:59:00+00:00 2.011489e+08 2425462.662 15:59:00 2020-04-15
2020-04-15 20:00:00+00:00 3.356134e+08 2800677.880 16:00:00 2020-04-15
In [4]:
# on a rolling basis, compute the difference in dollar volume z-scores using a minutely trailing window
# credit to Quantopian user Michael Van Kleeck for this efficient code
# ref.: https://www.quantopian.com/posts/modified-heatmap-based-on-grant-kiehnes-example
WINDOW_BASELINE = 260*390
WINDOW = 390
dollar_volumes = data.iloc[:, 0:2]
means = dollar_volumes.apply(lambda x: pd.rolling_mean(x, window=WINDOW_BASELINE))
sds = dollar_volumes.apply(lambda x: pd.rolling_std(x, window=WINDOW_BASELINE))
dv = dollar_volumes.apply(lambda x: pd.rolling_mean(x,window=WINDOW))
zs = ((dv - means) / sds)
data['z_diff'] = zs.iloc[:, 1] - zs.iloc[:, 0] # DV_SHY - DV_SPY (where DV is the z-score normalized dollar volume)
data.tail(3)

/venvs/py35/lib/python3.5/site-packages/ipykernel_launcher.py:7: FutureWarning: pd.rolling_mean is deprecated for Series and will be removed in a future version, replace with
Series.rolling(center=False,window=101400).mean()
import sys
/venvs/py35/lib/python3.5/site-packages/ipykernel_launcher.py:8: FutureWarning: pd.rolling_std is deprecated for Series and will be removed in a future version, replace with
Series.rolling(center=False,window=101400).std()

/venvs/py35/lib/python3.5/site-packages/ipykernel_launcher.py:9: FutureWarning: pd.rolling_mean is deprecated for Series and will be removed in a future version, replace with
Series.rolling(center=False,window=390).mean()
if __name__ == '__main__':

Out[4]:
Equity(8554 [SPY]) Equity(23911 [SHY]) time date z_diff
2020-04-15 19:58:00+00:00 9.646300e+07 2043647.365 15:58:00 2020-04-15 -0.190347
2020-04-15 19:59:00+00:00 2.011489e+08 2425462.662 15:59:00 2020-04-15 -0.188791
2020-04-15 20:00:00+00:00 3.356134e+08 2800677.880 16:00:00 2020-04-15 -0.201223
In [5]:
# apply a pivot table, with trading day along the vertical & trading time along the horizontal
# fill empty minutes with zeros
ht_map = pd.pivot_table(data.dropna(),'z_diff',index=data['date'], columns=data['time'],fill_value=0)

Out[5]:
time 09:31:00 09:32:00 09:33:00 09:34:00 09:35:00 09:36:00 09:37:00 09:38:00 09:39:00 09:40:00 ... 15:51:00 15:52:00 15:53:00 15:54:00 15:55:00 15:56:00 15:57:00 15:58:00 15:59:00 16:00:00
date
2020-01-02 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 ... 0.075536 0.072645 0.070868 0.069149 0.080360 0.080714 0.081550 0.081627 0.079980 0.083495
2020-01-03 0.079450 0.073163 0.068872 0.068119 0.064576 0.063314 0.063303 0.061802 0.061733 0.061492 ... -0.052615 -0.056775 -0.059131 -0.054206 -0.062841 -0.058439 -0.054516 -0.054224 -0.054934 -0.065722
2020-01-06 -0.063902 -0.062022 -0.062930 -0.061531 -0.058221 -0.053802 -0.052982 -0.050551 -0.050907 -0.049717 ... 0.074308 0.073071 0.073616 0.068305 0.063599 0.061883 0.060479 0.058415 0.061409 0.086364

3 rows Ã— 390 columns

In [6]:
# plot heat map of z-score difference
# lighter color means z_diff > 0 (DV_SHY > DV_SPY) & darker color mean z_diff < 0 (DV_SHY < DV_SPY)
# see http://matplotlib.org/examples/color/colormaps_reference.html to switch colormap
plt.imshow(ht_map,cmap='plasma',aspect=5)
plt.grid(False)
plt.colorbar()
plt.clim(ht_map.min().min(),ht_map.max().max())
plt.title('SPY & SHY z-score-normalized minutely dollar volume difference, 1/2/2020-present')

<matplotlib.text.Text at 0x7fb9e5fc5748>