Data Differences Between trading_calendars and pandas_market_calendars

[gerrymanoim]

re rsheftel/pandas_market_calendars#41

Summary status

• XHKG
  • Data Differences Between trading_calendars and pandas_market_calendars #152 (comment)
• CFE
  • Data Differences Between trading_calendars and pandas_market_calendars #152 (comment)
  • Note: We don't know what to do about 1994-04-27 in pmc but not in tc
• XSWX
• LSE
  • Issue: LSE special holidays #163
  • PR: ENH: Update XLON calendars #166
• XSHG
• XASX
• TSX
  • Data Differences Between trading_calendars and pandas_market_calendars #152 (comment))
• BMF

Extras not covered by diff below:

• Pre 1990 NYSE holidays
  • NYSE Calendar differences prior to 1990 #162

Details

code generating the table below

from collections import namedtuple 

import pandas_market_calendars as pmc
import trading_calendars as tc

# todo port this back to trading calendars
def get_tc_calendar_names():
  return list(tc.calendar_utils._default_calendar_factories.keys())+list(tc.calendar_utils._default_calendar_aliases.keys())

tc.get_calendar_names = get_tc_calendar_names
# end todo

pmc_calendars = set(pmc.get_calendar_names())
tc_calendars = set(tc.get_calendar_names())
tc_to_pmc_mapping = {
    'XASX': 'ASX',
    'XHKG': 'HKEX',
    'XTKS': 'JPX',
    'XOSL': 'OSE',
    'XSWX': 'SIX',
    'XSHG': 'SSE',
}

in_both = (tc_calendars & pmc_calendars) | set(tc_to_pmc_mapping.keys())

CompareResult = namedtuple("Result", ["only_tc", "only_pmc"])

def compare_calendars(cal: str):
  tc_cal = tc.get_calendar(cal).all_sessions
  pmc_cal_name = tc_to_pmc_mapping.get(cal, cal)
  pmc_cal = pmc.get_calendar(pmc_cal_name).valid_days(min(tc_cal), max(tc_cal))
  if not (tc_cal ^ pmc_cal).empty:
    return CompareResult

out = {}
for cal in in_both:
  diff = compare_calendars(cal)
  if diff:
    out[cal] = diff

# everything below this is markdown formatting
out_str = []
for cal, result in out.items():
  out_str.append(f"## {cal} \n")
  out_str.append("Trading Days only in `trading_calendars` \n")
  out_str.extend([f"- {item} \n" for item in result.only_tc])
  out_str.append("\n")
  out_str.append("Trading Days only in `pandas_market_calendars` \n")
  out_str.extend([f"- {item} \n" for item in result.only_pmc])
  out_str.append("\n")
md = "".join(out_str); print(md)

---

XHKG

Trading Days only in trading_calendars

• 1990-06-18 00:00:00+00:00
• 1990-08-27 00:00:00+00:00
• 1991-02-14 00:00:00+00:00
• 1991-06-18 00:00:00+00:00
• 1991-08-26 00:00:00+00:00
• 1992-06-15 00:00:00+00:00
• 1992-07-22 00:00:00+00:00
• 1992-08-31 00:00:00+00:00
• 1993-06-14 00:00:00+00:00
• 1993-08-30 00:00:00+00:00
• 1993-09-17 00:00:00+00:00
• 1994-06-14 00:00:00+00:00
• 1994-08-29 00:00:00+00:00
• 1995-06-19 00:00:00+00:00
• 1995-08-28 00:00:00+00:00
• 1996-06-17 00:00:00+00:00
• 1996-08-26 00:00:00+00:00
• 1997-02-06 00:00:00+00:00
• 1997-06-30 00:00:00+00:00
• 1997-07-02 00:00:00+00:00
• 1997-08-18 00:00:00+00:00
• 1997-10-02 00:00:00+00:00
• 1998-08-17 00:00:00+00:00
• 1998-10-02 00:00:00+00:00
• 1999-09-16 00:00:00+00:00
• 1999-12-31 00:00:00+00:00
• 2001-07-06 00:00:00+00:00
• 2001-07-25 00:00:00+00:00

Trading Days only in pandas_market_calendars

• 1990-05-01 00:00:00+00:00
• 1990-05-02 00:00:00+00:00
• 1990-07-02 00:00:00+00:00
• 1990-10-01 00:00:00+00:00
• 1991-05-01 00:00:00+00:00
• 1991-05-21 00:00:00+00:00
• 1991-07-01 00:00:00+00:00
• 1991-10-01 00:00:00+00:00
• 1992-05-01 00:00:00+00:00
• 1992-05-11 00:00:00+00:00
• 1992-07-01 00:00:00+00:00
• 1992-10-01 00:00:00+00:00
• 1993-05-28 00:00:00+00:00
• 1993-07-01 00:00:00+00:00
• 1994-05-02 00:00:00+00:00
• 1994-05-18 00:00:00+00:00
• 1994-07-01 00:00:00+00:00
• 1995-05-01 00:00:00+00:00
• 1995-05-08 00:00:00+00:00
• 1995-10-02 00:00:00+00:00
• 1996-05-01 00:00:00+00:00
• 1996-05-24 00:00:00+00:00
• 1996-07-01 00:00:00+00:00
• 1996-10-01 00:00:00+00:00
• 1997-05-01 00:00:00+00:00
• 1997-05-14 00:00:00+00:00
• 1998-05-01 00:00:00+00:00
• 1998-05-04 00:00:00+00:00
• 2021-04-06 00:00:00+00:00

CFE

Trading Days only in trading_calendars

Trading Days only in pandas_market_calendars

• 1994-04-27 00:00:00+00:00
• 2004-06-11 00:00:00+00:00
• 2007-01-02 00:00:00+00:00
• 2012-10-29 00:00:00+00:00
• 2012-10-30 00:00:00+00:00
• 2018-12-05 00:00:00+00:00

XSWX

Trading Days only in trading_calendars

• 1994-12-30 00:00:00+00:00
• 1995-12-29 00:00:00+00:00
• 2000-12-29 00:00:00+00:00
• 2005-12-30 00:00:00+00:00
• 2006-12-29 00:00:00+00:00
• 2011-12-30 00:00:00+00:00
• 2016-12-30 00:00:00+00:00
• 2017-12-29 00:00:00+00:00

Trading Days only in pandas_market_calendars

LSE

Trading Days only in trading_calendars

• 1995-05-08 00:00:00+00:00
• 1999-12-31 00:00:00+00:00

Trading Days only in pandas_market_calendars

• 1995-05-01 00:00:00+00:00

XSHG

Trading Days only in trading_calendars

• 2020-01-31 00:00:00+00:00
• 2022-01-31 00:00:00+00:00
• 2022-04-04 00:00:00+00:00
• 2022-05-02 00:00:00+00:00
• 2022-09-12 00:00:00+00:00
• 2022-10-06 00:00:00+00:00
• 2022-10-07 00:00:00+00:00
• 2023-06-23 00:00:00+00:00
• 2023-10-06 00:00:00+00:00
• 2024-02-09 00:00:00+00:00
• 2024-04-05 00:00:00+00:00
• 2024-09-16 00:00:00+00:00
• 2024-10-07 00:00:00+00:00
• 2025-01-28 00:00:00+00:00
• 2025-05-02 00:00:00+00:00
• 2025-06-02 00:00:00+00:00
• 2025-10-06 00:00:00+00:00
• 2025-10-07 00:00:00+00:00

Trading Days only in pandas_market_calendars

• 2020-10-08 00:00:00+00:00

XASX

Trading Days only in trading_calendars

• 1993-01-27 00:00:00+00:00
• 1994-01-27 00:00:00+00:00
• 1995-01-27 00:00:00+00:00
• 1996-01-29 00:00:00+00:00
• 1998-01-27 00:00:00+00:00
• 1999-01-27 00:00:00+00:00
• 2000-01-27 00:00:00+00:00
• 2001-01-29 00:00:00+00:00
• 2004-01-27 00:00:00+00:00
• 2005-01-27 00:00:00+00:00
• 2006-01-27 00:00:00+00:00
• 2007-01-29 00:00:00+00:00
• 2009-01-27 00:00:00+00:00
• 2010-01-27 00:00:00+00:00
• 2011-01-27 00:00:00+00:00
• 2012-01-27 00:00:00+00:00
• 2015-01-27 00:00:00+00:00
• 2016-01-27 00:00:00+00:00
• 2017-01-27 00:00:00+00:00
• 2018-01-29 00:00:00+00:00
• 2021-01-27 00:00:00+00:00

Trading Days only in pandas_market_calendars

• 1990-12-31 00:00:00+00:00
• 1994-01-26 00:00:00+00:00
• 1994-12-27 00:00:00+00:00
• 1995-01-26 00:00:00+00:00
• 1996-01-26 00:00:00+00:00
• 1998-01-26 00:00:00+00:00
• 1999-01-26 00:00:00+00:00
• 1999-12-31 00:00:00+00:00
• 2000-01-26 00:00:00+00:00
• 2001-01-26 00:00:00+00:00
• 2004-01-26 00:00:00+00:00
• 2005-01-26 00:00:00+00:00
• 2005-12-27 00:00:00+00:00
• 2006-01-26 00:00:00+00:00
• 2007-01-26 00:00:00+00:00
• 2009-01-26 00:00:00+00:00
• 2010-01-26 00:00:00+00:00
• 2010-04-26 00:00:00+00:00
• 2011-01-26 00:00:00+00:00
• 2011-04-26 00:00:00+00:00
• 2011-12-27 00:00:00+00:00
• 2012-01-26 00:00:00+00:00
• 2015-01-26 00:00:00+00:00
• 2016-01-26 00:00:00+00:00
• 2016-12-27 00:00:00+00:00
• 2017-01-26 00:00:00+00:00
• 2018-01-26 00:00:00+00:00
• 2021-01-26 00:00:00+00:00

TSX

Trading Days only in trading_calendars

• 1993-12-24 00:00:00+00:00
• 1999-12-24 00:00:00+00:00
• 2004-12-24 00:00:00+00:00
• 2010-12-24 00:00:00+00:00

Trading Days only in pandas_market_calendars

• 2001-09-11 00:00:00+00:00
• 2001-09-12 00:00:00+00:00

BMF

Trading Days only in trading_calendars

• 1997-07-09 00:00:00+00:00
• 2020-07-09 00:00:00+00:00

Trading Days only in pandas_market_calendars

• 1995-12-29 00:00:00+00:00
• 2000-12-29 00:00:00+00:00
• 2006-12-29 00:00:00+00:00
• 2014-06-12 00:00:00+00:00
• 2017-12-29 00:00:00+00:00

[gerrymanoim]

cc @jmccorriston (FYI - @rsheftel )

[jmccorriston]

For TSX:

I can't find any sources saying that Christmas Eve was a holiday, but I found this source suggesting that the market was open on 2010-12-24 (but closed by 2:41pm ET, suggesting it closed early): https://toronto.ctvnews.ca/modest-tsx-gain-as-trading-takes-a-christmas-break-1.589362

There's also this wiki page on the 9/11 closes around the world (includes TSX): https://en.wikipedia.org/wiki/Closings_and_cancellations_following_the_September_11_attacks

So I think the Toronto Stock exchange implementation in trading_calendars is correct.

[gerrymanoim]

FWIW https://finance.yahoo.com/quote/RY.TO/history?period1=1292112000&period2=1293753600&interval=1d&filter=history&frequency=1d shows volume on TSX on 2010-12-24.

[jmccorriston]

For HKG:

It looks like Hong Kong only started observing Labour Day in 1999 (https://en.wikipedia.org/wiki/International_Workers%27_Day, "In Hong Kong, 1 May is known as Labour Day and has been considered a public holiday since 1999."). pmc is correct here, tc needs to be updated.

Establishment day started in 1997: https://www.scmp.com/news/hong-kong/education-community/article/1983718/everything-you-need-know-about-hong-kongs-return. pmc is correct here, tc needs to be updated.

National Day also first observed in 1997: http://www.cnn.com/WORLD/9710/01/china.bday/. pmc is correct, tc needs to be updated.

Buddha's birthday also only started as a public holiday in 1998: https://www.cantoneseclass101.com/blog/2020/04/24/celebrating-buddhas-birthday-in-hong-kong/, https://www.officeholidays.com/holidays/hong-kong/the-buddhas-birthday. We should trim the calendar in tc to reflect this. However, it looks like there's still a discrepancy in 1998. pmc suggests that 1998-05-04 should be a trading day while tc suggests it should be a holiday (1998-05-03 actual holiday, observed on the Monday).

There are several British holidays that were observed during British rule pre-1997 as well:

• Liberation Day: https://en.wikipedia.org/wiki/Liberation_Day_(Hong_Kong) (Last Monday of August, stopped observing in 1997.
• Queen's Bday: https://en.wikipedia.org/wiki/Public_holidays_in_Hong_Kong#:~:text=Under%20the%20administration%20of%20the,the%20People's%20Republic%20of%20China.
• Potentially August Bank holiday?

It looks like there are a lot of details in the pmc Hong Kong calendar implementation that we are missing in tc. I pinged the person who added them to pmc to see if we can get more info, but my inclination is to pull in the historical calendars that they added to pmc. From what I can tell so far, they are correct.

One exception: 2021-04-06 was a public holiday per https://www.hkex.com.hk/News/HKEX-Calendar?sc_lang=en (Ching Ming festival). This is correct in tc.

[jmccorriston]

For CFE:

• Was closed on 2004-06-11 for National Day of Mourning (https://www.isda.org/2004/06/11/u-s-exchange-and-market-closings-on-june-11-2004-for-mourning-president-ronald-reagan/). tc is correct here.
• Was closed on 2007-01-02 for National Day of Mourning (https://www.isda.org/2007/01/02/u-s-exchange-closings-on-january-2-2007-for-mourning-president-gerald-ford/#:~:text=(ISDA)%20announces%20the%20following%20guidance,former%20U.S.%20President%20Gerald%20Ford.). tc is correct here.
• Was closed on 2018-12-05 for National Day of Mourning (https://www.prnewswire.com/news-releases/cboe-global-markets-exchanges-to-close-wednesday-for-day-of-mourning-to-honor-president-george-hw-bush-300758638.html#:~:text=1%2C%202018%20%2FPRNewswire%2F%20%2D%2D,who%20passed%20away%20November%2030.). tc is correct here.
• Was closed on 2012-10-29 and 2012-10-30 for Hurricane Sandy (http://ir.cboe.com/press-releases/2012/29-oct-2012a). tc is correct here.

I can't seem to find any references for Nixon's Day of Mourning on 1994-04-27... Not sure what to do about this one.

[rsheftel]

Discussion of pre-1990 NYSE (XNYS) calendar here: #162

[rsheftel]

Discussion of LSE differences here: #163

[rsheftel]

Discussion of XKTS (JPX) differences here: #164