manually fix incorrect times

[mzelinka]

I am trying to read in a file that apparently has incorrect times. Is it possible for me to define what I believe to be the correct times and assign those corrected times to this dataset?

If I read this in with decoding:

ds = xc.open_mfdataset('/p/user_pub/climate_work/zelinka1/pattern-effectMIP/Vaccaro_et_al_2021/vaccaro_et_al_2021_T63_bil_absolute_TS_values01.nc')

ds.time:

array([cftime.DatetimeGregorian(3700, 1, 17, 12, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeGregorian(3700, 2, 16, 0, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeGregorian(3700, 3, 17, 12, 0, 0, 0, has_year_zero=False),
       ...,
       cftime.DatetimeGregorian(3867, 10, 18, 12, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeGregorian(3867, 11, 18, 0, 0, 0, 0, has_year_zero=False),
       cftime.DatetimeGregorian(3867, 12, 18, 12, 0, 0, 0, has_year_zero=False)],
      dtype=object)

Reading it in without decoding:

ds = xc.open_mfdataset(
    '/p/user_pub/climate_work/zelinka1/pattern-effectMIP/Vaccaro_et_al_2021/vaccaro_et_al_2021_T63_bil_absolute_TS_values01.nc', 
    decode_times = False)

ds.time:

array([675715.5, 675745. , 675774.5, ..., 736984.5, 737015. , 737045.5],
      dtype=float32)

units are days since 1850-01-01

Clearly this should instead be days since 0000-01-01 (if that is possible) since (675715.5/365.25) = 1850 whereas 1850 + (675715.5/365.25) = 3700 as we are seeing in the decoded times. Is there a good way to override and apply the correct times to a dataset?

Thanks.

[pochedls]

@mzelinka - there is some ambiguity here (read about has_year_zero) – I think this should be a 360_day or proleptic_gregorian calendar? I think the snippet below might solve the issue, though (since you know what you want to happen).

# import xarray which has encoding / decoding functionality
import xarray
# define / open dataset
fn = '/p/user_pub/climate_work/zelinka1/pattern-effectMIP/Vaccaro_et_al_2021/vaccaro_et_al_2021_T63_bil_absolute_TS_values01.nc'
ds = xc.open_dataset(fn, decode_times=False) 
# decode options (first one is probably what you want?)
time = xarray.coding.times.decode_cf_datetime(ds.time, 'day since 0000-01-01', use_cftime=True, calendar='proleptic_gregorian')
time_360 = xarray.coding.times.decode_cf_datetime(ds.time, 'day since 0000-01-01', use_cftime=True, calendar='360_day')
time_standard = xarray.coding.times.decode_cf_datetime(ds.time, 'day since 0001-01-01', use_cftime=True)
# put in dataset
ds['time'] = time

[pochedls]

One other option that comes to mind (via pandas):

import pandas as pd
time = pd.date_range(start="1850-01-15", end="2017-12-31", freq='M')
ds['time'] = time

This yields numpy.Datetime64 time objects, which I think should work fine with xcdat utilities.

[mzelinka]

Yep, that one line is all I needed. Thanks!

[lee1043]

@pochedls that is a great tip, thank you for sharing!

[tomvothecoder]

Xarray provides xr.date_range which has support for cftime objects (use_cftime) and calendar type (calendar).