Skip to content
This repository has been archived by the owner on Jul 22, 2024. It is now read-only.

CSM BDS: findJobTimeRange.py is dependent on date+time format #995

Open
thanh-lam opened this issue Feb 25, 2021 · 3 comments
Open

CSM BDS: findJobTimeRange.py is dependent on date+time format #995

thanh-lam opened this issue Feb 25, 2021 · 3 comments

Comments

@thanh-lam
Copy link
Member

thanh-lam commented Feb 25, 2021

Describe the bug
The python script findJobTimeRange.py displays begin_time, history.end_time, and related info for an allocation. As described in #992, the date+time format is required to be updated to "yyyy-MM-ddThh:mm:ss.SSSZ". The problem is that findJobTimeRange.py defined this date format based on the old date string "yyyy-MM-dd hh:mm:ss.SSSZ". This caused a failure:

[csm04][/u/tlam/git/CAST/csm_big_data/python]> ./findJobTimeRange.py -a 259
# Found 1 matches for specified the job.
Traceback (most recent call last):
  File "./findJobTimeRange.py", line 149, in <module>
    sys.exit(main(sys.argv))
  File "./findJobTimeRange.py", line 121, in main
    start_time=datetime.strptime(tr_data["begin_time"], '%Y-%m-%d %H:%M:%S.%f')
  File "/usr/lib64/python3.6/_strptime.py", line 565, in _strptime_datetime
    tt, fraction = _strptime(data_string, format)
  File "/usr/lib64/python3.6/_strptime.py", line 362, in _strptime
    (data_string, format))
ValueError: time data '2021-02-12T16:17:28.393028' does not match format '%Y-%m-%d %H:%M:%S.%f'

To Reproduce
Steps to reproduce the behavior:

  1. Go to /opt/ibm/csm/bigdata/python
  2. Run ./findJobTimeRange.py -a #The allocation ID should exist
  3. See error

Expected behavior
Example of a successful run:

[csm04][/u/tlam/git/CAST/csm_big_data/python]> ./findJobTimeRange.py -a 259
# Found 1 matches for specified the job.allocation-id: 259
job-id: 1146 - 0
user-name: tlam 
user-id: 11745
begin-time: 2021-02-12.16:17:28:393 
end-time: 2021-02-12.16:17:48:421

Environment (please complete the following information):

  • Machine [IST BDS cluster]
  • Version [CSM 1.2.8-3577]

Additional context
Following two lines in findJobTimeRange.py are where date format is defined.

        date_format= '%Y-%m-%d %H:%M:%S.%f'
...
        start_time=datetime.strptime(tr_data["begin_time"], '%Y-%m-%d %H:%M:%S.%f')

Issue Source:
Because the date format is "hard coded" in the script. It required to be changed every time the date+time format is changed.

@thanh-lam
Copy link
Member Author

To fix the date+time format in the script, modify the two lines: Replace the blank between date and time with the letter T.

        date_format= '%Y-%m-%dT%H:%M:%S.%f'
...
        start_time=datetime.strptime(tr_data["begin_time"], '%Y-%m-%dT%H:%M:%S.%f')

@thanh-lam
Copy link
Member Author

Found another script that also has similar date+time format hard coded: findJobKeys.py:

    date_format= '%Y-%m-%d %H:%M:%S.%f'

That needs to be modified similarly:

    date_format= '%Y-%m-%dT%H:%M:%S.%f'

@thanh-lam
Copy link
Member Author

thanh-lam commented Mar 5, 2021

A 3rd script that uses old date+time format that needs to be corrected: findJobMetrics.py

from datetime import datetime
    date_format= '%Y-%m-%d %H:%M:%S.%f'
    start_time=datetime.strptime(allocation.get("begin_time"), date_format)
        end_time=datetime.strptime(allocation.get("history").get("end_time"), date_format)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants