why is this strptime example not working??

[jefftemplon]

I use strftime to convert 1970-based number of seconds to a string, then use the exact same format to convert it back: error!

└─[$] mlr -n put 'end {print strftime(1709552468, "%a %b %e %T %Y");}' 
Mon Mar  4 11:41:08 2024
└─[$] mlr -n put 'end { s="Mon Mar  4 11:41:08 2024"; print strptime(s, "%a %b %e %T %Y");}' 

(error)

I don't get it! A bug, or am I missing something?

[johnkerl]

@jefftemplon looks like #1355

That issue is a bit of an annoyance tbh :(. Problem is I'm using a third-party pbnjay/strptime library function, and (as an implementation detail) it uses the same offset to walk through the format string as through the to-be-parsed input string. So two digits match two and one matches one but one doesn't match two. :(

You're doing nothing wrong, and this should work. It's a bug that it doesn't. I need to do a patch on Miller's fork of the pbnjay/strptime library function.

[jefftemplon]

Thanks! As a side note, I looked here

https://miller.readthedocs.io/en/latest/reference-dsl-time/#custom-formats-strptime-and-strftime

and noticed that if I read down a bit further, there is a second, much shorter list of available codes for strptime, that list does not contain %e. It is not clear from how things are written, if these are the ONLY codes accepted by strptime - I think not, because when the date contains two digits, my format works, even though %T is not on that second list. It's in any case unclear!

I will hold off on trying to make a workaround, pending your investigation.

[johnkerl]

It's in any case unclear!

Yes, it needs fixing. :(

I will hold off on trying to make a workaround, pending your investigation.

This isn't top-of-stack at the moment and Miller is a spare-time open-source project so I wouldn't want to keep you waiting for me :(

The best workaround at the moment is if you can get two-digit (zero-padded) dates in your input ... but then I see you're using Miller's strftime so let me 👀 ...

[johnkerl]

Unfortunately

strptime(strftime(1709552468, "%a %b %d %T %Y"), "%a %b %d %T %Y")

gives

date format contains unsupported percent-encodings

so there is more work to be done here ...

[jefftemplon]

Hi,

I will work around it otherwise. The tool that's feeding the info to mlr can actually do the conversion already, it's just passing the raw values through in "mlr mode" since mlr could do it (I thought). It used to work, until I upgraded to a recent version of mlr lately.

[johnkerl]

It used to work, until I upgraded to a recent version of mlr lately.

@jefftemplon indeed. :(

Mlller 5 and below used the C standard library which was of higher quality in this regard. The Go port (Miller 6 and above)

• https://miller.readthedocs.io/en/latest/new-in-miller-6/
• https://miller.readthedocs.io/en/latest/release-docs/index.html

which was released in 2022 was, on the whole, a massive feature improvement. The downgrade has been strptime and strftime which will need more work to bring them back up to C-library standards.

(More details: it's not the case that the Go standard library has a defective strptime and strftime. In fact these routines don't exist in the Go standard library, as the Go standard library has a completely different way of doing time <-> string conversions. The open-source lestrrat-go/strftime and pbnjay/strptime Go packages are good-faith efforts on the part of open-source developrs to retrofit strptime/strftime format-string syntax onto the Go standard library. I've forked one of these already and will be forking the other.)

[jefftemplon]

Thanks for all your work. I used Miller 3? 4? 5? not sure - for many years (since 2015 AFAICT), only recently decided I should think about upgrading, mostly because some things documented for Miller 6 didn't work in the version I had.

It took me a bit of work, but the problem is solved for me at the moment, by having the upstream tool spit out epoch seconds and me parsing them with sec2locatime and supply the correct timezone.

[johnkerl]

Glad you could use an upstream data tweak @jefftemplon -- not all people are so fortunate :)

[jefftemplon]

Related: why does this not work?

└> mlr put '$c=round(5000.)' then put '$a=sec2hms($c);$b=sec2hms(5000)'
d
1=d,c=5000,a=(error),b=01:23:20

[torbiak]

round() rounds to the nearest integer, but it returns a float, and sec2hms() expects an int. Doing an int conversion makes it work:

$ mlr -n put -q 'end{emit1 {"a": sec2hms(int(round(5000.)))}}'
a=01:23:20

[jefftemplon]

@torbiak thanks for your answer!