Handling doubly-escaped strings

[nyurik]

My CSV data has many "json string inside CSV string" columns. I would like to strip the quotes on all fields, without listing the fields one by one if possible:

foo,bar,baz
"""ts""","""some value""",10
...

How can I strip extra quotes, converting it into json:

[
  {"foo": "ts", "bar": "some value", "baz": 10},
  ...
]

[johnkerl]

@nyurik I think you want the json-parse verb

$ cat input.csv
foo,bar,baz
"""ts""","""some value""",10

$ mlr --icsv --ojson --from input.csv json-parse
[
{
  "foo": "ts",
  "bar": "some value",
  "baz": 10
}
]

$ mlr json-parse -h
Usage: mlr json-parse [options]
Tries to convert string field values to parsed JSON, e.g. "[1,2,3]" -> [1,2,3].
Options:
-f {...} Comma-separated list of field names to json-parse (default all).
-h|--help Show this message.

There is also a json_parse DSL function which you can invoke cell-by-celel if you want

[nyurik]

Thanks @johnkerl ! I think the only issue with json_parse is that it outputs (error) whenever it can't parse fields. Is there a way to keep the original value in case it can't be parsed? Should this be a feature request? (I have 20-30 fields, so easier not to list the fields explicitly).

[johnkerl]

Yeah @nyurik good call, there should be an option to json_parse but keep the original on error. Maybe mlr json_parse -k where k is to keep the original on failure.

[johnkerl]

@nyurik #1291