documentation for different record formats

[saikumare-a]

Question

could you help with providing the differences and in which scenario's the corresponding record formats should be used?

1. .option("record_format", "D)"
2. .option("record_format", "D2)"

[yruslan]

.option("record_format", "D") should be used for most ASCII use cases with the latest Cobrix.

Basically, 'D' uses Hadoop ASCII file splitter which is faster. But until recently Cobrix didn't support ASCII charsets other than UTF8 using that splitter. So we implemented dour own and called it D2. But now 'D' supports ASCII charsets as well, so these record formats are almost identical. Please, use 'D' with the latest Cobrix.