about zero-copy parsing

[Jimmy-Z]

I previously read about this issue (and commented)
#229

Indeed, iterator returned by a zero-copy parser from a Reader can't be collected into a Vec, (I think) I understood that.

Stream processing should be possible, but AFAICT this crate doesn't support that, since csv::Reader::deserialize requires DeserializeOwned.

Am I missing something?

[Jimmy-Z]

Wait, just found out there's a Serde and zero allocation section in the tutorial, should have RTFM before asking that...

[BurntSushi]

Sorry, I'm not quite sure what your question is here. The Serde and zero allocation section is probably what I would have linked you to otherwise.

Note also that "zero copy" is not quite correct here. The csv-core parser, for example, does do a copy in order to unescape fields, but it doesn't do an alloc. It is possible to build a real zero copy CSV parser, but the architecture looks quite a bit different as it requires unescaping fields lazily. (I know I sometimes say "zero copy" on the issue tracker when describing this crate, but I wasn't being carefully precise when I did that.)

[Jimmy-Z]

Sorry about the question being not clear, I do have a specific case in mind but I'm also learning rust so the generic title.

My case:
I have a long CSV, say 100,000 lines, it has a string field, and a couple numeric values, I need to keep the parsed results in memory(in a Vec) for quite a while, if I use the csv::Reader:deserialize interface, I'll have to deserialize into a DeserializeOwned type, which will allocate 100,000 Strings, in comparison, I'd rather keep the csv String in memory, and deserialize that string field into &str, better a big allocation than 100,000 small ones right?

I should probably write my own simple parser by just split it, since the CSV is generated and doesn't have escapes, it should be easy, just feels a little weird to write my own wheels.

[BurntSushi]

Yes, it is indeed weird. Is having 100,000 strings in memory actually a problem? Because it doesn't seem like one to me.

You are correct though that you cannot just borrow from the csv data. The csv crate doesn't support that architecture because it instead opts to eagerly unescape fields.