How well does this forum post describe CSVHelper?

[SteveCollins]

This link leads to a forum posting on SQL Server Central where I attempted to describe how we use CSVHelper to fix uploaded files before they get read into the database. My background is mostly SQL and only recently C# as well. Are there parts which are not accurate and/or could be improved? Thanks for everything

[JoshClose]

Supposedly the author worked with Microsoft to make sure there is 100% consistency.

I have not worked with MS. Where did you hear this?

CSVHelper's preferred object format to read/write files to/from is List where T is a type.

GetRecords<T> actually returns an IEnumerable<T> that yields records one at a time. If you do something like ToList(), that will load the entire CSV into memory. It's recommended to handle a single record at a time if possible. Then you will only have that record in memory.

Afaik that means to make the columns include the escaped quotations necessary such that the entire file is unambiguous to read.

It gives you the ability to determine if the field should be escaped. Returning true from that function like you have will always quote them.

This feature used to be documented (I couldn't find any longer when I looked) but then years ago there was an update and the method was renamed.

Yes, the documentation is lacking. ;)

[SteveCollins]

Supposedly the author worked with Microsoft to make sure there is 100% consistency.

I have not worked with MS. Where did you hear this?

If it's not true then I probably didn't hear it anywhere. In my mind CSV was somehow MS controlled maybe because they make so much use of it. I'll update that to just say standards compliant

CSVHelper's preferred object format to read/write files to/from is List where T is a type.

GetRecords<T> actually returns an IEnumerable<T> that yields records one at a time. If you do something like ToList(), that will load the entire CSV into memory. It's recommended to handle a single record at a time if possible. Then you will only have that record in memory.

In SQL loops are really frowned on. What's the better way a foreach loop?

Afaik that means to make the columns include the escaped quotations necessary such that the entire file is unambiguous to read.

It gives you the ability to determine if the field should be escaped. Returning true from that function like you have will always quote them.

It would properly escape any inner quote marks and/or commas, no? In my experience always quoting the fields makes it so whatever eventually opens the CSV file can be really clueless. SQL is pretty rigid and throws on any anomaly. The tiny amount of extra file size eliminates many headaches.

This feature used to be documented (I couldn't find any longer when I looked) but then years ago there was an update and the method was renamed.

Yes, the documentation is lacking. ;)

Ha, the code is nicely organized and documented too. We've been using it for years with nothing but positive results. Many thanks

[JoshClose]

I'll update that to just say standards compliant

It's RFC 4180 compliant. It also does 2 other modes of parsing; Escape and NoEscape.

In SQL loops are really frowned on. What's the better way a foreach loop?

Oh, this is C# code in SQL Server, like in a stored procedure? I think I missed that part. I'm not sure how SQL Server would do that then. I suppose you could profile it and see what happens. A foreach loop would probably be better I would guess. I'm assuming the iterator in SQL Server doesn't pull the entire table into memory... but I don't know.

[SteveCollins]

It's a C# .NET Core API project. We're using Azure SQL as paas so there's no access to the CLR which is a bummer. My C# suffers from what are good habits in SQL ;)

[SteveCollins]

Sorry I'm wondering how to implement that in my code. I appreciate your patience. Instead of have 2 separate method(s), ReadCSV and WriteCSV, would it be better to iterate through 1 at a time in one method and read/write within the same loop? It seems like the answer is yes. Thank you again

[JoshClose]

Reading and writing back at the same time would be good. You could call it streaming the data. That will take the least amount of memory to do for sure.