Incorrect parsing of quotes inside elements

[kendallb]

This file used to parse correctly in older versions of CsvHelper, and now it barfs saying there is bad data. But this is not bad data at all. Having a quote character in the middle of a cell should not cause the parser to barf. The quotes in a cell should only be parsed if they are immediately following the delimiter character and then the end one should be expected to be at the end. It should be possible to also have a single quote in the middle of the quoted section as well.

Now you could argue this is malformed data, but it is very common to see this and it used to parse correctly. It also loads just fine in Excel.

SKU,LENGTH,WIDTH,HEIGHT
3T-ARX-100,7,4.2,2.1
WT-TSC-26"-K,16,14,5.3

[JoshClose]

If you set

csv.Configuration.BadDataFound = null;

it will work as you expect.

[JoshClose]

void Main()
{
	var s = new StringBuilder();
	s.AppendLine("SKU,LENGTH,WIDTH,HEIGHT");
	s.AppendLine("3T-ARX-100,7,4.2,2.1");
	s.AppendLine("WT-TSC-26\"-K,16,14,5.3");
	using (var reader = new StringReader(s.ToString()))
	using (var csv = new CsvParser(reader))
	{
		csv.Configuration.BadDataFound = null;
		
		while (true)
		{
			var value = csv.Read();
			if (value == null)
			{
				break;
			}
			
			value.Dump();
		}
	}
}

[kendallb]

Why is this considered 'bad data'? The quote is embedded in the middle of the record, not at the start and end of it?

[JoshClose]

If a field contains a quote or a comma, it's supposed to be wrapped in quotes, and the quote escaped.

https://tools.ietf.org/html/rfc4180#page-2

If fields are not enclosed with double quotes, then double quotes may not appear inside the fields.

Fields containing line breaks (CRLF), double quotes, and commas should be enclosed in double-quotes.

If double-quotes are used to enclose fields, then a double-quote appearing inside a field must be escaped by preceding it with another double quote.

[greatiswork]

Hi Josh, I have a row in my csv with "|" delimited as shown below for some reason my first column value is appended with doublequote infront when read the testrecord whenever my description contains unwanted doublequote("). when i convert this serialized json it is adding " infront of the testrecord can you help what should be configuration to avoid " at the begining for testrecord . I am using Mode = CsvMode.NoEscape,

testrecord|fieldid|descrption|val
100004|8080010 |2341 Adj. Ring -2" for 152 |59.36|

output:
{
"testrecord": ""100004",
"Description": "2341 Adj. Ring -2"" for 152"
"val":"59.36"
}

desired:
{
"testrecord": "100004",
"Description": "2341 Adj. Ring -2"" for 152"
"val":"59.36"
}

[JoshClose]

Can you give a small failing example that shows this? Also, it looks like you're writing when this question was about reading.

This seems to work as expected:

void Main()
{
    var records = new List<Foo>
    {
        new Foo { TestRecord = 100004, FieldId = 8080010, Description = "2341 Adj. Ring -2\" for 152", Value = 59.36m },
    };
    var config = new CsvConfiguration(CultureInfo.InvariantCulture)
    {
        Mode = CsvMode.NoEscape,
    };
    using (var writer = new StringWriter())
    using (var csv = new CsvWriter(writer, config))
    {
        csv.WriteRecords(records);
        writer.Dump();
    }
}

public class Foo
{
    [Index(0)]
    [Name("testrecord")]
    public int TestRecord { get; set; }

    [Index(1)]
    [Name("fieldid")]
    public int FieldId { get; set; }

    [Index(2)]
    [Name("descrption")]
    public string Description { get; set; }

    [Index(3)]
    [Name("val")]
    public decimal Value { get; set; }
}

[mark1145]

I'm also encountering the same issue. In my case there is a text field that a user can manually input a description into and quite often they are putting things in quotes, example is below.

1,2,3,Tune!FM.net "Feeder Ninja" April - May,RandomDate,RandomNumber

The solution you gave seems to be to set BadDataFound to null, however, when I actually encounter a cell that contains bad data, I wont know about it or have any way to handle it.

csv.Configuration.BadDataFound = null;