MARC::XMLWriter will generate invalid XML documents.

[corylown]

MARC::XMLWriter is used by MarcRecordWriterService and OaiMarcRecordWriterService to generate MARC XML documents from contributor supplied MARC records. The supplied MARC records can include invalid byte sequences, non-printing control characters, and other anomalies that are invalid in XML.

This ticket is a placeholder for determining what, if anything, we ought to do about this.

Interestingly, because we're using Nokogiri to produce OAI-PMH responses from documents containing XML record snippets, the OAI feed is valid XML because Nokogiri appears to remove data that would be invalid in XML. However, MARC XML formatted normalized full dumps and deltas served via the web app and ResourceSync may be invalid.

[JohnMarkOckerbloom]

Do we know how Nokogiri handles the invalid data? (Do they just remove invalid characters, or do they do something more than that?) Might not hurt for us to do the same thing when normalizing, if that's easily done; it'd be a straightforward way to bring the different MARC XML views into sync.

[corylown]

In the case I observed Nokogiri removed the invalid bytes.