Coral Extract Workflow: Duplicate Coral IDs causing unique constraint violation.

[kaladay]

Something is wrong with Coral Extract processing that is resulting in attempts at creating duplicate keys.

Some possibilities:

1. The coral Id uniqueness detection might be insufficient.
2. The database data is not being properly reset (could be a permissions issue).
3. There could be an encoding issue with unicode characters.

The documentation for this will also need to be updated to correctly represent the configuration and permissions:

• https://github.com/TAMULib/fw-registry#coral-extract

The following (long read) is a good explanation of problems with Unicode in javascript:

• https://dmitripavlutin.com/what-every-javascript-developer-should-know-about-unicode/

TL;DR

• Most JavaScript engines use UTF-16 encoding.
• The way source code is stored or interchanged is not relevant to ECMAScript specification, but usually is encoded in UTF-8 (the preferred encoding for the web).
• https://262.ecma-international.org/6.0/#sec-ecmascript-language-types-string-type
• In JavaScript to normalize a string invoke myString.normalize([normForm]) method, available in ES2015. (normalization should be used for string comparisons such as str1.normalize() === str2.normalize()
• Note that most of the string methods in JavaScript are not Unicode-aware: like myString.indexOf(), myString.slice(), etc.
• The new regular expression flag u enables Unicode-aware string matching.
• It creates misunderstanding when processing strings that contain surrogate pairs or combining character sequences:
  • Getting the string length
  • Character positioning
  • Regular expression matching
• Javascript library for processing unicode: https://github.com/mathiasbynens/punycode.js
• Using a java library might be a better alternative than some custom javascript for character encoding issues/concerns.

[ghost]

org.camunda.bpm.engine.impl.pvm.PvmException: couldn't execute activity <serviceTask id="database_query_task_55c95511_a571_4763_9233_34307594b19e" ...>: ERROR: duplicate key value violates unique constraint "coral_extract_pkey"
  Detail: Key (coralid)=(1614) already exists.
	at