Feature Request: retain important vreplication messages

[shlomi-noach]

Feature Description

Today, vreplication can overwrite useful terminal error message with generic messages. This happens when we run setState. A sample scenario:

• vreplication workflow fails on an unrecoverable error, say duplicate key, errno 1062.
• This sets state to error and message in _vt.vreplication to appropriate values ("failed inserting rows: Duplicate entry ...")
• Later on someone retries the workflow, or any other call to setState,and the information in the above message is overwritten by e.g. "Picked source tablet..."

My suggestion is very simple and without adding risks:

• Columns state and message can stay as they were. These indicate the last known state + message, that's fine.
• We add a new terminal_error text default null column.
• We only populate the column on terminal errors.
• We only populate the columns once, ie we issue something like UPDATE _vt.vreplication SET terminal_error=COALESCE(terminal_error, ?).

Online DDL will also use this change. When it reads the error message from _vt.vreplication, it will run: SELECT ...., COALESCE(terminal_error, message) as message, ... FROM _vt.vreplication.

Use Case(s)

Better analysis of vreplication / Online DDL errors.

[shlomi-noach]

Or... come to think of it, we do persist all messages in vreplication_log, so it's more a matter of extracting the most relevant message.

[shlomi-noach]

Fixed in #16925 via vreplication_log.