mysql_: MariaDB 10.10+ may have INSERT BUFFER section in InnoDB stats that breaks plugin

[GreenReaper]

Describe the bug
I recently upgraded to Debian bookworm which contains MariaDB 10.11. My 'mysql' graphs no longer appeared. Running the mysql plugins resulted in an error: Unknown section: INSERT BUFFER at /etc/munin/plugins/mysql_innodb_insert_buf line 1386.

The issue seemsto be in parsing SHOW /*!50000 ENGINE*/ INNODB STATUS. It seems the section INSERT BUFFER AND ADAPTIVE HASH INDEX is now just INSERT BUFFER on MariaDB 10.10+.

This change appears to have been caused in MDEV-28542. The section may also only be visible in certain circumstances.

As part of this change, I will also hide the INSERT BUFFER section when the change buffer is empty. By default, it should be empty, because the default was changed to innodb_change_buffering=none in MDEV-27734.

In my case that variable is 'none' but it is still visible - but I have replication from another 10.5 database, so maybe that's impacting it.

Adding the line:
'INSERT BUFFER' => \&parse_insert_buffer_and_adaptive_hash_index,
to sub parse_innodb_status stopped the error and restored the graphs. It did not return any values for the plugin mysql_innodb_insert_buf. It seems like this is not correct for MariaDB and possibly MySQL as well.

The number of merges (0 in my case, presumably because the feature was disabled) can be obtained by changing the section starting:

m/\GIbuf: size (\d+), free list len (\d+), seg size (\d+),(?: (\d+) merges)?\n/gc && do {

to make the Ibuf: at the start optional and adjust the captures accordingly, i.e.

      m/\G(Ibuf: )?size (\d+), free list len (\d+), seg size (\d+),(?: (\d+) merges)?\n/gc && do {
        $data->{ib_ibuf_size}     = $2;
        $data->{ib_ibuf_free_len} = $3;
        $data->{ib_ibuf_seg_size} = $4;
        $data->{ib_ibuf_merges}   = $5 if defined $5; # MySQL >= 5.5
        return 1;
      };

The section above, with the regex
m/\Gmerged operations:\n insert (\d+), delete mark \d+, delete \d+\ndiscarded operations:\n insert (\d+), delete mark \d+, delete \d+\n/gc && do {
also did not match because there were no spaces before each mention of 'insert' after the \n. Removing these spaces (i.e. \ninsert in both cases) caused the correct (0) values to be returned.

To Reproduce
Steps to reproduce the behavior:

1. Have a MariaDB 10.5 database.
2. Have munin MySQL plugins using it with an InnoDB database
3. Upgrade to MariaDB 10.11 [or probably 10.10
4. See error when running mysql plugins (and the graphs disappear).

Expected behavior
I expected the MySQL plugin to gracefully handle MariaDB changing the section header. Perhaps even trying to continue through the parsing error.

Screenshots & Logs

| Type   | Name | Status 
=====================================
2023-06-14 21:41:10 0x7fdcbb3676c0 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 56 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 0 srv_active, 0 srv_shutdown, 8117 srv_idle
srv_master_thread log flush and writes: 8117
----------
SEMAPHORES
----------
------------
TRANSACTIONS
------------
Trx id counter 5803685961
Purge done for trx's n:o < 5803685959 undo n:o < 0 state: running
History list length 131
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION (0x7fdcbbc02780), not started
0 lock struct(s), heap size 1128, 0 row lock(s)
---TRANSACTION (0x7fdcbbbffb80), not started flushing log
0 lock struct(s), heap size 1128, 0 row lock(s)
--------
FILE I/O
--------
Pending flushes (fsync): 0
69877 OS file reads, 21734 OS file writes, 0 OS fsyncs
0 pending reads, 1 pending writes
0.04 reads/s, 16384 avg bytes/read, 2.48 writes/s, 0.00 fsyncs/s
-------------
INSERT BUFFER
-------------
size 4, free list len 845, seg size 850, 0 merges
merged operations:
insert 0, delete mark 0, delete 0
discarded operations:
insert 0, delete mark 0, delete 0
---
LOG
---
Log sequence number 788686016837
Log flushed up to   788686016488
Pages flushed up to 788647030743
Last checkpoint at  788647030743
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 4328521728
Dictionary memory allocated 30638760
Buffer pool size   259584
Free buffers       199430
Database pages     68108
Old database pages 25121
Modified db pages  1413
Percent of dirty pages(LRU & free pages): 0.528
Max dirty pages percent: 90.000
Pending reads 0
Pending writes: LRU 0, flush list 0
Pages made young 17574, not young 197991
0.11 youngs/s, 277.64 non-youngs/s
Pages read 67616, created 492, written 0
0.04 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 999 / 1000, young-making rate 0 / 1000 not 2 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 68108, unzip_LRU len: 20613
I/O sum[2]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 read views open inside InnoDB
state: sleeping
----------------------------
END OF INNODB MONITOR OUTPUT
============================

Desktop (please complete the following information):

• OS+Distribution Version: Debian bullseye -> bookworm
• Munin Version [e.g. 2.0.35]: v2.0.73

[gnikolaidis]

Kudos for both the great bug description, and the suggested fix. In my case it was a fresh install of Debian bookworm, with Galera WSREP enabled and replication from another 10.5 database.