feat: Add PeerDB input plugin for handling Postgres CDC

[iskakaushik]

• Introduced a new PeerDB input plugin for Benthos that reads from a Postgres DB using CDC (Change Data Capture) with pglogrepl - https://github.com/jackc/pglogrepl.
• Change Data Capture (CDC) is a process that captures changes made to a database and ensures that these changes are reflected in downstream systems. This is crucial for keeping data in sync across distributed systems, enabling real-time data processing, and supporting various data integration scenarios.
• The implementation includes maintaining a persistent replication connection to ensure continuous data capture. The last offset LSN (Log Sequence Number) is stored in a cache to keep track of the replication progress, allowing the system to resume from the last processed point in case of disruptions. This ensures data consistency and reliability in the replication process. The cache has to be a persistent cache for correctness.
• Follow-up updates will include support for initial load and more comprehensive documentation.

[srenatus]

I'm confused about the name here. Has this something to do with PeerDB and their (commercial) offering? Or would you just share the name because the concept is the same...? https://github.com/jackc/pglogrepl doesn't mention "peerdb" anywhere.

[srenatus]

Is this significant? Would the stock postgres image be missing extensions..? 🤔

[iskakaushik]

This image has some changes to the default postgres image, it isn't hard to get this in there but was more convenient to just is it, specifically

wal_level = logical
max_wal_senders = <N > 0>
max_replication_slots = < N > 0>

[srenatus]

I'm just a passer-by reviewing what looked like a very cool addition. Please bear with me 😅

[iskakaushik]

I'm just a passer-by reviewing what looked like a very cool addition. Please bear with me 😅

Thanks for taking the time to look at the PR =)

[srenatus]

@iskakaushik I'm still confused about the "peerdb" name (see previous comment). Could you clear that up for me? 😅