Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pika一致性开发 #2944

Open
1 of 8 tasks
chejinge opened this issue Nov 8, 2024 · 3 comments
Open
1 of 8 tasks

Pika一致性开发 #2944

chejinge opened this issue Nov 8, 2024 · 3 comments
Labels
✏️ Feature New feature or request

Comments

@chejinge
Copy link
Collaborator

chejinge commented Nov 8, 2024

Which PikiwiDB functionalities are relevant/related to the feature request?

No response

Description

  • 确认当前的单key apply DB的一致性 起龙
  • 主实例增量同步binlog和改造 起龙
  • 主实例向从实例发送ACK请求 起龙
  • 从实例增量同步改造 浩宇
  • 从实例写完binlog向主实例发送ACK请求 浩宇
  • 原来不合理的部分改造 wyyyyy
  • 测试及压测. wyyyyy
  • review 小帅、俊华、少一

排期大概1个月,每周周会结束对一下进度

Proposed solution

问题记录:

Alternatives considered

###解决方案:
方案设计地址:
#2938

###开发分支
https://github.com/OpenAtomFoundation/pika/tree/consistency

@chejinge chejinge added the ✏️ Feature New feature or request label Nov 8, 2024
@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Title: Pika consistency development

@QlQlqiqi
Copy link
Contributor

QlQlqiqi commented Nov 12, 2024

确认当前的单 key apply DB 的一致性

目前的设计是没问题的,因为同 db(那肯定也包含同 key)的 binlog 会被分到相同的 worker 中,因为这个 worker 是 binlog worker,它只有一个 bg_thread 对任务进行取出和应用,所以不存在一致性的问题。调用顺序如下:

  1. PikaReplClientConn::DealMessage 中对 response 分类;
  2. PikaReplClientConn::DispatchBinlogRes 分批处理 binlog;
  3. ikaReplicaManager::ScheduleWriteBinlogTask;
  4. PikaReplClient::ScheduleWriteBinlogTask 根据 db name 采用不同的 binlog worker;

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Confirm the consistency of the current single key apply DB
The current design is no problem, because binlogs in the same db (which must also contain the same key) will be assigned to the same worker. Because this worker is a binlog worker, it only has one bg_thread to retrieve and apply tasks, so it does not There is a problem of consistency. The calling sequence is as follows:

  1. Classify response in PikaReplClientConn::DealMessage;
  2. PikaReplClientConn::DispatchBinlogRes processes binlog in batches;
  3. ikaReplicaManager::ScheduleWriteBinlogTask;
  4. PikaReplClient::ScheduleWriteBinlogTask uses different binlog workers according to db name;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✏️ Feature New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants