Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve RemoteSequencer tracing and logging #2286

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Improve RemoteSequencer tracing and logging #2286

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 27 additions & 8 deletions crates/bifrost/src/loglet/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,21 +156,40 @@ pub type SendableLogletReadStream = Pin<Box<dyn LogletReadStream + Send>>;

#[allow(dead_code)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we still need this?

pub(crate) struct LogletCommitResolver {
tx: oneshot::Sender<Result<LogletOffset, AppendError>>,
tx: Option<oneshot::Sender<Result<LogletOffset, AppendError>>>,
}

#[allow(dead_code)]
impl LogletCommitResolver {
pub fn sealed(self) {
let _ = self.tx.send(Err(AppendError::Sealed));
pub fn sealed(mut self) {
let _ = self
.tx
.take()
.expect("must be set")
.send(Err(AppendError::Sealed));
}

pub fn offset(self, offset: LogletOffset) {
let _ = self.tx.send(Ok(offset));
pub fn offset(mut self, offset: LogletOffset) {
let _ = self.tx.take().expect("must be set").send(Ok(offset));
}

pub fn error(self, err: AppendError) {
let _ = self.tx.send(Err(err));
pub fn error(mut self, err: AppendError) {
let _ = self.tx.take().expect("must be set").send(Err(err));
}
}

#[derive(Debug, Clone, Copy, thiserror::Error)]
#[error("Commit resolver was dropped")]
struct CommitCancelled;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The term "cancelled" often indicates a graceful user-requested cancellation of something. Abort on the other hand is used to denote an abrupt ungraceful one.

That said, I'm not sure if this error type adds enough context to the caller to understand why the commit was aborted. Do you have ideas on how either:
A) Making this error more precise to pin point why the append was aborted
B) Possibly remove the need for this error in lieu of an existing one?


/// If a LogletCommitResolver is dropped without being
/// 'resolved', we resolve it automatically as being cancelled
/// To make it distinguished from a Shutdown.
impl Drop for LogletCommitResolver {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure this is necessary. Since if the LogletCommitResolver is dropped and the sender channel is closed, this causes a RecvErr to be received at the receiver side which is then mapped into an AppendError::Shutdown.

I am wondering if a Shutdown is what we need to return here. This still can happen for example if a connection is lost in case of remote sequencer.

What do you think @AhmedSoliman

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can map the RecvErr to a more descriptive type. For instance, CommitStatusUnknown and mark it as retryable append error?

fn drop(&mut self) {
if let Some(tx) = self.tx.take() {
let _ = tx.send(Err(AppendError::retryable(CommitCancelled)));
}
}
}

Expand All @@ -194,7 +213,7 @@ impl LogletCommit {
#[allow(dead_code)]
pub(crate) fn deferred() -> (Self, LogletCommitResolver) {
let (tx, rx) = oneshot::channel();
(Self { rx }, LogletCommitResolver { tx })
(Self { rx }, LogletCommitResolver { tx: Some(tx) })
}
}

Expand Down
Loading
Loading