Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Milestone 1 code for review -- messages 0 - 7 #3

Open
wants to merge 43 commits into
base: master
Choose a base branch
from

Conversation

VhatAmI
Copy link

@VhatAmI VhatAmI commented Jul 31, 2022

No description provided.

Confluent.Kafka only non-Microsoft dependency
Still have questions about batches, left it empty for now
minus batch, and hash byte[] printing incorrectly -- values of byte array seem to be correct, just need to convert to hex
preping for post 0 messages -- postfix is hash of key
Not sure yet how to get encoded value out to set msg0.Value
Users/myale/add block impl for messages 0,1,2,3
Need to verify it is what consumer expects
Temporary hardcoding, I think value is calculated somehow based on key
Avro encoding added, plus message4 and code clean up
this will now handle topic keys that are not in the sample doc, as long as they are a single letter

Also fix AddDeletesToMessages value to empty byte array

Also AddUpdatesToMessages does not send messages for keys not in SchemaMap
GetNonSchemaUpdates to add to message 0 entries that are not part of SchemaMap
batches datatype still needs to be improved
now includes batch message.

Need to revisit batches datatype
Add SendBatch parameters, updates to GetUpdatesCountStrings and Deletes/Updates messages
using this in two places and about to put it in a third spot, so refactoring to keep things dry

First draft of messages 5 - 7
Users/myale/new batch data type - first draft of messages 5 - 7
Messages all look correct again, using assumption made in slack chat
Users/myale/regex and polishing Regex work from yesterday
message0 blocks so it is always first
other messages fly out as fast as possible, do not wait for completion
…essagesagain

Users/myale/testing cleanup add send message logic back in, message 0 blocks to make sure it is always first
@VhatAmI VhatAmI changed the title Avro Encoding, Messages 0 - 4 Milestone 1 code for review -- messages 0 - 7 Sep 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant