-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Is there any way to compress the data while using mongo persistence with NEventStore? #62
Comments
@AGiorgetti @andreabalducci @Iridio Can you guys at least share your initial thoughts? We're really gated with this. Thanks. |
Hi @josevfrancis , you can try this:
let me know if it solved your problem. |
imho is useless unless you want to encrypt. Just enable compression for mongo (https://www.mongodb.com/blog/post/new-compression-options-mongodb-30) |
Hi @AGiorgetti Thanks for the quick response. Please let me know if I'm doing something wrong. |
Thanks for the quick response. I understand that we can use the MongoDB compression options but can't we save storage space if we try to save data that is already compressed? Apart from that, when trying out SQL as commits persistence, with Are there any best practices around the same? |
Hi @josevfrancis , the BinaryFormatter was deprecated long time ago by the .NET team. |
Hi @AGiorgetti, I have now replaced the BinarySerializer with a custom serializer having the below methods:
And used it like ByteStreamDocumentSerializer(new GzipSerializer(new CustomSerializer())) But, While adding the events to stream, I am getting "Unable to cast object of type 'System.Byte[]' to type 'NEventStore.EventMessage" error. Please suggest if something is wrong. |
@AGiorgetti Sorry for disturbing you with back-to-back questions. |
@andreabalducci @Iridio Do you have any thoughts around the quoted message above? |
double compression is a waste of cpu and adds little or no benefits (could be worse). |
@andreabalducci |
I'm working with C#, Dotnet core, and NeventStore( version- 9.0.1), trying to evaluate various persistence options that it supports out of the box.
More specifically, when trying to use the mongo persistence, the payload is getting stored without any compression being applied.
Note: Payload compression is happening perfectly when using the SQL persistence of NEventStore whereas not with the mongo persistence.
I'm using the below code to create the event store and initialize:
private IStoreEvents CreateEventStore(string connectionString) { var store = Wireup.Init() .UsingMongoPersistence(connectionString, new NEventStore.Serialization.DocumentObjectSerializer()) .InitializeStorageEngine() .UsingBsonSerialization() .Compress() .HookIntoPipelineUsing() .Build(); return store; }
And, I'm using the below code for storing the events:
public async Task AddMessageTostore(Command command) { using (var stream = _eventStore.CreateStream(command.Id)) { stream.Add(new EventMessage { Body = command }); stream.CommitChanges(Guid.NewGuid()); } }
The workaround did: Implementing the PreCommit(CommitAttempt attempt) and Select methods in IPipelineHook and by using gzip compression logic the compression of events was achieved in MongoDB.
Attaching data store image of both SQL and mongo persistence:
So, the questions are:
Is there some other option or setting I'm missing so that the events get compressed while saving(fluent way of calling the compress method)?
Is the workaround mentioned above sensible to do or is it a performance overhead?
The text was updated successfully, but these errors were encountered: