ZMQ PUB/SUB JSON Types #330
Conversation
github-actions
bot
added
A-dependency
dimalinux
force-pushed
the
dimalinux/zmq-types
branch
from
c484488
to
d4a6998
Compare
| /// included. `do-not-relay` transactions *are* included. Values are not | ||
| /// republished during a re-org. | ||
| #[derive(Debug, Default, Clone, Serialize, Deserialize)] | ||
| pub struct TxPoolAdd { |
There was a problem hiding this comment.
There was a problem hiding this comment.
They are similar, but their JSON layout, unfortunately, is completely different.
There was a problem hiding this comment.
Ughh why does monero have 2 JSON tx formats 😭
There was a problem hiding this comment.
Provided a suggestion but you will need to test it to be sure its not rejected by any lint.
Also if you could add your crate (and ZMQ category) to https://github.com/Cuprate/cuprate/blob/main/books/architecture/src/appendix/crates.md with your PR.
When merging main don't forget to replace cuprate-types import with workspace = true instead of path = "../../types". Also define cuprate-zmq-types crate as workspace dependency with the rest of cuprate workspace members
Otherwise LGTM. Good works.
github-actions
bot
added
A-docs
|
|
||
| [dev-dependencies] | ||
| serde_json = { workspace = true, features = ["std"] } | ||
| assert-json-diff = "2.0.2" |
There was a problem hiding this comment.
IMO I would rather have worse error messages on tests than pull an extra dep in
There was a problem hiding this comment.
I pulled it in as a dev-only dependency, because testing JSON equality is a very non-trivial thing to do. Client readers are not supposed to care about whitespace or the order of fields within an object. If the input object is a map, its field output emit order can even vary between test runs.
In Golang, the feature to test JSON equality is built right into the most popular testing library (testify). assert-json-diff appeared to be the most widely used option for testing JSON-backwards-compatibility in Rust.
I did verify that the library differentiates between empty lists and nil lists, and the other types of backwards compatibility that we care about.
If, after reading this, we still don't want to use the assert-json-diff crate, let me know and I'll find a way to make the tests work without it. We are not using any maps, so it should be possible. The expected values will probably have to be on single, super-long lines, so they won't be human readable. Or I can get the test samples formatted in the same way as serde's pretty print and use multi-line raw strings.
There was a problem hiding this comment.
Seen as it is only a dev dep we can keep it for now, although it might get removed to reduce the number of deps in the future
Adds all JSON serialization types needed to implement the ZMQ pub/sub daemon interface or to be a client of the daemon. (Subtask of #199).
There is a separate repo for testing against
monerod. That program subscribes to all ZMQ messages, deserializes them into the types in this PR, then reserializes them back to JSON and verifies that the original and re-serialized JSON matches.I have a few hundred megabytes of output from that program if you are looking for a large number of samples to study. The in-file unit tests are using the original, small sample set:
https://gist.github.com/dimalinux/50cc09956618f4322520246b5ec132dc
When merging, feel free to delete the individual commit message entries to reduce commit message noise.