Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add carv2-messaging example for embedding messages in CARv2 #89

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

rvagg
Copy link
Member

@rvagg rvagg commented Jul 12, 2022

Background

This comes from a feature request by @mikeal to be able to signal additional, potentially arbitrary properties for backend communication, like supplying "PieceCID" for the CAR, and other things .. pushing CAR from just a block transport to also have a messaging layer that relates to those blocks that's more than just "here are the roots".

Proposal

So this is .. sneaky .. we take a standard CARv1 (in this case, just a single-block simplistic example), wrap it in a CARv2, push the "data offset" back enough to embed a length-prefixed dag-cbor "message" just after the CARv2 header.

We have a "characteristics" bitfield in CARv2 but only one bit that's currently being used, this proposes that the second-to-leftmost bit being used to indicate "message after v2 header".

The resulting .car file is readable by js-car as-is, the API shows you what you'd see if you passed it the original CARv1 contents, but it's wrapped in a CARv2 and you can poke at the bytes to see that there's a secret message in there just after the CARv2 header and before the CARv1 payload.

It's also valid according to go-car:

$ car list example-messaging.car 
bafkreihwkf6mtnjobdqrkiksr7qhp6tiiqywux64aylunbvmfhzeql2coa
$ car inspect example-messaging.car 
Version: 2
Characteristics: 40000000000000000000000000000000
Data offset: 158
Data (payload) length: 120
Index offset: 0
Index type: (none)
Roots: bafkreihwkf6mtnjobdqrkiksr7qhp6tiiqywux64aylunbvmfhzeql2coa
Root blocks present in data: Yes
Block count: 1
Min / average / max block length (bytes): 24 / 24 / 24
Min / average / max CID length (bytes): 36 / 36 / 36
Block count per codec:
        raw: 1
CID count per multihash:
        sha2-256: 1

I'm interested to see who might object to such flagrant abuse of CARv2 flexibility ..

@rvagg rvagg requested a review from mikeal July 12, 2022 12:21
@rvagg
Copy link
Member Author

rvagg commented Jul 12, 2022

I should also say - this is very manual in here, if we adopt this then the API would have to be made to handle at least some of this so you don't have to get so dirty with the bytes and bits... cause it's kind of nasty atm.

@rvagg rvagg force-pushed the rvagg/sneaky-message-carv2 branch from b646b67 to 9f26864 Compare July 12, 2022 12:26
rvagg added a commit to ipld/go-car that referenced this pull request Jul 13, 2022
EmbedMessage sets the second-to-leftmost bit to indicate that there is a
length-prefixed dag-cbor message object directly after the header. If that bit
is set, Reader#ReadEmbeddedMessage will decode and return that message.

Ref: ipld/js-car#89
@rvagg
Copy link
Member Author

rvagg commented Jul 13, 2022

go-car impl (including ability to read and recreate the .car created by this example): ipld/go-car#322

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant