Thread DIDs #295
Replies: 0 comments 19 replies
-
|
Howdy all! Here's a related discussion about service hubs: https://github.com/textileio/go-threads/discussions/484 |
Beta Was this translation helpful? Give feedback.
-
|
And here's a related discussion about integrating existing tools and services (see also the services discussion in this discussion): https://github.com/textileio/go-threads/discussions/485 |
Beta Was this translation helpful? Give feedback.
-
|
Another related discussion is the Bucket Pinning API for IPFS & Filecoin. It will likely be one of the first APIs to support some of the above methods, so take a look over there if interested. |
Beta Was this translation helpful? Give feedback.
-
|
DIDs for peers, users, and projects makes a ton of sense and I'm amped to see support for this in the threads protocol! I don't exactly follow what the benefit of using DIDs for the threads themselves are? Usually a DID and it's corresponding document is a way to bootstrap trust from an identifier to a set of public keys (which might change over time) that the subject controls. Threads don't really have control any private keys so the main value prop for using DIDs is unclear to me. |
Beta Was this translation helpful? Give feedback.
-
|
Hey @oed, thank you for diving in here! Maybe you can help me figure out if we're off base. The ethr-did-registry says, "A DID is an Identifier that allows you to lookup a DID document that can be used to authenticate you and messages created by you." That seems to be a narrow interpretation of the spec. The section D.1 What types of resources can a DID identify? states the following:
By identifying a thread as a global resource, any peer could determine the following from its DID:
Currently, this information is not globally available. So, if the thread is the resource we're trying to identify, i.e., the thread is the subject and the document itself is a representation of the subject, then the DID subject is not the controller. The controller is one or more user DIDs, e.g., The details of the document model aren't flushed out yet, but it might look something like the following: |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for clarifying @sanderpick. I had not really thought about this type of use case for DIDs before so it threw me off a bit, but seems like it makes sense 👍 One thing to look out for: |
Beta Was this translation helpful? Give feedback.
-
|
Hey @sanderpick this is exciting and really well articulated. I'm still trying to wrap my head around all of the moving parts, but some initial questions as they relate to daemon.land (The project my team and I are working on, which is working on Textile integrations with Ceramic/IDX):
A couple questions here, referencing the data structure you pasted: "authentication": [
{
// can authenticate as did:thread:id
"id": "did:key:foo#keys-1",
"type": "Ed25519VerificationKey2018",
"controller": "did:key:foo",
"publicKeyBase58": "..."
},
{
// can authenticate as did:thread:id
"id": "did:key:bar#keys-1",
"type": "Ed25519VerificationKey2018",
"controller": "did:key:bar",
"publicKeyBase58": "..."
}
],
I generally understand this (esp when it comes to frequently changing information like log heads), but just want to bring up a couple counter arguments in case they're useful (not sure where I personally stand on this yet). First, in some circumstances storing some metadata in the DID document can help promote interoperability between things. For example, storing the thread's schema would help other developers reuse threads when applicable, which is important in promoting application level interop. Second, storing encrypted encryption keys on the Document itself could enable certain "thread services" to roll their own access control mechanisms on top of threads, so you guys wouldn't have to care (as much) about supporting certain access mechanisms. You could encrypt to the thread DID, so only authenticated controllers could decrypt certain keys? (need to think about that more)
I think this is a fantastic idea, and something we've thought of for maintaining a list of popularly used schemas as well. Is this something Textile would promote and manage? Where would that list live? How would it get updated? How would users and developers know which to choose from and which to trust? Generally after reading this, I think thread DIDs sound like a great idea to move the system forward and enable more p2p interactions. However, still the big Q on my end is about more flexible thread and bucket access control mechanisms, and being able to dynamically alter them without causing massively expensive operations. |
Beta Was this translation helpful? Give feedback.
-
|
Hey @Schwartz10, thanks for chiming in!
Yep, that's the idea anyway
Yep, but I'm not sure about the best way to express those rules. I've been experimenting with a couple ideas, but the DID spec leaves ample room for interpretation of things like capability-invocation. Do you have any specific requirements?
The goal is for that to be any DID, but in practice we'll initially support
To clarify, a thread DID describes the lower level thread concept: a collection of logs. The DB layer that deals in schemas becomes a service: ThreadDB. That said, the idea of service metadata is really nice. In the current case of buckets, that metadata live in the thread, but that may be cumbersome for other services. Need to think more about it.
Nice, offloading things like access control sounds great. Storing encrypted data in a DID document sounds dangerous and seems to be discouraged, but maybe that should be an application level concern. |
Beta Was this translation helpful? Give feedback.
-
|
@andrewxhill passed me the link to this Issue when I started telling him about the ipfs-coord npm library I've been working on. There is large overlap between that idea and this one. I've been searching for other people interested in this topic. I've even been posting on the IPFS discussion forum. This blog post lays out my ideas, which could be considered a slightly different tweak to the ideas presented here. This YouTube video presentation present my early conceptualization of research into the idea. I love the idea of the DID and using a self-describing JSON document. This just happens to be the part of the problem I've been looking for a better solution to. This proposal seems like a fine way to do it. The DID document presented here could take the place of the schema library I'm using now.
I'd love to participate in nailing down specifics around an RPC specification. I'm just about to wrap up e2e encryption. I've been eyeballing this jsonrpc-lite npm library as way to bootstrap a very simple RPC system. This library would allow easy retrofitting of REST APIs to provide their same web services over IPFS. I've got an early prototype demo using the ipfs-coord library at chat.FullStack.cash. It's a real-time chat app. I'm hoping to find time to record a presentation on my progress and the inner workings of the demo and ipfs-coord library, once I finish getting e2e encryption and private chat working in the demo. |
Beta Was this translation helpful? Give feedback.
-
|
Cool stuff here @christroutner, happy to have you as part of the discussion, especially around the RPC stuff. I think we'll likely do some similar "discussion" style posts like this for things like this. I also have some work on generic rpc over libp2p streams, as well as http, and json rpc. I haven't "released" these 2nd two yet, but I'll aim to do that this week in case they are of interest. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for sharing, @christroutner. Very interesting! We'd love you help in figuring this all out.
As a POC, I've been using gRPC over libp2p using https://github.com/libp2p/go-libp2p-core/blob/master/mux/mux.go. Perhaps we can use a similar approach for a JSON RPC API that is more JS friendly. In a past project, we wrote a custom RPC mechanism on raw libp2p, and it was pretty awful to deal with. |
Beta Was this translation helpful? Give feedback.
-
|
Yeh nice. I'm defs a fan of JSON RPC, though it is a bit limiting in terms of the types of calls you can make, so something more generic, but still nicer for browsers would be really nice. This is a good start (https://github.com/carsonfarmer/libp2p-rpc), but it uses As a point of comparison, Ethereum nodes provide JSON RPC (among other) endpoints: https://eth.wiki/json-rpc/API, so there is some standardization there. |
Beta Was this translation helpful? Give feedback.
-
|
The crux is deciding how to type the request/response data. True, we can expose any number of endpoints, but I think having a single way to represent data is important. IPFS uses protos and CBOR. Perhaps supporting both wouldn't be too much of a pain. |
Beta Was this translation helpful? Give feedback.
-
|
I'm glad you're receptive to the work I've been doing! Picking a good RPC solution has a high risk of bike-shedding. There are many solutions to the problem, and everyone is going to gravitate toward their favorite. The determining factors in my mind is:
|
Beta Was this translation helpful? Give feedback.
-
|
Ping @dgtony and @requilence. We'd love some feedback from the Anytype team. I want to make sure we're solving something for your use cases. |
Beta Was this translation helpful? Give feedback.
-
|
I wanted to follow up on my previous comment. I've just published a couple demos to YouTube, which are linked in this blog post. I link to this thead and give a shout-out to the work done here. This represents the latest in my ongoing research. I'm all ears if you have any feedback. |
Beta Was this translation helpful? Give feedback.
-
|
Hey @christroutner, great videos. Thanks for sharing. It really does sound like we're well aligned here. We haven't done anything here on the JS end yet. In addition to the RPC design, @carsonfarmer there might be an opportunity here to collab on the JS libp2p host management: Circuit relays, etc. |
Beta Was this translation helpful? Give feedback.
-
|
I appreciate the feedback! I've also been brushing up on the DID specification, as well as JSON-LD. I'd love to see Textile build out their own DID implementation. That's certainly beyond the scope of what I want to do. I've decided to move forward with JSON-LD, focusing on the schema.org syntax. That will dovetail nicely with any DID implementation that might come down the road. I'm going to continue to improve chat.fullstack.cash, and it will eventually be forked into two apps: a database of deep-web websites, and a decentralized exchange (DEX) for cross-blockchain sale of cryptocurrencies. See this Twitter thread for more info. I'll continue to make efforts to stay in the loop, and look for opportunities to incorporate any work from Textile. I need to do a little deeper dive into the concepts behind Threads, and how they differ from OribitDB. |
Beta Was this translation helpful? Give feedback.
-
|
I came across this jsonld npm library an it lead me down the rabbit hole of the JSON-LD specification. I think this knowledge is complimentary to the idea of a DID and developing an API for decentralized services. This page has a lot of short introductory YouTube videos to the core concepts around JSON-LD: |
Beta Was this translation helpful? Give feedback.
-
An exploration of DIDs in a thread network.
Motivation
DIDs (Decentralized Identifiers) enable cross-protocol/blockchain interactions by defining a common interface to retrieve and validate entities, like users and data.
Threads would benefit from a native DID in numerous ways:
What is a DID?
A DID is a string identifier of a subject, controlled by a controller. For example, a DID might be used to encode an Ethereum address on the Ethereum Mainnet. For DIDs to be useful, they must be resolvable without reliance on a centralized network component. The figure below shows the interactions between DID components.
The basic components of DID architecture. Source: https://w3c.github.io/did-core/#architecture-overview
The Verifiable Data Registry could be a blockchain or peer-to-peer network. The meaning of verification differs between DID implementations. Some provide a high level of verification through blockchain transactions, while others rely purely on the assumption that the majority of peers are good actors, e.g., IPID (an IPNS-based DID) or Ceramic.
Key points
In short, DIDs point to relationships between the components of a decentralized network. What can this component do? Who/what can do it? Who/what dictates who can do it? Read the full spec for more info.
DIDs in a Thread Network
A thread network has multiple actors and structures (subjects) that can be described by DID documents. Subjects include any of the following entities:
Peers:
did:key:fooPeers (
go-threadspeers) can offer network services:A thread peer's DID is derived from its embedded LibP2P host key.
Users:
did:key:foo,did:3:foo,did:ethr:foo, etc.A User is any external identity that represents a network user, and that may interact with the network via a Peer. These may be identified by any verifiable DID.
Threads:
did:thread:idThreads are collections of single-writer logs. Thread documents contain verification methods for all valid log signers. Other thread info like the log head, log metadata, and thread encryption keys are not stored in the DID document, since this information is only needed by peers that are sharing a thread, and can be more efficiently exchanged using the thread protocol vs. the global document registry. Log addresses are referenced as service endpoints.
By identifying a thread as a global resource, any peer could determine the following from its DID:
Currently, this information is not globally available.
So, if the thread is the resource we're trying to identify, i.e., the thread is the subject and the document itself is a representation of the subject, then the DID subject is not the controller. The controller is one or more user DIDs, e.g.,
did:key:foo,did:ethr:foo,did:3:foo, etc.The details of the document model aren't flushed out yet, but it might look something like the following:
Thread DID Method Definition
The crux of any DID implementation is defining it's DID method. The DID spec defines a method as the following:
In this section, we define a DID method specification for threads which is composed of a method scheme and operations. Operations specify how a DID document is created, how to read/verify a document, as well as how a DID controller can update or even deactivate a DID document. Whereas the method scheme defines the structure of the DID implementation's string identifier(s).
Method Scheme
"did:<method>:<method-specific-id>"Generic DID structure.
A thread DID can simply prepend
did:thread:to its unique identifier."did:thread:<thread-id>"Thread DID structure.
"did:thread:bafk6npbypzgshgb4b6hzjju7zdc3bfghx3ugacl6aie3xrfxcg4x6mfuhoe6iesr"A Thread DID.
Method Operations
As mention perviously, DID methods define a set of operations that can be performed on/with the DID documents. DID implementations often use a smart contract on a blockchain like Ethereum to model the global data registry, and to implement these operations. For the implementation outlined here, we assume that documents are stored on the Filecoin blockchain as NFTs. Filecoin does not yet support NFTs, but there's an informal proposal to create a new actor type for them.
As a POC, we can develop a non-consensus driven global data registry using LibP2P PubSub along the lines of IPNS.
Create
To-do. DID documents are created by minting a new Filecoin NFT.
Read/Verify
The process of getting a DID subject's document from the verifiable data registry is called resolution. Any peer can resolve any document by querying the on-chain NFT representing the subject.
Update
To-do.
Deactivate
To-do.
Security Requirements
To-do.
DID method specifications MUST include their own Security Considerations sections. This section MUST consider all the requirements mentioned in section 5 of [RFC3552] (page 27) for the DID operations defined in the specification.
Privacy Requirements
To-do.
DID method specifications MUST include their own Privacy Considerations sections, if only to point to the DID spec, § 10. Privacy Considerations.
Architecture
The way in which a DID Method is leveraged is entirely up to the network itself. Here we outline some system requirements and walk through common network operations.
Vanilla Network
First, let's consider a network of completely open peers. These can be local or remote. Open mean no identity authorization. Anyone can create/add threads.
Requirements
A vanilla thread network (n=2) showing an external identity (A) creating a thread on one peer and writing to it from another peer.
Service-enabled Network
A major promise of the decentralized web is putting ownership back into the hands of users by shifting publishing, discovery, and services from a few core providers, onto the "edge" and into the hands of users/creators. Services such as those traditionally found in centralized systems via open (or closed) application programing interfaces (APIs) are less common in p2p systems, however, a DID-driven thread network offers an exciting opportunity to expose p2p services as first-class components of a p2p system.
In this network, one of the peers has a trusted relationship with a service. Peer can advertise their services using the
"services"DID field (read more about this here). For example, let's consider a hypothetical web-hook service that allows users to add web-hooks to a thread. Every time the thread receives an update, the web-hook fires on the user-defined endpoint. The peer's DID document includes the service info:{ "@context": "https://www.w3.org/ns/did/v1", "id": "did:key:peer-id", "authentication": [{ "id": "did:key:peer-id#keys-1", "type": "Ed25519VerificationKey2018", "controller": "did:key:peer-id", "publicKeyBase58": "..." }], "services": [ { "id": "did:key:peer-id#threads", "type": "ThreadService", "serviceEndpoint": "/p2p/peer-id", "serviceProtocol": "/thread/0.0.1" }, { "id": "did:key:peer-id#webhooks", "type": "ThreadWebHookService", "serviceEndpoint": "/p2p/peer-id", "serviceProtocol": "/thread/0.0.1", "cost": { "hook": { "amount": "xx nanoFIL", "currency": "FIL" }, "hit": { "amount": "xx nanoFIL", "currency": "FIL" } } } ] }Requirements
"type","serviceEndpoint", and"description"fields. The community can maintain a list of available services by type.A service-enabled thread network (n=2) with a hypothetical web-hook service. An external identity (A) has used the service to add a web-hook to a thread and then writes to the thread from another peer.
At a minimum, a peer will advertise it's own LibP2P thread API. Here are some examples of services a peer might offer:
go-datastoreinterface (key-value store)"serviceEndpoint": <connection_uri>) (see also Existing tools discussion)"serviceEndpoint": <connection_uri>) (see also Existing tools discussion)The diagram below shows how buckets and Filecoin can be orchestrated as peer services.
Buckets and Filecoin can be considered network services.
Examples
Dapps
Slate is a good example of an application that could be developed as a pure dapp on the thread network. Many other apps can leverage these same patterns: Slack, Notion, etc.
"did:fil:<address>"."controller"field).Smart Contracts
Smart contracts on any blockchain can access IPFS and Filecoin storage without the need for a centralized pinning service. This is possible because users (or client-side peers) are able to query the registry without depending on a specific peer or set of peers. Client-side peers may be lightweight thread peers with read-only capabilities, or with the ability to query but not respond to registry events. All of this enables the use of a single token to pin content against a given peer (maybe the hub, or some bucket pinning API) without the need for a secret. This is because the receiver can validate that the caller is authorized to do so by looking at the thread DID doc.
Exposing Web2 via Web3
In tandem with an additional threads effort to wrap existing tools, a thread network can enable existing web2 services, APIs, and applications to read and write data using existing tools like MongoDB, PostgreSQL, Redis, etc. This opens Filecoin to a massive untapped source of data. But the thread network also allows service providers to take this one step further.
In addition to exposing more "traditional" web2 APIs (REST, gRPC, etc) as services, it is possible to expose these APIs over p2p protocols. Projects such as libp2p-http, allow peers to serve HTTP endpoints and make HTTP requests through libp2p using Go's standard "http" and "net" stack. This provides a simple on-ramp for web2 developers to expose their APIs over the threads p2p network.
For example, a peer may serve a RESTful pinning API that they expose via a p2p service. Any service that produces content-addressable outputs can be easily federated via the registry, and any "location-specific" APIs (e.g., direct database access) can benefit from libp2p-based routing — taking advantage of features like multi-routes, NAT transversal and stream multiplexing over a single connection. Additionally, these APIs can now take advantage of the authorization patterns of the threads network "for free", providing direct benefits to service providers, and opening the door to crypto-native payments for off-chain services.
References
Beta Was this translation helpful? Give feedback.
All reactions