remove experimental graphsync server #9747
Conversation
|
We should get https://github.com/protocol/network-measurements to add stream handler in indentify, this would allows us to see usage of the graphsync server in the wild. |
Jorropo
force-pushed
the
remove-graphsync
branch
from
d705eb9
to
ea29ff4
Compare
|
@Jorropo that information won't tell you quite what you need to know. The question you want to be asking is how many kubo nodes (discoverable by the tooling) are running GraphSync, not just how many are running GraphSync |
|
@aschmahmann I don't understand what you mean, the tooling will discover what the tooling can discover. If you want me to reframe what I'm asking:
|
From our latest crawl we can see that a maximum of 30 peers advertise support for graphsync. |
|
Might be worth poking at those peers to see who they are and why they're using graphsync then? |
|
We (dClimate) are looking into using GraphSync in order to improve some of our queries, please hold on removal 🙏 |
Jorropo
force-pushed
the
remove-graphsync
branch
from
ea29ff4
to
f2e8478
Compare
|
This also saves ~3M on a non stripped build of Kubo. |
Jorropo
force-pushed
the
remove-graphsync
branch
from
f2e8478
to
ef620aa
Compare
Updates: #9396 Closes: #6831 Closes: #6208 Currently the Graphsync server is not widely used due to lack of compatible software. There have been many years yet we are unable to find any production software making use of the graphsync server in Kubo. There exists some in the filecoin ecosystem but we are not aware of uses with Kubo. Even in filecoin graphsync is not the only datatransfer solution available like it could have been in the past. `go-graphsync` is also developped on many concurrent branches. The specification for graphsync are less clear than the trustless gateway one and lack a complete conformance test suite any implementation can run. It is not easily extansible either because selectors are too limited for interesting queries without sideloading ADLs, which for now are hardcoded solutions. Finaly Kubo is consistently one of the fastest software to update to a new go-libp2p release. This means the burden to track go-libp2p changes in go-graphsync falls on us, else Kubo cannot compile even if almost all users do not use this feature. We are then removing the graphsync server experiment. For people who want alternatives we would like you to try the Trustless-Gateway-over-Libp2p experiment instead, the protocol is simpler (request-response-based) and let us reuse both clients and servers with minimal injection in the network layer. If you think this is a mistake and we should put it back you should try to answer theses points: - Find a piece of opensource code which uses a graphsync client to download data from Kubo. - Why is Trustless-Gateway-over-Libp2p not suitable instead ? - Why is bitswap not suitable instead ? Implementation details such as go-graphsync performance vs boxo/gateway is not very interesting to us in this discussion unless they are really huge (in the range of 10x~100x+ more) because the gateway code is under high development and we would be interested in fixing theses.
Jorropo
force-pushed
the
remove-graphsync
branch
from
ef620aa
to
f230447
Compare
Updates: #9396
Closes: #6831
Closes: #6208
Currently the Graphsync server is not widely used due to lack of compatible software.
There have been many years yet we are unable to find any production software making use of the graphsync server in Kubo.
There exists some in the filecoin ecosystem but we are not aware of uses with Kubo.
Even in filecoin graphsync is not the only datatransfer solution available like it could have been in the past.
go-graphsyncis also developped on many concurrent branches.The specification for graphsync are less clear than the trustless gateway one and lack a complete conformance test suite any implementation can run.
It is not easily extansible either because selectors are too limited for interesting queries without sideloading ADLs, which for now are hardcoded solutions.
Kubo is consistently one of the fastest software to update to a new go-libp2p release.
This means the burden to track go-libp2p changes in go-graphsync falls on us, else Kubo cannot compile even if almost all users do not use this feature.
We are then removing the graphsync server experiment.
For people who want alternatives we would like you to try the Trustless-Gateway-over-Libp2p experiment instead, the protocol is simpler (request-response-based) and let us reuse both clients and servers with minimal injection in the network layer.
If you think this is a mistake and we should put it back you should try to answer theses points:
Implementation details such as go-graphsync cpu / memory usage vs boxo/gateway is not very interesting to us in this discussion unless they are really huge (in the range of 10x~100x+ more) because the gateway code is under high development and we would be interested in fixing theses.