Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement statistics to support Connection Management #157

Closed
hacdias opened this issue Feb 3, 2018 · 15 comments
Closed

Implement statistics to support Connection Management #157

hacdias opened this issue Feb 3, 2018 · 15 comments
Labels
exp/wizard Extensive knowledge (implications, ramifications) required help wanted Seeking public contribution on this issue P1 High: Likely tackled by core team if no one steps up

Comments

@hacdias
Copy link
Member

hacdias commented Feb 3, 2018

In order to implement stats/bw on js-ipfs, we need to have some way to measure the bandwidth that comes in and out of js-ipfs. It should also support peer and protocol basis metrics.

Some refs:

/cc @pgte @diasdavid

@daviddias daviddias added the status/ready Ready to be worked label Feb 5, 2018
@daviddias daviddias mentioned this issue Feb 5, 2018
13 tasks
@daviddias daviddias added help wanted Seeking public contribution on this issue exp/wizard Extensive knowledge (implications, ramifications) required P1 High: Likely tackled by core team if no one steps up labels Feb 5, 2018
@pgte
Copy link
Contributor

pgte commented Feb 12, 2018

It turns out I also need global stats to implement this connection manager API: https://github.com/libp2p/js-libp2p-connection-manager/tree/master#api

@pgte
Copy link
Contributor

pgte commented Feb 12, 2018

@diasdavid what would be, in your opinion, the best place in the libp2p stack to monitor total traffic (if that's even possible)?

@daviddias
Copy link
Member

It can be a module that implements that functionality and gets plugged into js-libp2p.

@pgte
Copy link
Contributor

pgte commented Feb 12, 2018

@diasdavid and how would it hook itself into watching the traffic?

@pgte
Copy link
Contributor

pgte commented Feb 14, 2018

@diasdavid ping :)

@daviddias
Copy link
Member

@pgte wanna make a suggestion for that one?

@pgte
Copy link
Contributor

pgte commented Feb 15, 2018

OK, took me a bit to figure out how switch is handling all the outgoing and incoming connections, but here goes:

Some requirements: we need to observe unencrypted traffic in protocol-specific streams so we can compute total (unencrypted) traffic by peer, protocol, direction (upload / download) and the combinations of these ({ peer, protocol }, {peer, direction}, {peer, protocol, direction}).

In my guess, we need the following hooks on js-libp2p-switch:

On self-initiated connections:

  • We have the multiplexed protocol-specific connection in this line. We must attach an observer to this stream.

On incoming connections:

@diasdavid thoughts?

@daviddias
Copy link
Member

we need to observe unencrypted traffic in protocol-specific streams so we can compute total (unencrypted)

Unencrypted? I guess you mean that you need to observe traffic as it reaches the socket before going to SECIO (which adds a lot of overhead). The traffic won't be in plaintext once the SECIO handshake happens though.

One other option is piggybacking on the fact that we already do Connection wraps to propagate things like Observed Addrs, we can also keep a reference to the socket so that we can measure that directly.

@pgte
Copy link
Contributor

pgte commented Feb 15, 2018

@diasdavid what I mean is the exact opposite: inside the encrypted stream is an unencrypted stream. I think we should not measure the encryption overhead. Also, not sure what go-ipfs does, we should miimc that..

@diasdavid about pigging back on Connection, what would be the insertion points?

@daviddias
Copy link
Member

what I mean is the exact opposite: inside the encrypted stream is an unencrypted stream.

I see, you meant to measure the outbound throughput, before encryption and after encryption.

pigging back on Connection, what would be the insertion points?

A Connection can be both the wrap around the socket or a wrap around a muxed stream. We should have a way to visualize how much traffic the stream is contribution to the overall traffic over the socket. libp2p-switch has access to all of these, it can do that work of attaching this readers to the Connectiob object.

@pgte
Copy link
Contributor

pgte commented Feb 15, 2018

@diasdavid yup, and Ithink I want inbound traffic too, also discounting the encryption overhead.

I see the insertion points on dialled connections here, but I don't see that when accepting a new connection. @diasdavid Am I missing something?

@pgte
Copy link
Contributor

pgte commented Feb 15, 2018

@diasdavid Ha, multistream-select already returns a Connection. That's what I was missing. Thanks!

@daviddias daviddias changed the title Implement statistics Implement statistics to support Connection Management Feb 15, 2018
@daviddias
Copy link
Member

I see this is all done now -- https://github.com/libp2p/js-libp2p#switch-stats-api --, great work @pgte ❤️

@ghost ghost removed the status/ready Ready to be worked label Mar 19, 2018
maschad pushed a commit to maschad/js-libp2p that referenced this issue Jun 21, 2023
maschad pushed a commit to maschad/js-libp2p that referenced this issue Jun 21, 2023
…rs (libp2p#157)

BREAKING CHANGE: service name now defaults to `_p2p._udp.local` and no
longer uses A and AAA records -> replaced by TXT records

Added random peer name option

resolves libp2p#101

---------

Co-authored-by: Alex Potsides <[email protected]>
maschad pushed a commit to maschad/js-libp2p that referenced this issue Jun 21, 2023
## [7.0.0](libp2p/js-libp2p-mdns@v6.0.0...v7.0.0) (2023-03-07)

### ⚠ BREAKING CHANGES

* service name now defaults to `_p2p._udp.local` and no
longer uses A and AAA records -> replaced by TXT records

Added random peer name option

### Features

* update to latest spec, added peer name, announces all multiaddrs ([libp2p#157](libp2p/js-libp2p-mdns#157)) ([5edcc16](libp2p/js-libp2p-mdns@5edcc16)), closes [libp2p#101](libp2p/js-libp2p-mdns#101)

### Trivial Changes

* Update .github/workflows/semantic-pull-request.yml [skip ci] ([28c668e](libp2p/js-libp2p-mdns@28c668e))
* Update .github/workflows/semantic-pull-request.yml [skip ci] ([9dccd84](libp2p/js-libp2p-mdns@9dccd84))
* upgrade aegir to `38.1.2`  ([libp2p#182](libp2p/js-libp2p-mdns#182)) ([f86328c](libp2p/js-libp2p-mdns@f86328c))
achingbrain added a commit that referenced this issue Jun 27, 2023
Updates to latest interfaces

BREAKING CHANGE: connection and registrar APIs have changed
achingbrain pushed a commit that referenced this issue Jun 27, 2023
## [3.0.0](libp2p/js-libp2p-floodsub@v2.0.0...v3.0.0) (2022-06-17)

### ⚠ BREAKING CHANGES

* connection and registrar APIs have changed

### Trivial Changes

* update deps ([#157](libp2p/js-libp2p-floodsub#157)) ([61c95f7](libp2p/js-libp2p-floodsub@61c95f7))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
exp/wizard Extensive knowledge (implications, ramifications) required help wanted Seeking public contribution on this issue P1 High: Likely tackled by core team if no one steps up
Projects
None yet
Development

No branches or pull requests

3 participants