Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blog: matrix-appservice-irc ignoring dead state reset homeservers if appservice doesn't have power? What to not do with bridging? #242

Open
Mikaela opened this issue Jul 15, 2021 · 3 comments
Assignees
Labels
blog Blog ideas and issues [m] The Matrix protocol or touching it somehow question

Comments

@Mikaela
Copy link
Owner

Mikaela commented Jul 15, 2021

Apparently I misunderstood the log in #241 and what is wanted is the story of what can go wrong with bridging.


What can go wrong with bridging? People may want to bridge to IRC without understanding IRC and thus neglete to register themselves to NickServ, not register their channels in ChanServ and in case of IRCnet not have real IRC users or set mode +R to recover ops.


Case Disroot.

  • Matrix: Room admins lost after originating server goes down matrix-org/synapse#9139
  • freenode: used a separate project account which held control until it expired in an old database reset (archive.org link?) leaving me as the only one ChanServ registered
  • freenode vs freenode classic period: #disroot is the biggest public channel of freenode (as everyone who isn't a ghost Matrix user has moved to Libera.Chat and #freenode itself was +s) and due to the above issue @appservice:matrix.org has no power. (Screenshots of IRC side exist, find them?)
    • result: users on #disroot:disroot.org are immune to disconnection kicker including all those users removed in 2018 and some other dead homeservers (including at least one belonging to my friend) keep getting connected to freenode since then taking connection slots in connection exemption and using resources by receiving the protocol traffic (ping/pong/privmsg/...)
    • meta: during the mass migrations Disroot attempts to regain control of the channel by emailing (the new) freenode ticketing system and asking staff for getting the channel back, but the staff nicely gives the channel ownership to me upon my first asking nicely request likely without checking at all whether I am authorized to do that (or that I am the only one in the old access list)
    • meta2: Disroot mildly takes offense on the channel being given to someone random (what if I hadn't just given ownership back to the real owner?) and discusses with staff other questionable incidents such as takeover of a lot of channels (didn't affect Disroot due to topic not mentioning other networks) such as Wesnoth project's private channel and staff representative gets hung up on CVE's being public information (ignoring existence of reserved CVE numbers) not caring about the point that network staff shouldn't join channels that are private/invite-only etc.
    • aftermath: Disroot setups a temporary(?) homeserver with the old signing key to regain power in the room and performs a "room upgrade" resulting the bridge to finally disconnects hundreds of users from freenode from 1000+ to 50+.
    • aftermath2: Disroot no longer uses freenode and a couple of days later freenode drops their services.db

The above has been written from memory, I don't like keeping IRC logs by myself, the existence of the linked issue can be confirmed and I imagine a lot of people who used to be on #disroot did keep logs or someone curious enough could dig them up from Matrix. Mentioning the problem to bridge maintainer in an offtopic room may have resulted into a hah or something similar small and the discussion may be findable from said room history (if it's still alive) possibly using the dead-homeserver-owning-friend's nickname as a keyword. I mentioned a screenshot, but I am not going to start digging for one at 01 am as I have bad sleep cycle already.

I have no experience running bridges by myself, other than maybe matterbridge that I consider as a relaybot (or stateless bridge). I haven't selfhosted either Matrix or XMPP, but do selfhost IRC opering on a couple of other networks too. I am not sure of this event's noteworthiness especially as I don't personally have much to back the claims I make.

@Mikaela Mikaela added question blog Blog ideas and issues labels Jul 15, 2021
@Mikaela Mikaela self-assigned this Jul 15, 2021
@Mikaela
Copy link
Owner Author

Mikaela commented Jul 15, 2021

The #disroot being biggest public channel of freenode was so absurd with me being the only op, that any logs may look like I was having hypomania or something similar.

I also remember that at the time there were read-only ##ob-* channels on Libera.Chat which users may also have logs of staff member granting me the ownership of #disroot.

@muppeth
Copy link

muppeth commented Jul 15, 2021

It must have felt awesome for a while while the freenode issue unfolded wasnt it hahahahaha (being on top of the list)....

Agree the whole irc bridging using matrix irc bridge to freenode early on was not that well handled.
I made a big mistake of registering a nick just for bridging on freenode not knowing that the nick will be stripped off the room ownership if it does not login for a while (my mistake of not reading documentation correctly). As a result we have lost the ownership of the room. I should have work on regaining the ownership as soon as I noticed that but I was fullishly thinking that freenode aint going anywhere and if we need ownership we could easily prove we are owners of the channel and get it back. How this ended up, you described above, so yeah thanks for taking care of this mess and a big middle finger to those in charge of freenode at the time our request was ignored and ownership was given to a random person that asked (thankfully it was you)

The matrix issue on the other hand was a seperate bag and IMO just shows how badly things with matrix are (apparently resolved in later so called versions of the room but still I could imagine issue we had affecting some more critical usecase where recovery was impossible). Before we have switched off the lights on our server, I have created a user account on another server and gave all the rights to that users (and some of the other users in community including you actually) to prevent situation where we loose the room. However, looks like the moment the original disroot server was down for a while, and because of the fact that user on disroot.org was the owner, for some reason all admins were dropped by matrix. I did still had the signing keys of the server in some of the backups, so I could actually recreate the server, but that took a while to find the keys as I removed everything ,matrix related the moment we switched off the lights (all thanks to first deployments of synapse where I took a snapshot of the synapse config and keys and saved it on some temp dir on the server). It took me a while to find it and I initially thought I dont have 'em, so imagine how stressed I have been everytime some troll would show up. I was only hoping noone will notice there is no admin in the room and people cold just do whatever they want. Ton of stress for quite few weeks :P

Fortunatelly things went fine but I surely learn my lesson and now that you mention this, I should give more admin privs to both matrix room and irc ones :P

@Mikaela
Copy link
Owner Author

Mikaela commented Jan 9, 2022

Case Disroot may happen again with Feneas disassemblying.

My comments on #feneas.org:matrix.org:

https://ems.element.io/tools/matrix-migration

on Matrix migrations, if there are room version older than 6, it may be desirable to /upgraderoom 9 to avoid state reset and losing power from new account. This mainly affects versions 1 and 2 to my knowledge, 6 is the one Conduit homeserver software supports the best and 9 is the latest one to my knowledge

Sources: Room admins lost in v1 when origin homeserver goes down, State resets still happen in v2 rooms, A lot of power levels lost through state reset in version 4, Conduit shouldn't claim support for room version 5, Conduit issue on room versions 7, 8, 9

Tulir adds:

note that "state resets still happen in v2 rooms" means state resets still happen in all room versions

Additionally it may be interesting that [Disroot room v1](matrix:roomid/uDQoIebqsjEEtmWLrO:disroot.org?action=join&via=matrix.org&via=tedomum.net&via=the-apothecary.club) is still accessible through Matrix URI scheme and contains 1370 users according to matrix.org.

Neither GitHub or Element Web, Element Android and Element iOS supports the Matrix URI scheme.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blog Blog ideas and issues [m] The Matrix protocol or touching it somehow question
Projects
None yet
Development

No branches or pull requests

2 participants