Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

automod/hepa refactors #700

Open
7 tasks
bnewbold opened this issue Jul 10, 2024 · 0 comments
Open
7 tasks

automod/hepa refactors #700

bnewbold opened this issue Jul 10, 2024 · 0 comments

Comments

@bnewbold
Copy link
Collaborator

  • refactor the "consumption" code (firehose and ozone events) out of the cmd/hepa/ folder and in to automod/something/
  • entirely rip out PDS/entryway private metadata queries, once we are confident ozone is sufficient
  • get publicly-available account creation timestamp working consistently. this could include tweaks to appview+ozone endpoints, and possibly even going to PLC directory for more authoritative timestamps
  • refactor account metadata struct in to more clearly distinct "bsky-specific" (from appview), "ozone-specific" (private metadata), and public account/identity info. this will require a careful deploy
  • should probably handle account metadata more consistently across ozone, automod, and other systems. related to the #identity and #account refactors, we now have a concept of account status at various pieces of infrastructure (active, takendown, deactivated, suspended, deleted), and should represent that cleanly
  • purge identity caches separate from account metadata caches
  • automod "flags" and ozone "tags" are kind of duplicate. I originally thought these would have automod "flags" show up as "tags" in Ozone. but should think that through again (maybe flags are useful outside ozone?) and how to refactor things if they should be the same concept
bnewbold added a commit that referenced this issue Jul 12, 2024
This has automod consume events from Ozone and feed them in to the rules
engine, similar to firehose events. It uses the queryEvents endpoint,
polling every 5 seconds. This is all optional; if an ozone admin client
isn't configured, it doesn't happen.

- some types of ozone events on an account subject result in account
metadata getting flushed. this ensures that, eg, manually-added labels
or takedowns end up in automod-visible cached account metadata
- harassment protection rule is updated to look in private tag state
from ozone, not just a deployed config JSON file. this was actually more
complex than I would thought, and results in additional account metadata
fetches, but those are all cached (with cache purging!) so should be
fine
- private/admin account metadata queries go to ozone, not PDS/entryway,
by default. this could increase API load on ozone a bunch; we might need
to optimize that code path
- ignores events caused by mod service automation (eg, automod itself)
- does not currently have any tests 😞 but I tested a fair
bit in staging. would go slow and careful with any prod deploy

Some follow-ups for future work (recording these from my local dev
notes):

- automod "flags" and ozone "tags" are kind of duplicate. I originally
thought these would have automod "flags" show up as "tags" in Ozone. but
should think that through again (maybe flags are useful outside ozone?)
and how to refactor things if they should be the same concept
- entirely rip out PDS/entryway private metadata queries, once we are
confident ozone is sufficient
- get publicly-available account creation timestamp working
consistently. this could include tweaks to appview+ozone endpoints, and
possibly even going to PLC directory for more authoritative timestamps
- should probably refactor account metadata struct in to more clearly
distinct "bsky-specific" (from appview), "ozone-specific" (private
metadata), and public account/identity info
- should probably handle account metadata more consistently across
ozone, automod, and other systems. related to the `#identity` and
`#account` refactors, we now have a concept of account status at various
pieces of infrastructure (active, takendown, deactivated, suspended,
deleted), and should represent that cleanly
- `OzoneEventContext` currently extends `AccountContext`, with the
subject account as "the account". currently all ozone events have an
associated subject account, while record is optional. we might end up
with events that don't have a meaningful subject account? or want to
have better ergonomics with a record-variant of ozone event context?
those are more complex, feels like things are fine for now
- would like to refactor the "consumption" code (firehose and ozone
events) out of the `cmd/hepa/` folder and in to `automod/`

Update: created a separate issue to track these refactors:
#700
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant