Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(monorepo): Merge rust services into Posthog as a subtree #22845

Merged
merged 270 commits into from
Jun 12, 2024

Conversation

neilkakkar
Copy link
Collaborator

Problem

Adjusts CI so things work for the rust services, while not borking original CI.

Currently, I'm not adding any deep integrations between the two folders, this is just step 1 of making the workflow work.

For developing in posthog, nothing changes.

For developing rust services, you need to: cd rust/ and then nothing changes, still preserving all docker compose files generated here.

Preserves git history correctly.

Changes

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Does this work well for both Cloud and self-hosted?

How did you test this code?

Ellie Huxtable and others added 30 commits September 6, 2023 11:46
* update and fix dockerfile

* Update dependencies
The other version has glibc errors. I should investigate, but don't have
time rn.
* Initial workspace setup

I'd like to tidy a few things (dependencies etc), but get this setup
asap to avoid awkward rebases going forwards.

* Format and delete
* Add metrics middleware

* Organize a bit better

* Track a couple more things

* Do not install metrics in tests

* Add some lifetimes and fixes

* Run formatter

* Simplify routes

* Fixes and feedback

* Patch builds
It's taking SO long to build on my laptop I suspect something is up with
docker. I'm 99% sure this should fix the image build though, as the
binary has moved.
I rolled out the new image, saw the old took ages to kill, and realised
we hadn't setup a SIGTERM handler. Add it!
* fill the data field in analytics events

* fix todos
* Add Redis lib

* `cargo update`

* fmt

* Add base implementation of billing limiter

Supports

1. A fixed set of limits, with no redis update
2. A fixed set, subsequently updated from redis
3. No fixed set, updates from redis

I still need to figure out how to nicely mock the redis connection that
stll leaves enough not mocked to be worth testing.

I really don't want integration tests on it :(

Also still needs connecting to the API. Reading through the python for
this is like 😵‍💫

* Rework

I've reworked it a bunch. Honestly the background loop worked but it
became really horrible and the locking behaviour a little sketchy. While
this will slow down some requests a bit, unless it becomes measurably
slow let's keep it that way rather than introducing a bit of a horrible
pattern.

* hook it all up

* Add redis read timeout

* Add non-cluster client

* Respond to feedback
Turns out we never reset updated.
* Add tracing setup

* fix

---------

Co-authored-by: Xavier Vello <[email protected]>
Copy link
Contributor

github-actions bot commented Jun 10, 2024

Size Change: 0 B

Total Size: 1.06 MB

ℹ️ View Unchanged
Filename Size
frontend/dist/toolbar.js 1.06 MB

compressed-size-action

@neilkakkar neilkakkar marked this pull request as ready for review June 11, 2024 14:43
@neilkakkar neilkakkar requested review from benjackwhite, Twixes and a team June 11, 2024 14:44
@neilkakkar
Copy link
Collaborator Author

[the snapshot tests and e2e are running on this PR because I touched those files for a test, it doesn't run by default, so shouldn't bork any rust/ developing experience]

Copy link
Contributor

@benjackwhite benjackwhite left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ignored the rust folder and focused on the workflows - all looks good to me and is a great starting point so that we can failover to this without a lot of back and forth 👍

@PostHog PostHog deleted a comment from posthog-bot Jun 11, 2024
@PostHog PostHog deleted a comment from posthog-bot Jun 11, 2024
@PostHog PostHog deleted a comment from posthog-bot Jun 11, 2024
@neilkakkar neilkakkar force-pushed the merge-repos-try2 branch 2 times, most recently from a30c668 to 65d1f80 Compare June 12, 2024 09:42
@PostHog PostHog deleted a comment from posthog-bot Jun 12, 2024
@PostHog PostHog deleted a comment from posthog-bot Jun 12, 2024
@PostHog PostHog deleted a comment from posthog-bot Jun 12, 2024
@PostHog PostHog deleted a comment from posthog-bot Jun 12, 2024
@PostHog PostHog deleted a comment from posthog-bot Jun 12, 2024
@PostHog PostHog deleted a comment from posthog-bot Jun 12, 2024
@PostHog PostHog deleted a comment from posthog-bot Jun 12, 2024
@posthog-bot
Copy link
Contributor

📸 UI snapshots have been updated

3 snapshot changes in total. 0 added, 3 modified, 0 deleted:

  • chromium: 0 added, 3 modified, 0 deleted (diff for shard 2)
  • webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

@posthog-bot
Copy link
Contributor

📸 UI snapshots have been updated

3 snapshot changes in total. 0 added, 3 modified, 0 deleted:

  • chromium: 0 added, 3 modified, 0 deleted (diff for shard 2)
  • webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

@neilkakkar neilkakkar merged commit 336367f into master Jun 12, 2024
81 checks passed
@neilkakkar neilkakkar deleted the merge-repos-try2 branch June 12, 2024 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants