Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(api): Add Pydantic schemas in OpenAPI annotations #19408

Merged
merged 9 commits into from
Jan 12, 2024

Conversation

webjunkie
Copy link
Contributor

@webjunkie webjunkie commented Dec 19, 2023

Problem

We don't or can't use our Pydantic models as schemas in our OpenAPI spec.
We also don't use discriminate unions to have the model validation work better.

Changes

  • upgrade libraries: drf-spectacular can now handle Pydantic 🥳 -> moved to chore: Upgrade pydantic, mypy, and others #19660
  • adjust query endpoint, add correct schemas
    • thus we can get rid of the manual parameter definitions
  • simplified some stuff
  • added discriminator by kind
    • this will make errors point to the specific models
  • used isinstance checks since we now can work with the models there
  • remove Pydantic parser:
    • it's more straightforward to just use a method (now in a mixin) to validate the model
    • a parser is useful to parse different types of data, we still have JSON and still want dict
    • side note: I experimented with keeping the parser and make it return request.data as the Pydantic model, but everything in Django/DRF expects request.data to be a dict, so that's a no-go

How did you test this code?

  • checked tests
  • added tests

@webjunkie webjunkie changed the title Feature/openapi pydantic feat(api): Add Pydantic schemas in OpenAPI annotations Dec 19, 2023
Copy link
Contributor

github-actions bot commented Dec 19, 2023

Size Change: 0 B

Total Size: 2 MB

ℹ️ View Unchanged
Filename Size
frontend/dist/toolbar.js 2 MB

compressed-size-action

@posthog-bot
Copy link
Contributor

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

  • chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
  • webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

@posthog-bot
Copy link
Contributor

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

  • chromium: 0 added, 1 modified, 0 deleted (diff for shard 2)
  • webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

posthog/api/parsers.py Fixed Show fixed Hide fixed
@posthog-bot
Copy link
Contributor

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

@webjunkie webjunkie force-pushed the feature/openapi-pydantic branch from 764a7c6 to a1ab542 Compare January 10, 2024 10:11
posthog/api/mixins.py Dismissed Show dismissed Hide dismissed
@webjunkie webjunkie marked this pull request as ready for review January 10, 2024 16:09
@webjunkie webjunkie requested a review from mariusandra January 10, 2024 16:09
Copy link
Collaborator

@mariusandra mariusandra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work 👍

Looks good to my untrained eye. I noticed some weirdness in one new generated Python class, but nothing blocking if you think this is ready to go.

Comment on lines 136 to 142
// Dynamically make a union type out of all the "response" fields in QuerySchema
type ResponseType<T> = T extends { response: infer R } ? R : never
type AllResponses = ResponseType<QuerySchema>
type Unionize<T> = Partial<{
[P in keyof T]: T[P]
}>
export type QueryCombinedResponse = Unionize<AllResponses>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

neat!

posthog/schema.py Outdated Show resolved Hide resolved
posthog/schema.py Outdated Show resolved Hide resolved
# query_json has been parsed by QuerySchemaParser
# it _should_ be impossible to end up in here with a "bad" query
query_kind = query_json.get("kind")
tag_queries(query=query_json)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mariusandra not sure what this does, and I don't have a dict anymore that I can pass to it

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds a copy of the input query into ClickHouse's system.query_log table.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what can we do? I don't want to serialize again.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tag the queries where I still have the dict.

@webjunkie webjunkie merged commit 456af51 into master Jan 12, 2024
100 checks passed
@webjunkie webjunkie deleted the feature/openapi-pydantic branch January 12, 2024 12:01
webjunkie added a commit that referenced this pull request Jan 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants