Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Fix rendering of nan and inf in JSON responses #20302

Merged
merged 6 commits into from
Feb 15, 2024

Conversation

webjunkie
Copy link
Contributor

@webjunkie webjunkie commented Feb 13, 2024

Problem

We still have a problem with inf and nan values in JSON, if they appear in tuples.

Changes

  • we can just swap out the whole JSON serializer 👏🏻
  • use orjson 🚀
  • it makes nan and inf values null by default, no more traversing the whole response with our own loops 🚀
  • it's also anyways faster than standard JSON serialization because Rust 🚀
  • adjusted some dict keys to strings: previously they also got serialized to strings, and now orjson would've complained
  • and also added a test for these tuples ^^

How did you test this code?

  • tests

@webjunkie webjunkie changed the title fix: Fix rendering of nan and inf fix: Fix rendering of nan and inf in JSON responses Feb 13, 2024
@webjunkie webjunkie marked this pull request as ready for review February 13, 2024 20:21
@webjunkie webjunkie requested a review from a team February 14, 2024 08:07
Copy link
Collaborator

@mariusandra mariusandra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I get some of the changes to string, but why not.

@@ -187,7 +187,7 @@ def property_definitions(self, request: request.Request, **kw):

group_type_index_to_properties = defaultdict(list)
for group_type_index, key, count in rows:
group_type_index_to_properties[group_type_index].append({"name": key, "count": count})
group_type_index_to_properties[str(group_type_index)].append({"name": key, "count": count})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand this change 🤔

Copy link
Contributor Author

@webjunkie webjunkie Feb 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They became string keys anyway with the previous JSON serializer. This one threw an error, unless we flip an option. Since we had it as string, I explicitly made it string so the serializer does not complain.
Sorry, could have mentioned in changes.

@@ -565,7 +565,7 @@ def local_evaluation(self, request: request.Request, **kwargs):
seen_cohorts_cache[id] = cohort or ""

if cohort and not cohort.is_static:
cohorts[cohort.pk] = cohort.properties.to_dict()
cohorts[str(cohort.pk)] = cohort.properties.to_dict()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... or this. Why convert them to strings?

@webjunkie webjunkie merged commit b834212 into master Feb 15, 2024
74 checks passed
@webjunkie webjunkie deleted the fix/json-render-inf-nan branch February 15, 2024 15:56
Copy link

sentry-io bot commented Feb 15, 2024

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

  • ‼️ TypeError: str is not valid UTF-8: surrogates not allowed /api/projects/{parent_lookup_team_id}/session_r... View Issue
  • ‼️ TypeError: Integer exceeds 64-bit range /api/projects/{parent_lookup_team_id}/persons/ View Issue

Did you find this useful? React with a 👍 or 👎

@neilkakkar
Copy link
Collaborator

@webjunkie friendly reminder to tag me/Juraj whenever anything changes to feature flags / local_evaluation / decide api endpoints, no matter how small. No expectation to block on us, but would just like to be aware. Glad the tests were sufficient here to get the exact format we needed 👍

@webjunkie
Copy link
Contributor Author

Got it! Was there some issue with it?

@neilkakkar
Copy link
Collaborator

Nope, and I don't expect there to be any in the future either 😅 #WeTrustOurTests - but just for awareness

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants