Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: discard diffs that are just reordering of lines #1035

Closed
wants to merge 1 commit into from

Conversation

adityathebe
Copy link
Member

@adityathebe adityathebe commented Oct 7, 2024

resolves: #1015

@moshloop I think this is a more efficient way to filter out changes caused by re-ordering of .status.conditions.

We don't have to unmarshal and sort and works for diffs from any scrapers.

if we really want to limit this to .status.conditions check then that's also possible.

Copy link

gitguardian bot commented Oct 7, 2024

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them.
Once a secret has been leaked into a git repository, you should consider it compromised, even if it was deleted immediately.
Find here more information about risks.


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@adityathebe adityathebe changed the title feat: disregard diffs that are just reordering of lines feat: discard diffs that are just reordering of lines Oct 7, 2024
@adityathebe
Copy link
Member Author

adityathebe commented Oct 7, 2024

With reordering

- Completed scrape with types=59, updated=1225, unchanged=3, ignored=1, orphaned=4 in 17.315s (allocs=36.1b, heap_allocs=17.3G heap_increase=111.9M, gc_count=0 rss=549.2M peak=549.2M)
- Completed scrape with types=59, updated=1225, unchanged=3, ignored=1, orphaned=4 in 18.172s (allocs=19.8b, heap_allocs=14.5G heap_increase=69.4M, gc_count=0 rss=347M peak=347M)

Without reordering

- Completed scrape with types=59, updated=1225, unchanged=3, ignored=1, orphaned=4 in 17.37s (allocs=35.5b, heap_allocs=17.2G heap_increase=85.7M, gc_count=0 rss=448.5M peak=448.5M)
- Completed scrape with types=59, updated=1225, unchanged=3, ignored=1, orphaned=4 in 16.957s (allocs=34.1b, heap_allocs=17.1G heap_increase=61.5M, gc_count=0 rss=477.6M peak=477.6M)
- Completed scrape with types=59, updated=1225, unchanged=3, ignored=1, orphaned=4 in 17.807s (allocs=35.4b, heap_allocs=17.2G heap_increase=133M, gc_count=0 rss=620.5M peak=620.5M)

This new check performed on ~2000 diffs of kubernetes objects adds an additional overhead of ~200ms on average

@moshloop

@moshloop
Copy link
Member

@adityathebe - I am not comfortable with this - I think there is too many edge cases, it also doesn't improve the diff which will still have many reordering changes if a condition does change - We should just sort the node.conditions in the kubernetes scraper before json marshalling

@moshloop moshloop closed this Oct 29, 2024
@adityathebe adityathebe deleted the feat/ignore-reordering-diffs branch November 6, 2024 03:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Kubernetes - Sort status.conditions by reason
2 participants