PSL auto-update #3
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
When the rule is stored, we can remove the value from the Rule as
the value if effectively the key of the Hash.
➜ publicsuffix-ruby git:(before) ruby test/profilers/initialization_profiler.rb
Total allocated: 5882690 bytes (52219 objects)
Total retained: 1375819 bytes (24188 objects)
➜ publicsuffix-ruby git:(before) ruby test/profilers/execution_profiler.rb
Total allocated: 15170 bytes (160 objects)
Total retained: 0 bytes (0 objects)
➜ publicsuffix-ruby git:(after) ✗ ruby test/profilers/initialization_profiler.rb
Total allocated: 6205130 bytes (60280 objects)
Total retained: 1052404 bytes (16127 objects)
➜ publicsuffix-ruby git:(after) ✗ ruby test/profilers/execution_profiler.rb
Total allocated: 15330 bytes (164 objects)
Total retained: 0 bytes (0 objects)
compared to master
➜ publicsuffix-ruby git:(master) ruby test/profilers/initialization_profiler.rb
Total allocated: 6525758 bytes (72086 objects)
Total retained: 1020387 bytes (19234 objects)
➜ publicsuffix-ruby git:(master) ruby test/profilers/execution_profiler.rb
Total allocated: 204162 bytes (4420 objects)
Total retained: 0 bytes (0 objects)
Execution time is unchanged.
➜ publicsuffix-ruby git:(before) ruby test/benchmarks/bm_find.rb
user system total real
NAME_SHORT 0.260000 0.000000 0.260000 ( 0.262684)
NAME_SHORT (noprivate) 0.370000 0.010000 0.380000 ( 0.372534)
NAME_MEDIUM 0.330000 0.000000 0.330000 ( 0.335683)
NAME_MEDIUM (noprivate) 0.490000 0.000000 0.490000 ( 0.494590)
NAME_LONG 0.510000 0.010000 0.520000 ( 0.519750)
NAME_LONG (noprivate) 0.590000 0.000000 0.590000 ( 0.594626)
NAME_WILD 0.480000 0.000000 0.480000 ( 0.490432)
NAME_WILD (noprivate) 0.580000 0.010000 0.590000 ( 0.594776)
NAME_EXCP 0.460000 0.000000 0.460000 ( 0.470119)
NAME_EXCP (noprivate) 0.590000 0.010000 0.600000 ( 0.601316)
IAAA 0.300000 0.000000 0.300000 ( 0.305301)
IAAA (noprivate) 0.400000 0.000000 0.400000 ( 0.410586)
IZZZ 0.280000 0.000000 0.280000 ( 0.283711)
IZZZ (noprivate) 0.400000 0.010000 0.410000 ( 0.408137)
PAAA 0.490000 0.000000 0.490000 ( 0.501869)
PAAA (noprivate) 0.600000 0.000000 0.600000 ( 0.612187)
PZZZ 0.510000 0.010000 0.520000 ( 0.519206)
PZZZ (noprivate) 0.590000 0.000000 0.590000 ( 0.600264)
JP 0.390000 0.000000 0.390000 ( 0.404432)
JP (noprivate) 0.540000 0.010000 0.550000 ( 0.558351)
IT 0.290000 0.000000 0.290000 ( 0.298931)
IT (noprivate) 0.410000 0.000000 0.410000 ( 0.420742)
COM 0.290000 0.010000 0.300000 ( 0.300935)
COM (noprivate) 0.400000 0.000000 0.400000 ( 0.409309)
➜ publicsuffix-ruby git:(after) ✗ ruby test/benchmarks/bm_find.rb
user system total real
NAME_SHORT 0.320000 0.000000 0.320000 ( 0.320201)
NAME_SHORT (noprivate) 0.430000 0.000000 0.430000 ( 0.443678)
NAME_MEDIUM 0.380000 0.000000 0.380000 ( 0.388169)
NAME_MEDIUM (noprivate) 0.490000 0.010000 0.500000 ( 0.491073)
NAME_LONG 0.480000 0.000000 0.480000 ( 0.483376)
NAME_LONG (noprivate) 0.620000 0.010000 0.630000 ( 0.634896)
NAME_WILD 0.570000 0.020000 0.590000 ( 0.628489)
NAME_WILD (noprivate) 0.700000 0.030000 0.730000 ( 0.769070)
NAME_EXCP 0.580000 0.020000 0.600000 ( 0.618683)
NAME_EXCP (noprivate) 0.740000 0.030000 0.770000 ( 0.799244)
IAAA 0.410000 0.030000 0.440000 ( 0.474761)
IAAA (noprivate) 0.550000 0.040000 0.590000 ( 0.645329)
IZZZ 0.380000 0.020000 0.400000 ( 0.432898)
IZZZ (noprivate) 0.520000 0.020000 0.540000 ( 0.579073)
PAAA 0.680000 0.040000 0.720000 ( 0.760276)
PAAA (noprivate) 0.720000 0.020000 0.740000 ( 0.773864)
PZZZ 0.700000 0.040000 0.740000 ( 0.782113)
PZZZ (noprivate) 0.650000 0.010000 0.660000 ( 0.664647)
JP 0.470000 0.000000 0.470000 ( 0.478473)
JP (noprivate) 0.580000 0.010000 0.590000 ( 0.589827)
IT 0.360000 0.000000 0.360000 ( 0.379309)
IT (noprivate) 0.450000 0.010000 0.460000 ( 0.471794)
COM 0.330000 0.010000 0.340000 ( 0.334253)
COM (noprivate) 0.530000 0.030000 0.560000 ( 0.592813)
Using the new benchmarks introduced in dec53e6, the allocation is clearly lower even during execution time. ➜ publicsuffix-ruby git:(master) ✗ ruby test/profilers/find_profiler.rb Total allocated: 31472 bytes (691 objects) Total retained: 0 bytes (0 objects) ➜ publicsuffix-ruby git:(master) ✗ ruby test/profilers/domain_profiler.rb Total allocated: 37410 bytes (744 objects) Total retained: 0 bytes (0 objects) vs ➜ publicsuffix-ruby git:(thesis-hash) ruby test/profilers/find_profiler.rb Total allocated: 1264 bytes (22 objects) Total retained: 0 bytes (0 objects) ➜ publicsuffix-ruby git:(thesis-hash) ruby test/profilers/domain_profiler.rb Total allocated: 7202 bytes (75 objects) Total retained: 0 bytes (0 objects)
.new now takes all parameters, as you would create a completely new instance when you have the data. A new method called .build is used to create a new Rule from a rule content.
Better distinguish between a Rule (public API) and an Entry (internal API).
Just keep it simple. The difference is not very noticeable. There is now a separate benchmark to check extensively on all rules.
A very simple memory profiles that checks the full size of a variable by serializing into a binary file. Yes, I know this is very rough, but there are cases where ObjectSpace.memsize_of doesn't cooperate, and this is one of the possible workarounds.
It doesn't support keyword arguments with no default, and proper memory profiling.
Switch List implementation to use Hash-based lookup.
Before
$ ruby test/benchmarks/bm_find.rb
user system total real
NAME_SHORT 1.540000 0.000000 1.540000 ( 1.560285)
NAME_MEDIUM 1.740000 0.020000 1.760000 ( 1.774570)
NAME_LONG 2.050000 0.010000 2.060000 ( 2.101608)
NAME_WILD 0.630000 0.010000 0.640000 ( 0.633376)
NAME_EXCP 0.660000 0.000000 0.660000 ( 0.663655)
IAAA 0.710000 0.000000 0.710000 ( 0.712431)
IZZZ 0.620000 0.000000 0.620000 ( 0.621207)
PAAA 6.900000 0.060000 6.960000 ( 7.105149)
PZZZ 0.930000 0.000000 0.930000 ( 0.932058)
JP 51.190000 0.430000 51.620000 ( 52.718784)
IT 9.110000 0.030000 9.140000 ( 9.183792)
COM 7.580000 0.010000 7.590000 ( 7.591188)
$ ruby test/profilers/list_profsize.rb
301,518 PublicSuffix::List size
247,194 Size of rules
54,287 Size of indexes
$ ruby test/profilers/initialization_profiler.rb
Total allocated: 6525680 bytes (72086 objects)
Total retained: 1020309 bytes (19234 objects)
allocated memory by class
-----------------------------------
3819072 Hash
1826448 String
557440 Array
320080 PublicSuffix::Rule::Normal
2040 PublicSuffix::Rule::Wildcard
320 PublicSuffix::Rule::Exception
240 File
40 PublicSuffix::List
allocated objects by class
-----------------------------------
38284 String
16124 Hash
9615 Array
8002 PublicSuffix::Rule::Normal
51 PublicSuffix::Rule::Wildcard
8 PublicSuffix::Rule::Exception
1 File
1 PublicSuffix::List
retained memory by class
-----------------------------------
389541 String
320080 PublicSuffix::Rule::Normal
229560 Array
78728 Hash
2040 PublicSuffix::Rule::Wildcard
320 PublicSuffix::Rule::Exception
40 PublicSuffix::List
retained objects by class
-----------------------------------
9617 String
8002 PublicSuffix::Rule::Normal
1554 Array
51 PublicSuffix::Rule::Wildcard
8 PublicSuffix::Rule::Exception
1 Hash
1 PublicSuffix::List
Allocated String Report
-----------------------------------
1796 "jp"
1712 ""
761 "no"
...
Retained String Report
-----------------------------------
2 "aaa"
2 "aarp"
2 "abarth"
...
$ ruby test/profilers/find_profiler.rb
Total allocated: 31472 bytes (691 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
-----------------------------------
26640 String
2840 Array
584 Hash
584 RubyVM::Env
400 Proc
288 Enumerator::Lazy
48 Enumerator::Generator
48 Enumerator::Yielder
40 PublicSuffix::Rule::Wildcard
allocated objects by class
-----------------------------------
666 String
5 Array
5 Hash
5 Proc
5 RubyVM::Env
2 Enumerator::Lazy
1 Enumerator::Generator
1 Enumerator::Yielder
1 PublicSuffix::Rule::Wildcard
retained memory by class
-----------------------------------
NO DATA
retained objects by class
-----------------------------------
NO DATA
After
$ ruby test/benchmarks/bm_find.rb
user system total real
NAME_SHORT 0.370000 0.000000 0.370000 ( 0.376614)
NAME_MEDIUM 0.480000 0.000000 0.480000 ( 0.489633)
NAME_LONG 0.590000 0.010000 0.600000 ( 0.603704)
NAME_WILD 0.570000 0.000000 0.570000 ( 0.577077)
NAME_EXCP 0.700000 0.010000 0.710000 ( 0.709454)
IAAA 0.400000 0.000000 0.400000 ( 0.406585)
IZZZ 0.440000 0.000000 0.440000 ( 0.436526)
PAAA 0.790000 0.010000 0.800000 ( 0.833797)
PZZZ 0.740000 0.000000 0.740000 ( 0.758879)
JP 0.760000 0.010000 0.770000 ( 0.777570)
IT 0.400000 0.000000 0.400000 ( 0.394240)
COM 0.400000 0.000000 0.400000 ( 0.399312)
$ ruby test/profilers/list_profsize.rb
263,481 PublicSuffix::List size
263,451 Size of rules
$ ruby test/profilers/initialization_profiler.rb
Total allocated: 6205052 bytes (60280 objects)
Total retained: 1052326 bytes (16127 objects)
allocated memory by class
-----------------------------------
4143744 Hash
1416148 String
322440 PublicSuffix::Rule::Entry
320080 PublicSuffix::Rule::Normal
2040 PublicSuffix::Rule::Wildcard
320 PublicSuffix::Rule::Exception
240 File
40 PublicSuffix::List
allocated objects by class
-----------------------------------
28032 String
16124 Hash
8061 PublicSuffix::Rule::Entry
8002 PublicSuffix::Rule::Normal
51 PublicSuffix::Rule::Wildcard
8 PublicSuffix::Rule::Exception
1 File
1 PublicSuffix::List
retained memory by class
-----------------------------------
403400 Hash
326446 String
322440 PublicSuffix::Rule::Entry
40 PublicSuffix::List
retained objects by class
-----------------------------------
8064 String
8061 PublicSuffix::Rule::Entry
1 Hash
1 PublicSuffix::List
Retained String Report
-----------------------------------
1 "*.compute.amazonaws.com.cn"
1 "*.githubcloudusercontent.com"
1 "0.bg"
...
$ ruby test/profilers/find_profiler.rb
Total allocated: 1728 bytes (24 objects)
Total retained: 0 bytes (0 objects)
allocated memory by class
-----------------------------------
1048 Hash
520 String
80 Array
40 PublicSuffix::Rule::Normal
40 PublicSuffix::Rule::Wildcard
allocated objects by class
-----------------------------------
13 String
7 Hash
2 Array
1 PublicSuffix::Rule::Normal
1 PublicSuffix::Rule::Wildcard
retained memory by class
-----------------------------------
NO DATA
retained objects by class
-----------------------------------
NO DATA
[DOCS] corrects param name in List#parse
…-comments Since by default, the `*` rule is being applied, some code examples were incorrect in code comments. I've also added strict checking examples to the README.
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 5 to 6. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v5...v6) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Co-authored-by: weppos <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Automated changes by create-pull-request GitHub action