Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache normalized string #2777

Closed

Conversation

turbolent
Copy link
Member

@turbolent turbolent commented Sep 11, 2023

Work towards #2752

Description

String functions, like checking for equality, already operate on the normalized form of the string.
However, the results are currently discarded.

As strings are immutable, we can cache the normalized form and re-use it.

As the hash value is changed, this will need a migration. Given that we can perform breaking changes in the Stable Cadence release, we should use this opportunity.


  • Targeted PR against master branch
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work
  • Code follows the standards mentioned here
  • Updated relevant documentation
  • Re-reviewed Files changed in the Github PR explorer
  • Added appropriate labels

@turbolent turbolent self-assigned this Sep 11, 2023
@turbolent turbolent added the Storage Breaking Change Breaks existing stored data (needs a storage migration) label Sep 11, 2023
@github-actions
Copy link

Cadence Benchstat comparison

This branch with compared with the base branch onflow:feature/stable-cadence commit 9a146ab
The command for i in {1..N}; do go test ./... -run=XXX -bench=. -benchmem -shuffle=on; done was used.
Bench tests were run a total of 7 times on each branch.

Collapsed results for better readability

old.txtnew.txt
time/opdelta
DecodeBatchEventsCCF-2166ms ± 0%165ms ± 0%~(p=1.000 n=1+1)
DecodeBatchEventsJSON-2488ms ± 0%719ms ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowFees.FeesDeducted-24.85µs ± 0%5.05µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowFees.TokensWithdrawn-23.72µs ± 0%4.01µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-23.66µs ± 0%5.42µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-24.11µs ± 0%6.12µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowIDTableStaking.NewWeeklyPayout-22.78µs ± 0%2.77µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowIDTableStaking.RewardsPaid-23.23µs ± 0%3.21µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowToken.TokensDeposited-23.27µs ± 0%3.29µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowToken.TokensDeposited_with_nil_receiver-23.48µs ± 0%3.15µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowToken.TokensMinted-22.71µs ± 0%2.68µs ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowToken.TokensWithdrawn-23.30µs ± 0%3.34µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowFees.FeesDeducted-212.3µs ± 0%12.2µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowFees.TokensWithdrawn-27.13µs ± 0%7.15µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-210.9µs ± 0%10.9µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-214.9µs ± 0%15.1µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowIDTableStaking.NewWeeklyPayout-27.22µs ± 0%7.21µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowIDTableStaking.RewardsPaid-29.24µs ± 0%9.23µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowToken.TokensDeposited-29.70µs ± 0%9.64µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowToken.TokensDeposited_with_nil_receiver-28.74µs ± 0%8.75µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowToken.TokensMinted-27.19µs ± 0%7.24µs ± 0%~(p=1.000 n=1+1)
DecodeJSON/FlowToken.TokensWithdrawn-29.66µs ± 0%9.63µs ± 0%~(p=1.000 n=1+1)
EncodeBatchEventsCCF-2127ms ± 0%128ms ± 0%~(p=1.000 n=1+1)
EncodeBatchEventsJSON-2143ms ± 0%143ms ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowFees.FeesDeducted-23.10µs ± 0%3.10µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowFees.TokensWithdrawn-22.23µs ± 0%2.20µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-22.83µs ± 0%2.79µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-23.61µs ± 0%3.59µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowIDTableStaking.NewWeeklyPayout-22.23µs ± 0%2.23µs ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.RewardsPaid-22.46µs ± 0%2.46µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowToken.TokensDeposited-22.57µs ± 0%2.56µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowToken.TokensDeposited_with_nil_receiver-22.53µs ± 0%2.52µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowToken.TokensMinted-22.20µs ± 0%2.20µs ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowToken.TokensWithdrawn-22.57µs ± 0%2.56µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowFees.FeesDeducted-23.55µs ± 0%3.46µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowFees.TokensWithdrawn-22.14µs ± 0%2.02µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-22.97µs ± 0%2.94µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-24.19µs ± 0%4.17µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowIDTableStaking.NewWeeklyPayout-22.04µs ± 0%2.03µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowIDTableStaking.RewardsPaid-22.58µs ± 0%2.63µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowToken.TokensDeposited-22.98µs ± 0%2.99µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowToken.TokensDeposited_with_nil_receiver-22.33µs ± 0%2.35µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowToken.TokensMinted-22.05µs ± 0%2.03µs ± 0%~(p=1.000 n=1+1)
EncodeJSON/FlowToken.TokensWithdrawn-22.94µs ± 0%2.92µs ± 0%~(p=1.000 n=1+1)
ExportType/composite_type-2406ns ± 0%408ns ± 0%~(p=1.000 n=1+1)
ExportType/simple_type-283.8ns ± 0%85.1ns ± 0%~(p=1.000 n=1+1)
InterpretRecursionFib-22.64ms ± 0%2.63ms ± 0%~(p=1.000 n=1+1)
NewInterpreter/new_interpreter-21.23µs ± 0%1.23µs ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-2630ns ± 0%622ns ± 0%~(p=1.000 n=1+1)
ParseArray-27.64ms ± 0%7.49ms ± 0%~(p=1.000 n=1+1)
ParseDeploy/byte_array-211.4ms ± 0%11.3ms ± 0%~(p=1.000 n=1+1)
ParseDeploy/decode_hex-21.22ms ± 0%1.30ms ± 0%~(p=1.000 n=1+1)
ParseFungibleToken/With_memory_metering-2205µs ± 0%193µs ± 0%~(p=1.000 n=1+1)
ParseFungibleToken/Without_memory_metering-2152µs ± 0%152µs ± 0%~(p=1.000 n=1+1)
ParseInfix-27.31µs ± 0%7.09µs ± 0%~(p=1.000 n=1+1)
QualifiedIdentifierCreation/One_level-22.41ns ± 0%2.41ns ± 0%~(all equal)
QualifiedIdentifierCreation/Three_levels-2108ns ± 0%116ns ± 0%~(p=1.000 n=1+1)
RuntimeScriptNoop-25.85µs ± 0%4.93µs ± 0%~(p=1.000 n=1+1)
SuperTypeInference/arrays-2299ns ± 0%300ns ± 0%~(p=1.000 n=1+1)
SuperTypeInference/composites-2141ns ± 0%140ns ± 0%~(p=1.000 n=1+1)
SuperTypeInference/integers-2157ns ± 0%157ns ± 0%~(p=1.000 n=1+1)
ValueIsSubtypeOfSemaType-293.5ns ± 0%85.4ns ± 0%~(p=1.000 n=1+1)
 
alloc/opdelta
DecodeBatchEventsCCF-266.1MB ± 0%66.1MB ± 0%~(p=1.000 n=1+1)
DecodeBatchEventsJSON-2244MB ± 0%244MB ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowFees.FeesDeducted-21.39kB ± 0%1.39kB ± 0%~(all equal)
DecodeCCF/FlowFees.TokensWithdrawn-21.20kB ± 0%1.20kB ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-21.47kB ± 0%1.47kB ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-21.48kB ± 0%1.48kB ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.NewWeeklyPayout-21.25kB ± 0%1.25kB ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.RewardsPaid-21.37kB ± 0%1.37kB ± 0%~(all equal)
DecodeCCF/FlowToken.TokensDeposited-21.31kB ± 0%1.31kB ± 0%~(all equal)
DecodeCCF/FlowToken.TokensDeposited_with_nil_receiver-21.30kB ± 0%1.30kB ± 0%~(all equal)
DecodeCCF/FlowToken.TokensMinted-21.20kB ± 0%1.20kB ± 0%~(all equal)
DecodeCCF/FlowToken.TokensWithdrawn-21.32kB ± 0%1.32kB ± 0%~(all equal)
DecodeJSON/FlowFees.FeesDeducted-26.00kB ± 0%6.00kB ± 0%~(all equal)
DecodeJSON/FlowFees.TokensWithdrawn-23.60kB ± 0%3.60kB ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-25.43kB ± 0%5.43kB ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-27.35kB ± 0%7.35kB ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.NewWeeklyPayout-23.64kB ± 0%3.64kB ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.RewardsPaid-24.54kB ± 0%4.54kB ± 0%~(all equal)
DecodeJSON/FlowToken.TokensDeposited-24.88kB ± 0%4.88kB ± 0%~(all equal)
DecodeJSON/FlowToken.TokensDeposited_with_nil_receiver-24.46kB ± 0%4.46kB ± 0%~(all equal)
DecodeJSON/FlowToken.TokensMinted-23.60kB ± 0%3.60kB ± 0%~(all equal)
DecodeJSON/FlowToken.TokensWithdrawn-24.88kB ± 0%4.88kB ± 0%~(all equal)
EncodeBatchEventsCCF-262.4MB ± 0%62.4MB ± 0%~(p=1.000 n=1+1)
EncodeBatchEventsJSON-239.1MB ± 0%39.1MB ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowFees.FeesDeducted-21.22kB ± 0%1.22kB ± 0%~(all equal)
EncodeCCF/FlowFees.TokensWithdrawn-21.17kB ± 0%1.17kB ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-21.44kB ± 0%1.44kB ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-21.41kB ± 0%1.41kB ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.NewWeeklyPayout-21.34kB ± 0%1.34kB ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.RewardsPaid-21.42kB ± 0%1.42kB ± 0%~(all equal)
EncodeCCF/FlowToken.TokensDeposited-21.22kB ± 0%1.22kB ± 0%~(all equal)
EncodeCCF/FlowToken.TokensDeposited_with_nil_receiver-21.20kB ± 0%1.20kB ± 0%~(all equal)
EncodeCCF/FlowToken.TokensMinted-21.17kB ± 0%1.17kB ± 0%~(all equal)
EncodeCCF/FlowToken.TokensWithdrawn-21.22kB ± 0%1.22kB ± 0%~(all equal)
EncodeJSON/FlowFees.FeesDeducted-2864B ± 0%864B ± 0%~(all equal)
EncodeJSON/FlowFees.TokensWithdrawn-2504B ± 0%504B ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-2888B ± 0%888B ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-21.08kB ± 0%1.08kB ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.NewWeeklyPayout-2552B ± 0%552B ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.RewardsPaid-2752B ± 0%752B ± 0%~(all equal)
EncodeJSON/FlowToken.TokensDeposited-2776B ± 0%776B ± 0%~(all equal)
EncodeJSON/FlowToken.TokensDeposited_with_nil_receiver-2640B ± 0%640B ± 0%~(all equal)
EncodeJSON/FlowToken.TokensMinted-2512B ± 0%512B ± 0%~(all equal)
EncodeJSON/FlowToken.TokensWithdrawn-2768B ± 0%768B ± 0%~(all equal)
ExportType/composite_type-2120B ± 0%120B ± 0%~(all equal)
ExportType/simple_type-20.00B 0.00B ~(all equal)
InterpretRecursionFib-21.00MB ± 0%1.00MB ± 0%~(all equal)
NewInterpreter/new_interpreter-2928B ± 0%928B ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-2200B ± 0%200B ± 0%~(all equal)
ParseArray-22.72MB ± 0%2.65MB ± 0%~(p=1.000 n=1+1)
ParseDeploy/byte_array-24.25MB ± 0%4.09MB ± 0%~(p=1.000 n=1+1)
ParseDeploy/decode_hex-2214kB ± 0%214kB ± 0%~(p=1.000 n=1+1)
ParseFungibleToken/With_memory_metering-229.7kB ± 0%29.7kB ± 0%~(p=1.000 n=1+1)
ParseFungibleToken/Without_memory_metering-229.7kB ± 0%29.7kB ± 0%~(all equal)
ParseInfix-21.92kB ± 0%1.91kB ± 0%~(p=1.000 n=1+1)
QualifiedIdentifierCreation/One_level-20.00B 0.00B ~(all equal)
QualifiedIdentifierCreation/Three_levels-264.0B ± 0%64.0B ± 0%~(all equal)
RuntimeScriptNoop-23.21kB ± 0%3.21kB ± 0%~(all equal)
SuperTypeInference/arrays-296.0B ± 0%96.0B ± 0%~(all equal)
SuperTypeInference/composites-20.00B 0.00B ~(all equal)
SuperTypeInference/integers-20.00B 0.00B ~(all equal)
ValueIsSubtypeOfSemaType-248.0B ± 0%48.0B ± 0%~(all equal)
 
allocs/opdelta
DecodeBatchEventsCCF-21.48M ± 0%1.48M ± 0%~(p=1.000 n=1+1)
DecodeBatchEventsJSON-24.70M ± 0%4.70M ± 0%~(p=1.000 n=1+1)
DecodeCCF/FlowFees.FeesDeducted-230.0 ± 0%30.0 ± 0%~(all equal)
DecodeCCF/FlowFees.TokensWithdrawn-226.0 ± 0%26.0 ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-230.0 ± 0%30.0 ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-232.0 ± 0%32.0 ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.NewWeeklyPayout-226.0 ± 0%26.0 ± 0%~(all equal)
DecodeCCF/FlowIDTableStaking.RewardsPaid-229.0 ± 0%29.0 ± 0%~(all equal)
DecodeCCF/FlowToken.TokensDeposited-231.0 ± 0%31.0 ± 0%~(all equal)
DecodeCCF/FlowToken.TokensDeposited_with_nil_receiver-229.0 ± 0%29.0 ± 0%~(all equal)
DecodeCCF/FlowToken.TokensMinted-226.0 ± 0%26.0 ± 0%~(all equal)
DecodeCCF/FlowToken.TokensWithdrawn-231.0 ± 0%31.0 ± 0%~(all equal)
DecodeJSON/FlowFees.FeesDeducted-2126 ± 0%126 ± 0%~(all equal)
DecodeJSON/FlowFees.TokensWithdrawn-271.0 ± 0%71.0 ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-2102 ± 0%102 ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-2159 ± 0%159 ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.NewWeeklyPayout-270.0 ± 0%70.0 ± 0%~(all equal)
DecodeJSON/FlowIDTableStaking.RewardsPaid-287.0 ± 0%87.0 ± 0%~(all equal)
DecodeJSON/FlowToken.TokensDeposited-295.0 ± 0%95.0 ± 0%~(all equal)
DecodeJSON/FlowToken.TokensDeposited_with_nil_receiver-286.0 ± 0%86.0 ± 0%~(all equal)
DecodeJSON/FlowToken.TokensMinted-271.0 ± 0%71.0 ± 0%~(all equal)
DecodeJSON/FlowToken.TokensWithdrawn-295.0 ± 0%95.0 ± 0%~(all equal)
EncodeBatchEventsCCF-2950k ± 0%950k ± 0%~(p=1.000 n=1+1)
EncodeBatchEventsJSON-2853k ± 0%853k ± 0%~(p=1.000 n=1+1)
EncodeCCF/FlowFees.FeesDeducted-219.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowFees.TokensWithdrawn-219.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.DelegatorRewardsPaid-219.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.EpochTotalRewardsPaid-219.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.NewWeeklyPayout-219.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowIDTableStaking.RewardsPaid-219.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowToken.TokensDeposited-220.0 ± 0%20.0 ± 0%~(all equal)
EncodeCCF/FlowToken.TokensDeposited_with_nil_receiver-220.0 ± 0%20.0 ± 0%~(all equal)
EncodeCCF/FlowToken.TokensMinted-219.0 ± 0%19.0 ± 0%~(all equal)
EncodeCCF/FlowToken.TokensWithdrawn-220.0 ± 0%20.0 ± 0%~(all equal)
EncodeJSON/FlowFees.FeesDeducted-219.0 ± 0%19.0 ± 0%~(all equal)
EncodeJSON/FlowFees.TokensWithdrawn-212.0 ± 0%12.0 ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.DelegatorRewardsPaid-216.0 ± 0%16.0 ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.EpochTotalRewardsPaid-225.0 ± 0%25.0 ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.NewWeeklyPayout-212.0 ± 0%12.0 ± 0%~(all equal)
EncodeJSON/FlowIDTableStaking.RewardsPaid-215.0 ± 0%15.0 ± 0%~(all equal)
EncodeJSON/FlowToken.TokensDeposited-219.0 ± 0%19.0 ± 0%~(all equal)
EncodeJSON/FlowToken.TokensDeposited_with_nil_receiver-214.0 ± 0%14.0 ± 0%~(all equal)
EncodeJSON/FlowToken.TokensMinted-213.0 ± 0%13.0 ± 0%~(all equal)
EncodeJSON/FlowToken.TokensWithdrawn-218.0 ± 0%18.0 ± 0%~(all equal)
ExportType/composite_type-23.00 ± 0%3.00 ± 0%~(all equal)
ExportType/simple_type-20.00 0.00 ~(all equal)
InterpretRecursionFib-218.9k ± 0%18.9k ± 0%~(all equal)
NewInterpreter/new_interpreter-215.0 ± 0%15.0 ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-24.00 ± 0%4.00 ± 0%~(all equal)
ParseArray-259.6k ± 0%59.6k ± 0%~(p=1.000 n=1+1)
ParseDeploy/byte_array-289.4k ± 0%89.4k ± 0%~(p=1.000 n=1+1)
ParseDeploy/decode_hex-263.0 ± 0%63.0 ± 0%~(all equal)
ParseFungibleToken/With_memory_metering-2778 ± 0%778 ± 0%~(all equal)
ParseFungibleToken/Without_memory_metering-2778 ± 0%778 ± 0%~(all equal)
ParseInfix-248.0 ± 0%48.0 ± 0%~(all equal)
QualifiedIdentifierCreation/One_level-20.00 0.00 ~(all equal)
QualifiedIdentifierCreation/Three_levels-22.00 ± 0%2.00 ± 0%~(all equal)
RuntimeScriptNoop-251.0 ± 0%51.0 ± 0%~(all equal)
SuperTypeInference/arrays-23.00 ± 0%3.00 ± 0%~(all equal)
SuperTypeInference/composites-20.00 0.00 ~(all equal)
SuperTypeInference/integers-20.00 0.00 ~(all equal)
ValueIsSubtypeOfSemaType-21.00 ± 0%1.00 ± 0%~(all equal)
 

Copy link
Contributor

@darkdrag00nv2 darkdrag00nv2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks

@turbolent
Copy link
Member Author

There's a bit of a problem with the lazy approach:

Cadence currently stores Strings inline or as a separate slab, if too large. This is possible because strings are immutable. However, in this PR we're lazily normalizing the string, effectively "mutating" it (replacing it, with a potentially smaller or larger form).

In the ByteSize and Encode we should ideally use the normalized form, instead of the unnormalized form. If we don't, the calculated byte size might be too long or too short. The normalization allocates, which we need to meter. However, there's no metering available in either of the two functions.

Discussing this in https://discord.com/channels/613813861610684416/1108479699732152503/1151271319548211210

Ideas:

  • Should/can we pass in a metering gauge to ByteSize and Encode? This would need to be done on the atree side unfortunately, and the "threading through" of the metering gauge would be quite tedious.
  • Alternatively, we could switch from lazily normalizing, potentially during encoding, to eagerly normalizing during creation time.

I opened an implementation of the later as #2781

@turbolent
Copy link
Member Author

Closing in favor of #2781

@turbolent turbolent closed this Sep 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Improvement Storage Breaking Change Breaks existing stored data (needs a storage migration)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants