perf: use str instead of Cow to reduce Token size

[sssooonnnggg]

"Cow" has a relatively large layout size than &str, which can impact performance.

• size_of::<Option<Cow>>() == 32
• size_of::<Option<&str>>() == 16

It seems that using '&str' to represent the tag is sufficient.
For my USD parser, the parsing time for a 50MB USD file decreased from 800 milliseconds to 600 milliseconds.
https://github.com/sssooonnnggg/rusd

[tomtau]

pest_vm fails to compile: https://github.com/pest-parser/pest/actions/runs/7396775314/job/20127215025?pr=958#step:4:238 I'm not sure if a string slice can be used there

[tomtau]

Perhaps this optimization (if not possible to make it work in a non-semver-breaking way and work within pest_vm) can be feature-guarded under #[cfg(not(feature = "std"))] (the original code would be under #[cfg(feature = "std")])?

[NoahTheDuke]

Please don't use AI tools to make PRs to Pest.

[tomtau]

Please don't use AI tools to make PRs to Pest.

@NoahTheDuke I think @sssooonnnggg didn't use AI tool to make this PR, those coderabbitai comments were meant to "code review" this PR, because I enabled that bot some time ago to see what's like (and it seems it's not particularly useful for small PRs like this)... would you prefer to disable it? One other option is to limit it to review only PRs if we put some special labels on them

[NoahTheDuke]

Oh dang, my apologies. I thought that was brought in by the OP. If you enabled it, then I'll leave it be in the future. (I'm not a fan of such AI tools but that's just personal preference.) I only ask that folks don't come to use with their own AI tools.

I think the label idea is good, that will make it more deliberate.

[tomtau]

No problem. We'll see how it goes, so I put it that AI code review comments will only be added if "pr" label is added.

[coderabbitai]

Important

Auto Review Skipped

Auto reviews are limited to the following labels: pr. Please add one of these labels to enable auto reviews.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository.

To trigger a single review, invoke the @coderabbitai review command.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

---

Tips

Chat with CodeRabbit Bot (@coderabbitai)

• You can directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>.
  • Generate unit-tests for this file.
• You can tag CodeRabbit on specific lines of code or entire files in the PR by tagging @coderabbitai in a comment. Examples:
  • @coderabbitai generate unit tests for this file.
  • @coderabbitai modularize this function.
• You can tag @coderabbitai in a PR comment and ask questions about the PR and the codebase. Examples:
  • @coderabbitai generate interesting stats about this repository from git and render them as a table.
  • @coderabbitai show all the console.log statements in this repository.
  • @coderabbitai read src/utils.ts and generate unit tests.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid.
  • @coderabbitai read the files in the src/scheduler package and generate README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks.

CodeRabbit Commands (invoked as PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• The JSON schema for the configuration file is available here.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

[sssooonnnggg]

pest_vm fails to compile: https://github.com/pest-parser/pest/actions/runs/7396775314/job/20127215025?pr=958#step:4:238 I'm not sure if a string slice can be used there

Thank you for the review,
Fixed the compilation error in the vm/src/lib.rs. I think we can use a string slice here, as it's used similarly in lib.rs:188 with match_string.

[sssooonnnggg]

It's an API breaking change, but from the user's perspective, this API is usually not directly used by real users. In reality, it's used within the code generated by a derived macro, so I don't think it's a major issue.

According to the documentation, the APIs actually used by the users are as_node_tag, find_first_tagged, and find_tagged. These interfaces have not changed.

Additionally, this optimization significantly improves execution speed and reduces memory usage. I believe it's unrelated to the std feature.

[tomtau]

Thank you for the review,
Fixed the compilation error in the vm/src/lib.rs. I think we can use a string slice here, as it's used similarly in lib.rs:188 with match_string.

It still doesn't compile, unfortunately:

https://github.com/pest-parser/pest/actions/runs/7405863941/job/20155605432?pr=958#step:4:226

[sssooonnnggg]

Thank you for the review,
Fixed the compilation error in the vm/src/lib.rs. I think we can use a string slice here, as it's used similarly in lib.rs:188 with match_string.

It still doesn't compile, unfortunately:

https://github.com/pest-parser/pest/actions/runs/7405863941/job/20155605432?pr=958#step:4:226

My fault :), I forgot to check all features, now it should pass all checks by cargo check --all --features pretty-print,const_prec_climber,memchr,grammar-extras --all-targets command, pelease review again

[tomtau]

My fault :), I forgot to check all features, now it should pass all checks by cargo check --all --features pretty-print,const_prec_climber,memchr,grammar-extras --all-targets command, pelease review again

Thanks for fixing it!

that tag_node function signature change and those lifetime parameter changes in pest_vm may be technically semver-breaking, but hopefully it's benign.

It's an API breaking change, but from the user's perspective, this API is usually not directly used by real users. In reality, it's used within the code generated by a derived macro, so I don't think it's a major issue.

I'm not worried about the end user here, but more worried about the weird situations where cargo version resolution picks up different combinations of pest 2.X crates, such as older pest_generator 2.7 and the latest pest 2.7, which happened in the past and some users had compilation issues because of that.

Anyway, I manually ran cargo-semver-checks (because CI seems broken at the moment) between 2.5.0 and this PR, and between 2.7.0 and this PR... and it didn't report any new issues from this PR, so I assume it should be ok. I'll go ahead and merge it then; once the new version is out and if any users report issues with it, we could potentially yank it and try some feature-guarding workaround, but hopefully no need for it.