Start reusing rustc's format_args parser
#2822
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CohenArthur
commented
Jan 30, 2024
| } | ||
|
|
||
| auto format_string = fmt_expr->as_string (); | ||
| auto pieces = Fmt::Pieces::collect (format_string); |
There was a problem hiding this comment.
we need to figure out exactly what to give here, e.g if we do
format_args!("heyo {bit}", bit = 15);should we parse heyo {bit} or \"heyo {bit}\", bit = 15. I believe it is the second one but that's not what we do at the moment
There was a problem hiding this comment.
for this I think we can reuse the tokenstream from @P-E-P and turn the entire invocation tokenstream into a string which we'll then parse using the lib
There was a problem hiding this comment.
this is done per cfaf9fa but I think we do get garbage values when dropping the PieceSlice - needs to investigate
There was a problem hiding this comment.
now fixed in 71680f5 by keeping ownership of the string to parse. this is ready for merging I think?
CohenArthur
changed the title
format_args parser
CohenArthur
force-pushed
the
reuse-rustc-format-parser
branch
3 times, most recently
from
e747d1a to
71680f5
Compare
|
I'll open an issue to cleanup the implementation once it is merged, we can write better Rust in there and separate into better modules with better names. |
CohenArthur
force-pushed
the
reuse-rustc-format-parser
branch
4 times, most recently
from
344e7af to
99e1bd3
Compare
CohenArthur
modified the milestones:
AST Pipeline for libcore 1.49 Complete,
`format_args!()` support
CohenArthur
force-pushed
the
reuse-rustc-format-parser
branch
from
99e1bd3 to
3693bf0
Compare
This commit adds a base class for parsing the various constructs of a Rust format string, according to the grammar in the reference: https://doc.rust-lang.org/std/fmt/index.html#syntax gcc/rust/ChangeLog: * Make-lang.in: Compile rust-fmt object * ast/rust-fmt.cc: New file. * ast/rust-fmt.h: New file.
CohenArthur
force-pushed
the
reuse-rustc-format-parser
branch
4 times, most recently
from
65efcdd to
7d2d639
Compare
Compile libformat_parser and link to it. gcc/rust/ChangeLog: * Make-lang.in: Compile libformat_parser. * ast/rust-fmt.cc: New FFI definitions. * ast/rust-fmt.h: Likewise. * expand/rust-macro-builtins.cc (MacroBuiltin::format_args_handler): Call into libformat_parser. * expand/rust-macro-builtins.h: Define format_args!() handler proper. libgrust/ChangeLog: * libformat_parser/Cargo.lock: New file. * libformat_parser/Cargo.toml: New file. * libformat_parser/generic_format_parser/Cargo.toml: New file. * libformat_parser/generic_format_parser/src/lib.rs: New file. * libformat_parser/src/bin.rs: New file. * libformat_parser/src/lib.rs: New file.
libgrust/ChangeLog: * libformat_parser/generic_format_parser/src/lib.rs: Add generic library. * libformat_parser/src/lib.rs: Add base for FFI interface.
libgrust/ChangeLog: * libformat_parser/cbindgen.toml: New file. * libformat_parser/libformat-parser.h: New file. gcc/rust/ChangeLog: * ast/rust-fmt.h: Add remaining FFI types.
ChangeLog: * .gitignore: Add libgrust target folders to the ignore list.
gcc/rust/ChangeLog: * ast/rust-fmt.cc (Pieces::collect): Use new Pieces API. * ast/rust-fmt.h: Update interface with new FFI bindings. libgrust/ChangeLog: * libformat_parser/src/lib.rs: Add IntoFFI trait. * libformat_parser/libformat-parser.h: Removed.
gcc/rust/ChangeLog: * ast/rust-fmt.cc (Pieces::~Pieces): Call libformat_parser's release function in destructor. * ast/rust-fmt.h (struct PieceSlice): Add capacity. (destroy_pieces): New. (struct Pieces): Add destructor. libgrust/ChangeLog: * libformat_parser/src/lib.rs: Leak Boxes properly for C++ to see them, add memory release function.
gcc/rust/ChangeLog: * expand/rust-macro-builtins.cc (MacroBuiltin::format_args_handler): Construct string to parser properly.
gcc/rust/ChangeLog: * expand/rust-macro-builtins.cc (MacroBuiltin::format_args_handler): Transform entire invocation token stream into string for the parser.
gcc/rust/ChangeLog: * ast/rust-fmt.cc (Pieces::collect): Fix signature to take ownership of the given string. * ast/rust-fmt.h (struct Pieces): Store parsed string in the struct. libgrust/ChangeLog: * libformat_parser/src/lib.rs: Add debug prompt.
libgrust/ChangeLog: * libformat_parser/generic_format_parser/src/lib.rs: Remove unused deprecated attribute and unused import. * libformat_parser/src/lib.rs: Remove unused import.
Workaround for Ubuntu 18.04, since we still use it for the GCC 4.8 CI. The default Rust package is 1.65 (and unlikely to change I assume?), but the generic format parser library uses `is_some_and` which was introduced in 1.70. So this is a simple reimplementation, directly taken from the standard library sources. libgrust/ChangeLog: * libformat_parser/generic_format_parser/src/lib.rs: Add IsSomeAnd<T> trait, impl it for Option<T>.
ChangeLog: * .github/workflows/ccpp.yml: Install cargo for GCC 4.8 job.
This was referenced Feb 27, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request adds a base for reusing the rustc_format_parser crate to parse the format strings in
format_args!()invocations.There are still a lot of things missing, and the build system stuff is probably wrong. The commit history needs to be fixed as well.
Fixes #2832
This PR depends on
#2802#2853 for the proper libgrust Changelog file