Export proc macros in proc macro crates #2603
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
P-E-P
force-pushed
the
export_proc_macros
branch
3 times, most recently
from
5af27b6 to
448e1b1
Compare
P-E-P
force-pushed
the
export_proc_macros
branch
from
ab58c44 to
8a4ffc7
Compare
P-E-P
force-pushed
the
export_proc_macros
branch
2 times, most recently
from
9ec37f7 to
291ce43
Compare
P-E-P
force-pushed
the
export_proc_macros
branch
9 times, most recently
from
58f1e17 to
c43e95c
Compare
P-E-P
force-pushed
the
export_proc_macros
branch
from
26a5f92 to
2de23fd
Compare
We need to make sure proc macros have the C abi in order to be called by the compiler with dlopen. gcc/rust/ChangeLog: * backend/rust-compile-base.cc (HIRCompileBase::setup_fndecl): Add proc macro handlers dispatch. (handle_proc_macro_common): Add a function for common behavior between all kinds of proc macros. * backend/rust-compile-base.h: Add function prototypes. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
Change the way the ABI is setup on a function to avoid duplicates. ABI is setup by the setup function only now. Add a new attribute to the function "gccrs_proc_macro" in order to differentiate it from another type of function. gcc/rust/ChangeLog: * backend/rust-compile-base.cc (handle_proc_macro_common): Add new attribute "gccrs_proc_macro" to all procedural macro functions. (get_abi): Add a function to retrieve the correct ABI depending on wether the function is a proc macro or not. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
We often need to retrieve the underlying tokentree without modifying it, this getter will help achieve this. gcc/rust/ChangeLog: * ast/rust-ast.h: Add const getter. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
Add one container for each kind of procedural macro mapping. Also add a new structure to gather informations required for derive procedural macros. Add different functions to fill those new containers. gcc/rust/ChangeLog: * backend/rust-compile-context.h (struct CustomDeriveInfo): Add new derive procedural macro metadata information holder for mappings. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
gcc/rust/ChangeLog: * backend/rust-compile-base.cc (HIRCompileBase::setup_abi_options): Reformat uncorrectly formatted comments. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
Collect informations on procedural macros in the compiled crate. For attribute and bang procedural macros we only require the final address as well as the name of the function. Derive procedural macros are a bit different, we collect the fonction's address through it's fndecl tree as well as the trait's name and the multiple attributes. gcc/rust/ChangeLog: * backend/rust-compile-base.cc (HIRCompileBase::setup_fndecl): Make the function non static in order to be able to access the compile context. Also add the whole proc macro infomrmation collection. (get_attributes): Add a function to retrieve the different attributes from a derive procedural macro definition attribute. (get_trait_name): Add a function to retrieve the trait name from a derive procedural macro definition attribute. * backend/rust-compile-base.h: Add function prototypes. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
Add three different getters, one for each proc macro type. gcc/rust/ChangeLog: * backend/rust-compile-context.h: Add getters. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
Add some utility function to build proc macro entrypoint related types. Those functions will help generate all required metadata in order for proc macros to be expanded properly. gcc/rust/ChangeLog: * backend/rust-compile.cc (build_attribute_array): Add a function to build the attribute array type. (build_derive_proc_macro): Add a function to build the derive proc macro type. (build_bang_proc_macro): Add a function to build the bang proc macro type. (build_attribute_proc_macro): Add a function to build the attribute proc macro type. (build_proc_macro): Add a function to build the proc macro tagged union type. (build_proc_macro_buffer): Add a function to build the proc macro buffer type. (build_entrypoint): Add a function to build the proc macro entrypoint type. * backend/rust-compile.h: Add function prototype. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
Change proc macro entrypoint from a fixed constant declaration to a proper generation from the stable crate id. Although the stable crate id is not in use yet, the mechanism to handle it is. gcc/rust/ChangeLog: * expand/rust-proc-macro.cc (CustomDeriveProcMacro::CustomDeriveProcMacro): Remove constant string declaration. (load_macros_array): Add call to the new generation function. (generate_proc_macro_decls_symbol): Add a new function to generate the entrypoint symbol name from the stable crate id. (PROC_MACRO_DECLS_FMT_ARGS): New macro to keep formats arguments in sync between each call. * expand/rust-proc-macro.h (generate_proc_macro_decls_symbol): Add function prototype. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
We'll need this value in the final binary, it should therefore be kept explicit. ChangeLog: * libgrust/libproc_macro_internal/proc_macro.h (enum ProcmacroTag): Add explicit value for proc macro tag enum. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
Export a new symbol containing the proc macros. gcc/rust/ChangeLog: * backend/rust-compile-base.h: Make static function address_expression public. * backend/rust-compile.cc (CompileCrate::add_proc_macro_symbols): Add new global variable in export function. (build_bang_proc_macro): Add a function to build the bang proc macro structure type. (build_proc_macro): Add a function to build the proc macro structure type. (build_proc_macro_payload): Add a function to build the proc macro union used in proc macro structures. (init_derive_proc_macro): Add a function to initialize custom derive proc macros. (init_attribute_proc_macro): Add a function to initialize attribute proc macros. (init_bang_proc_macro): Add a function to initialize bang proc macros. (init_proc_macro): Add a function to initialize proc macro structures. (initialize_proc_macro_array): Add a function to initialize the proc macro buffer array. (CompileCrate::add_proc_macro_symbols): Add call to the new functions to correctly initialize proc macros as well as their entrypoint. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
P-E-P
force-pushed
the
export_proc_macros
branch
from
2de23fd to
4feb3f2
Compare
Half of the functions introduced recently had a similar goal while the other half had a similar goal too. Introducing some namespace to separate those will keep the code cleaner and avoid confusion. gcc/rust/ChangeLog: * backend/rust-compile.cc (build_attribute_array): Renamed from... (attribute_array): ...to attribute array. (build_derive_proc_macro): Likewise from... (derive_proc_macro): ... to derive_proc_macro. (build_bang_proc_macro): Likewise from... (bang_proc_macro): ...to bang_proc_macro. (build_attribute_proc_macro): Likewise from... (attribute_proc_macro): ... to attribute_proc_macro. (build_proc_macro_payload): Likewise from... (proc_macro_payload): to proc_macro_payload. (build_proc_macro): Likewise from... (proc_macro): ...to proc_macro. (build_proc_macro_buffer): Likewise from... (proc_macro_buffer): ... to proc_macro_buffer. (build_entrypoint): Likewise from... (entrypoint): ...to entrypoint. (init_derive_proc_macro): Renamed to it's shorter counterpart. (init_attribute_proc_macro): Likewise. (init_bang_proc_macro): Likewise. (init_proc_macro): Likewise. (initialize_proc_macro_array): Likewise. (proc_macro_array): Likewise. (CompileCrate::add_proc_macro_symbols): Update function calls. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
P-E-P
force-pushed
the
export_proc_macros
branch
from
4feb3f2 to
4edfeba
Compare
The compiler cannot infer the array length from the type, we should therefore hand it the information. The proc macro buffer missed that information. gcc/rust/ChangeLog: * backend/rust-compile.cc (proc_macro_buffer): Update type builder with array length information. (proc_macro_array): Update type initializer with array length information. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
philberty
requested changes
Oct 16, 2023
gcc/rust/backend/rust-compile.cc
Outdated
| // This namespace describes how to build and initialize those metadata | ||
| // structures. Those structure should be kept in sync with the structures in | ||
| // libproc_macro_internal/proc_macro.h describing how they should be read. | ||
| namespace { |
There was a problem hiding this comment.
I didn't want to export all those functions outside of this TU.
gcc/rust/backend/rust-compile.cc
Outdated
| namespace { | ||
|
|
||
| // Namespace containing all functions to build the different types. | ||
| namespace build { |
There was a problem hiding this comment.
Not sure i like the namespace build i feel like these functions could be in a seperate file like rust-compile-proc-macro.cc
There was a problem hiding this comment.
Yeah, I'm not entirely convinced myself. Do you have anything to suggest ? I'd like to keep build and initialization separated. I will move the whole anon namespace and the add_proc_macro_symbol in another file.
The code to build the required procedural macro symbols is rather long and could be placed in it's own file. gcc/rust/ChangeLog: * Make-lang.in: Add gcc/rust/backend/rust-compile-proc-macro.cc to the list of file to compile. * backend/rust-compile.cc (attribute_array): Move to rust-compile-proc-macro.cc (derive_proc_macro): Likewise. (bang_proc_macro): Likewise. (attribute_proc_macro): Likewise. (proc_macro_payload): Likewise. (proc_macro): Likewise. (proc_macro_buffer): Likewise. (entrypoint): Likewise. (proc_macro_array): Likewise. (CompileCrate::add_proc_macro_symbols): Likewise. * backend/rust-compile-proc-macro.cc: New file. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
P-E-P
force-pushed
the
export_proc_macros
branch
from
1b70add to
3e4e6a1
Compare
Done! |
CohenArthur
approved these changes
Oct 18, 2023
Multiple references to procedural macro token trees were left as magic number in the code. This commit introduces some constexpr for those along some explanation for the selected value. gcc/rust/ChangeLog: * backend/rust-compile-base.cc (get_attributes): Add documentation for indices 3 and 4. (get_trait_name): Add documentation for index 1. Signed-off-by: Pierre-Emmanuel Patry <[email protected]>
philberty
reviewed
Oct 31, 2023
| tree entrypoint_type = build::entrypoint (pm_buffer_type); | ||
|
|
||
| std::string decl_symbol_name = generate_proc_macro_decls_symbol ( | ||
| 0 /* FIXME: Change to stable crate id */); |
There was a problem hiding this comment.
You should be able to get mappings->get_current_crate for that.
There was a problem hiding this comment.
Are your sure we're talking about the same crate id ? IIRC the stable crate id is made from the hash of the crate's metadata. I opened #2696 for this a while ago, but since you wanted to rework the metadata format I left it as is while we wait for this rework.
philberty
approved these changes
Oct 31, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In order to fully support procedural macros we need to be able to compile those down to their binary representation (shared library). This means we should :
Note that #2461 should be handled separately in another PR.