-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: port over initial python schema extraction PoC #3332
Conversation
696bf74
to
88a0118
Compare
python-runtime/ftl/pyproject.toml
Outdated
@@ -9,6 +9,7 @@ authors = [ | |||
requires-python = ">=3.12" | |||
dependencies = [ | |||
"protobuf>=5.28.3", | |||
"google>=3.0.0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't seem necessary? No source links on the project page either.
Python bindings to the Google search engine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nope good catch. artifact from poc. removed it
print(f"Extracted Decl:\n{decl}") | ||
|
||
|
||
def analyze_file(global_ctx: GlobalExtractionContext, file_path, analyzer_batch): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing annotations here and elsewhere?
I'd like it if all the Python code were fully typed, and linted for typing too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yep agreed! just configured the linter to require type annotations for everything: e8b9bf2
will go through and annotate everything now
|
||
__all__ = [ | ||
"extract_type", | ||
"extract_slice", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do these other functions need to be exported?
return None | ||
|
||
|
||
def extract_slice( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe these should be _private?
def extract_class_type( | ||
local_ctx: LocalExtractionContext, type_hint: Type[Any] | ||
) -> Optional[schemapb.Type]: | ||
ref = schemapb.Ref(name=type_hint.__name__, module=type_hint.__module__) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure what state this is in, so maybe this is a bit premature, but using __module__
isn't going to be sufficient IIRC?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
aye chatted with @worstell , probably not going to be sufficient. will leave this to tackle in a subsequent PR
… into moe/py-schema-extraction
Co-authored-by: Alec Thomas <[email protected]>
… into moe/py-schema-extraction
Summary
This PR ports python schema extraction PoC into this repo
Warning
Still mostly unusable but this lays in the vast majority of a foundation.
Changes
verb
decorator. switched to using a pattern that doesn't require dynamic dangling mutable properties on functions. Instead encapsulates decorated verb functions in a class and extracts type information useful for runtime grpc server and schema extractionVerbExtractor
to useisinstance(func, Verb)
instead ofgetattr(func, "_is_ftl_verb", False)
.schema_extractor
(note: this will change in the next PR to a standalone executable. temporarily in acli
dir)Build
methodprops to @worstell for doing the actual schema extraction work! will co-author the merge commit
Usage
Note
From the
python-runtime/ftl
dir