Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add parse with layout and extract resume key value #60

Merged
merged 9 commits into from
Nov 7, 2024
Merged

feat: add parse with layout and extract resume key value #60

merged 9 commits into from
Nov 7, 2024

Conversation

Sdddell
Copy link
Collaborator

@Sdddell Sdddell commented Oct 31, 2024 •
edited
Loading

Add SDK for parse with layout and extract resume key value.

  • parse with layout
    • async: async_parse_with_layout
    • test
      • single page PDF with images and tables
      • single image with images and tables
      • multi page PDF with images
  • extract resume key value
    • sync: extract_resume_key_value
    • async: async_extract_resume_key_value
    • test
      • multi page resume PDF
      • single resume image

lingjiekong
lingjiekong reviewed Oct 31, 2024
View reviewed changes
any_parser/any_parser.py Outdated
@@ -221,6 +222,8 @@ def async_extract(
process_type = ProcessType.FILE
elif model == ModelType.PRO:
process_type = ProcessType.FILE_REFINED_QUICK
elif model == ModelType.ADVANCED:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: let's call this parse_with_layout in the ModelType.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack

any_parser/utils.py Outdated
@@ -8,6 +8,7 @@
class ModelType(Enum):
BASE = "base"
PRO = "pro"
ADVANCED = "advanced"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: let's call this parse with layout.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack

examples/async_parse_with_layout.ipynb Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: it looks like the image plot cannot be previewed in the ipynb?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it seems the vscode can render it, but ipynb can't.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I think it might be the timeout issue? I don't know how ipynb caches the results.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

qq: this is async API and what is the timeout issue that you are talking about?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it looks fine locally, but Github fails to render it properly.

lingjiekong
lingjiekong reviewed Nov 4, 2024
View reviewed changes
any_parser/any_parser.py Outdated
Comment on lines 286 to 287
elif model == ModelType.PARSE_WITH_LAYOUT:
process_type = ProcessType.PARSE_WITH_LAYOUT
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's not nest the all different parse into the same async_extract. Let's add a new SDK called async_parse_with_layout.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack

any_parser/utils.py Outdated
@@ -8,6 +8,7 @@
class ModelType(Enum):
BASE = "base"
PRO = "pro"
PARSE_WITH_LAYOUT = "parse_with_layout"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

like above. let's have a separated parse_with_layout instead of nesting parse_with_layout in current logic.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack

@lingjiekong
Copy link
Member

Add SDK for advanced full context parsing.

  • ProcessType: file_advanced
  • ModelType: file_advanced
  • API type: async only (reuse async_extract)

Let's update the description to follow, feat, test, and todo.

Also, this PR contains both adding extract_resume_key_value for both sync and async and the async_extract_with_layout, so let's clearly mention it in the PR title.

@lingjiekong
Copy link
Member

Also, is the extract_resume a rebase on top of #57?

@lingjiekong lingjiekong mentioned this pull request Nov 5, 2024
Closed
lingjiekong
lingjiekong approved these changes Nov 5, 2024
View reviewed changes
Copy link
Member

@lingjiekong lingjiekong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with minor comment

  1. check notebook display
  2. rename the PR to reflect the change on parse with layout and extract resume key value.

@Sdddell Sdddell changed the title feat: add advanced full context and example feat: add parse with layout and extract resume key value Nov 5, 2024
@Sdddell Sdddell merged commit 1590de6 into main Nov 7, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@CambioML CambioML CambioML approved these changes

@lingjiekong lingjiekong lingjiekong approved these changes

@goldmermaid goldmermaid Awaiting requested review from goldmermaid goldmermaid is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Sdddell @lingjiekong @CambioML