Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add notebook for VAIS automatic URL recrawl #161

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat: Add notebook for VAIS automatic URL recrawl #161

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f450fce
feat: Add notebook for VAIS automatic URL recrawl
hossein-mansour Jan 4, 2025
40dd793
Update README.md
hossein-mansour Jan 4, 2025
ec441c8
Update mkdocs.yml
hossein-mansour Jan 4, 2025
4cf5074
Update README.md
hossein-mansour Jan 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/mkdocs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
- content.footnote.tooltips
- content.tabs.link
- content.tooltips
- header.autohide

Check warning on line 28 in docs/mkdocs.yml

View workflow job for this annotation

GitHub Actions / Check Spelling 

`autohide` is not a recognized word. (unrecognized-spelling)
- navigation.expand
- navigation.footer
- navigation.indexes
Expand Down Expand Up @@ -124,22 +124,22 @@
- md_in_html
- toc:
permalink: true
- pymdownx.arithmatex:

Check warning on line 127 in docs/mkdocs.yml

View workflow job for this annotation

GitHub Actions / Check Spelling 

`pymdownx` is not a recognized word. (unrecognized-spelling)
generic: true
- pymdownx.betterem:

Check warning on line 129 in docs/mkdocs.yml

View workflow job for this annotation

GitHub Actions / Check Spelling 

`betterem` is not a recognized word. (unrecognized-spelling)
smart_enable: all
- pymdownx.caret
- pymdownx.details
- pymdownx.highlight:
anchor_linenums: true

Check warning on line 134 in docs/mkdocs.yml

View workflow job for this annotation

GitHub Actions / Check Spelling 

`linenums` is not a recognized word. (unrecognized-spelling)
line_spans: __span
pygments_lang_class: true
- pymdownx.inlinehilite
- pymdownx.keys
- pymdownx.magiclink:

Check warning on line 139 in docs/mkdocs.yml

View workflow job for this annotation

GitHub Actions / Check Spelling 

`magiclink` is not a recognized word. (unrecognized-spelling)
normalize_issue_symbols: true
repo_url_shorthand: true
user: squidfunk

Check warning on line 142 in docs/mkdocs.yml

View workflow job for this annotation

GitHub Actions / Check Spelling 

`squidfunk` is not a recognized word. (unrecognized-spelling)
repo: mkdocs-material
- pymdownx.mark
- pymdownx.smartsymbols
Expand All @@ -161,7 +161,7 @@
custom_checkbox: true
- pymdownx.tilde
- pymdownx.emoji:
emoji_index: !!python/name:materialx.emoji.twemoji

Check warning on line 164 in docs/mkdocs.yml

View workflow job for this annotation

GitHub Actions / Check Spelling 

`materialx` is not a recognized word. (unrecognized-spelling)
emoji_generator: !!python/name:materialx.emoji.to_svg

# Plugins
Expand All @@ -180,7 +180,7 @@

nav:
- Home: index.md
- GenAI on Vertex AI:

Check warning on line 183 in docs/mkdocs.yml

View workflow job for this annotation

GitHub Actions / Check Spelling 

`GenAI` matches a line_forbidden.patterns entry: `\b[gG]enAI\b`. (forbidden-pattern)
- Overview: genai-on-vertex-ai/README.md
- Foundation Models:
- Gemini 2.0:
Expand All @@ -207,7 +207,7 @@
- Advanced Prompting Training:
- Overview: genai-on-vertex-ai/advanced_prompting_training/README.md
- CoT & ReAct: genai-on-vertex-ai/advanced_prompting_training/cot_react.ipynb
- Vertex Foundation Tuning:

Check warning on line 210 in docs/mkdocs.yml

View workflow job for this annotation

GitHub Actions / Check Spelling 

`Vertex` matches a line_forbidden.patterns entry: `\b(?!Vertex AI\b)(?!.*[\(\)\{\},])(?<!import\s)(?<!\.)(?<!,\s)Vertex\s?[Aa]?[Ii]?\b`. (forbidden-pattern)
- Overview: genai-on-vertex-ai/vertex_foundation_tuning/README.md
- Model Tuning: genai-on-vertex-ai/vertex_foundation_tuning/vertexai-model-tuning.ipynb
- Evaluation:
Expand Down Expand Up @@ -249,6 +249,7 @@
- Inline Ingestion of Documents: genai-on-vertex-ai/vertex_ai_search/inline_ingestion_of_documents.ipynb
- Parsing and Chunking: genai-on-vertex-ai/vertex_ai_search/parsing_and_chunking_with_BYO.ipynb
- Query Level Boosting: genai-on-vertex-ai/vertex_ai_search/query_level_boosting_filtering_and_facets.ipynb
- Event-based Triggering of Manual Recrawl for Vertex AI Search Advanced Website Datastores: genai-on-vertex-ai/vertex_ai_search/manual_recrawl_urls_with_trigger.ipynb
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

For consistency with other entries, consider capitalizing "Event-based" and "Trigger". Also, consider rephrasing to be more concise, perhaps "Triggering Manual Recrawls for Vertex AI Search (Advanced Website Datastores)"

Suggested change
- Event-based Triggering of Manual Recrawl for Vertex AI Search Advanced Website Datastores: genai-on-vertex-ai/vertex_ai_search/manual_recrawl_urls_with_trigger.ipynb
- Triggering Manual Recrawls for Vertex AI Search (Advanced Website Datastores): genai-on-vertex-ai/vertex_ai_search/manual_recrawl_urls_with_trigger.ipynb

- DIY RAG:
- Overview: genai-on-vertex-ai/retrieval_augmented_generation/README.md
- Building Grounded RAG App: genai-on-vertex-ai/retrieval_augmented_generation/diy_rag_with_vertexai_apis/build_grounded_rag_app_with_vertex.ipynb
Expand Down
1 change: 1 addition & 0 deletions genai-on-vertex-ai/vertex_ai_search/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,4 @@ We will try to use REST APIs which will hopefully make the codes easier to under
3. [Defining custom attributes based on URL patterns in Vertex AI Search Website Datastores](./custom_attributes_by_url_pattern.ipynb)
4. [Query-Level Boosting, Filtering, and Facets for Vertex AI Search Website Datastores](./query_level_boosting_filtering_and_facets.ipynb)
5. [Inline Ingestion of Documents into Vertex AI Search](./inline_ingestion_of_documents.ipynb)
6. [Event-based Triggering of Manual Recrawl for Vertex AI Search Advanced Website Datastores](./manual_recrawl_urls_with_trigger.ipynb)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

For consistency with other entries, consider capitalizing "Event-based" and "Trigger". Also, consider rephrasing to be more concise, perhaps "Triggering Manual Recrawls for Vertex AI Search (Advanced Website Datastores)"

Suggested change
6. [Event-based Triggering of Manual Recrawl for Vertex AI Search Advanced Website Datastores](./manual_recrawl_urls_with_trigger.ipynb)
6. [Triggering Manual Recrawls for Vertex AI Search (Advanced Website Datastores)](./manual_recrawl_urls_with_trigger.ipynb)

405 changes: 405 additions & 0 deletions genai-on-vertex-ai/vertex_ai_search/manual_recrawl_urls_with_trigger.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

Loading