Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: shopify scraping #2553

Merged
merged 7 commits into from
Oct 10, 2024
Merged

feat: shopify scraping #2553

merged 7 commits into from
Oct 10, 2024

Conversation

drew-harris
Copy link
Contributor

@drew-harris drew-harris commented Oct 9, 2024

Add Option for scraping any shopify store

  • refactor openapi_options to a more generic scraping_options enum
  • add parameter to ScrapingOptions::Shopify for group_variants. This option:
    • ingests all variants with tracking_id being variant.id
    • places all variants in a group based on product.id
    • applies fulltext and semantic boosts based on variant_title
  • redoc isn't properly formatting the ScrapeOptions enum, but localhost:8090/swagger-ui is want to merge to see if mintlify will pick it up. If not will make an issue
  • resolve ts-errors

I need help making the ts-errors go away. 🙏

Will rebase and squash commits after everything is confirmed to work.

Tested on:

@cdxker cdxker marked this pull request as ready for review October 10, 2024 04:12
@cdxker cdxker linked an issue Oct 10, 2024 that may be closed by this pull request
3 tasks
@cdxker cdxker merged commit 8aa928e into main Oct 10, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feature: add shopify scraping to crawling options setup
2 participants