feat: Added qdrant support for BaseEngine and added MediaWiki RAG! #60
Conversation
|
Warning Rate Limit Exceeded@amindadgar has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 14 minutes and 1 seconds before requesting another review. How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. WalkthroughThis update enhances the project by incorporating QDrant for vector storage and retrieval, upgrading the Changes
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (3)
- docker-compose.test.yml (3 hunks)
- requirements.txt (1 hunks)
- utils/query_engine/base_engine.py (3 hunks)
Files skipped from review due to trivial changes (1)
- requirements.txt
Additional comments not posted (7)
utils/query_engine/base_engine.py (4)
3-5: Imports look good and are used correctly in the code.Also applies to: 7-7
27-27: The change fromself.dbnametoself.collection_nameis consistent and correctly implemented.
30-37: The changes in thepreparemethod to usevector_store_indexandVectorIndexRetrieverare consistent and correctly implemented.
58-66: The changes in the_setup_vector_store_indexmethod to usecollection_nameandQDrantVectorAccessare consistent and correctly implemented.docker-compose.test.yml (3)
37-39: The addition of environment variablesQDRANT_HOST,QDRANT_PORT, andQDRANT_API_KEYis correctly implemented and necessary for the new functionality.
49-50: The addition ofqdrant-healthcheckto thedepends_onsection is correctly implemented and necessary for the new functionality.
95-116: The new services forqdrantandqdrant-healthcheckare correctly configured and necessary for the new functionality.
amindadgar
changed the title
There was a problem hiding this comment.
Actionable comments posted: 1
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (4)
- subquery.py (4 hunks)
- tests/unit/test_mediawiki_query_engine.py (1 hunks)
- utils/query_engine/init.py (1 hunks)
- utils/query_engine/media_wiki.py (1 hunks)
Files skipped from review due to trivial changes (1)
- utils/query_engine/init.py
Additional comments not posted (5)
utils/query_engine/media_wiki.py (1)
1-7: The__init__method looks good and follows standard practices.tests/unit/test_mediawiki_query_engine.py (2)
8-10: ThesetUpmethod looks good and follows standard practices.
12-15: Verify the usage ofCustomVectorStoreRetriever.The test uses
CustomVectorStoreRetriever, but the changes inbase_engine.pysuggest a switch toVectorIndexRetriever. Ensure the test is consistent with the updated implementation.subquery.py (2)
29-29: The parameters look good and follow standard practices.
Line range hint
162-194: The query execution looks good and follows standard practices.
| if mediaWiki: | ||
| mediawiki_query_engine = MediaWikiQueryEngine( | ||
| community_id=community_id | ||
| ).prepare() | ||
| tool_metadata = ToolMetadata( | ||
| name="WikiPedia", | ||
| description="Hosts articles about any information on internet", | ||
| ) | ||
| query_engine_tools.append( | ||
| QueryEngineTool( | ||
| query_engine=mediawiki_query_engine, | ||
| metadata=tool_metadata, | ||
| ) | ||
| ) |
There was a problem hiding this comment.
Fix the typo in the tool metadata name.
- name="WikiPedia",
+ name="MediaWiki",Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| if mediaWiki: | |
| mediawiki_query_engine = MediaWikiQueryEngine( | |
| community_id=community_id | |
| ).prepare() | |
| tool_metadata = ToolMetadata( | |
| name="WikiPedia", | |
| description="Hosts articles about any information on internet", | |
| ) | |
| query_engine_tools.append( | |
| QueryEngineTool( | |
| query_engine=mediawiki_query_engine, | |
| metadata=tool_metadata, | |
| ) | |
| ) | |
| if mediaWiki: | |
| mediawiki_query_engine = MediaWikiQueryEngine( | |
| community_id=community_id | |
| ).prepare() | |
| tool_metadata = ToolMetadata( | |
| name="MediaWiki", | |
| description="Hosts articles about any information on internet", | |
| ) | |
| query_engine_tools.append( | |
| QueryEngineTool( | |
| query_engine=mediawiki_query_engine, | |
| metadata=tool_metadata, | |
| ) | |
| ) |
There was a problem hiding this comment.
Actionable comments posted: 0
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- docker-compose.test.yml (3 hunks)
Files skipped from review as they are similar to previous changes (1)
- docker-compose.test.yml
We were using the default `VectorIndexRetriever` for the new pipelines and it was wrong to assert those with CustomVectorStoreRetriever.
There was a problem hiding this comment.
Actionable comments posted: 0
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (4)
- tests/unit/test_gdrive_query_engine.py (2 hunks)
- tests/unit/test_github_query_engine.py (2 hunks)
- tests/unit/test_mediawiki_query_engine.py (1 hunks)
- tests/unit/test_notion_query_engine.py (2 hunks)
Files skipped from review as they are similar to previous changes (1)
- tests/unit/test_mediawiki_query_engine.py
Additional comments not posted (6)
tests/unit/test_gdrive_query_engine.py (2)
3-3: Updated import statement aligns with the newVectorIndexRetrieverusage.
15-15: Updated assertion correctly checks for the instance ofVectorIndexRetriever.tests/unit/test_github_query_engine.py (2)
3-3: Updated import statement aligns with the newVectorIndexRetrieverusage.
15-15: Updated assertion correctly checks for the instance ofVectorIndexRetriever.tests/unit/test_notion_query_engine.py (2)
3-3: Updated import statement aligns with the newVectorIndexRetrieverusage.
15-15: Updated assertion correctly checks for the instance ofVectorIndexRetriever.
getting it back from main branch but just a couple of renames
qdrant engine had different property from pg engine we had earlier!
Summary by CodeRabbit
New Features
MediaWikiQueryEnginefor querying MediaWiki sources.qdrantservice with health check in test environment.Bug Fixes
Tests
MediaWikiQueryEngine.VectorIndexRetriever.Dependencies
tc-hivemind-backendto version1.2.0.