Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Functionality to help users migrate data from PGVector to AlloyDB #235

Open
wants to merge 50 commits into
base: main
Choose a base branch
from

Conversation

twishabansal
Copy link
Contributor

Will add a migration guide once the code is pushed.

@product-auto-label product-auto-label bot added the api: alloydb Issues related to the googleapis/langchain-google-alloydb-pg-python API. label Sep 18, 2024
@twishabansal twishabansal changed the title Feat: Functionality to help users migrate data from PGVector to AlloyDB easily Feat: Functionality to help users migrate data from PGVector to AlloyDB Sep 18, 2024
@twishabansal twishabansal force-pushed the pg_migration_methods branch 3 times, most recently from 321b298 to 0b80185 Compare September 18, 2024 17:53
src/langchain_google_alloydb_pg/pg_to_alloy_migrator.py Outdated Show resolved Hide resolved
src/langchain_google_alloydb_pg/pg_to_alloy_migrator.py Outdated Show resolved Hide resolved
src/langchain_google_alloydb_pg/pg_to_alloy_migrator.py Outdated Show resolved Hide resolved
src/langchain_google_alloydb_pg/pg_to_alloy_migrator.py Outdated Show resolved Hide resolved
src/langchain_google_alloydb_pg/pg_to_alloy_migrator.py Outdated Show resolved Hide resolved
src/langchain_google_alloydb_pg/pg_to_alloy_migrator.py Outdated Show resolved Hide resolved
src/langchain_google_alloydb_pg/pg_to_alloy_migrator.py Outdated Show resolved Hide resolved
src/langchain_google_alloydb_pg/pg_to_alloy_migrator.py Outdated Show resolved Hide resolved
src/langchain_google_alloydb_pg/pg_to_alloy_migrator.py Outdated Show resolved Hide resolved
src/langchain_google_alloydb_pg/pg_to_alloy_migrator.py Outdated Show resolved Hide resolved
@averikitsch
Copy link
Collaborator

It may be easier to discuss moving forward once we outline the documentation and different user journeys.

src/langchain_google_alloydb_pg/__init__.py Outdated Show resolved Hide resolved
tests/util_tests/test_pgvector_migrator.py Show resolved Hide resolved

@pytest.fixture(scope="module")
def sample_embeddings(self) -> List[float]:
return [0.1] * (VECTOR_SIZE - 1) + [0.2]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: you could use DeterministicFakeEmbedding(size=VECTOR_SIZE).embed_query("test") to generate an embedding or you could use DeterministicFakeEmbedding(size=VECTOR_SIZE) and the vector store to add embeddings to the table

tests/util_tests/test_pgvector_migrator.py Outdated Show resolved Hide resolved
tests/util_tests/test_pgvector_migrator.py Outdated Show resolved Hide resolved
src/langchain_google_alloydb_pg/utils/pgvector_migrator.py Outdated Show resolved Hide resolved
src/langchain_google_alloydb_pg/pg_to_alloy_migrator.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@averikitsch averikitsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM! Great job on the updates.
I am going to ask Kurtis if he would like to be a final asyncio review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: alloydb Issues related to the googleapis/langchain-google-alloydb-pg-python API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants