Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cherry picking some milvus improvements. Updated milvus client to include backoff strategy on ingestion. Added extra configurations to ease remote benchmark run. #170

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Cherry picking some milvus improvements. Updated milvus client to include backoff strategy on ingestion. Added extra configurations to ease remote benchmark run. #170

wants to merge 4 commits into from

Conversation

filipecosta90
Copy link
Contributor

Fixes #54
Furthermore, it also eases the process of running the milvus benchmark against remote hosts (like zilliz cloud deployments, or other SW deployments ).

This has been used on our benchmarks, and was part of a bigger work in https://github.com/redis-performance/vector-db-benchmark.

Example of running targeting a zilliz cloud cluster:

MILVUS_USER="db_admin" \
MILVUS_PASS="<...>" \
MILVUS_PORT=<zilliz port> \
          python3 run.py  --engines milvus-m-* --datasets gist-960-euclidean \
                          --host https://<endpoint info>.zillizcloud.com:<zilliz port>

@filipecosta90 filipecosta90 changed the title Cherry picking some milvus improvemens. Updated milvus client to include backoff strategy on ingestion. Added extra configurations to ease remote benchmark run. Cherry picking some milvus improvements. Updated milvus client to include backoff strategy on ingestion. Added extra configurations to ease remote benchmark run. Jul 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

backoff strategy should be used for rate-limited errors on milvus or reducing batch_size config
1 participant
@filipecosta90