-Vectors, the output data format of Neural Network models, can effectively encode information and serve a pivotal role in AI applications such as knowledge base, semantic search, Retrieval Augmented Generation (RAG) and more.
+Vectors, the output data format of Neural Network models, can effectively encode information and serve a pivotal role in AI applications such as knowledge base, semantic search, Retrieval Augmented Generation (RAG) and more.
-Milvus is an open-source vector database that suits AI applications of every size from running a demo chatbot in Jupyter notebook to building web-scale search that serves billions of users. In this guide, we will walk you through how to set up Milvus locally within minutes and use the Python client library to generate, store and search vectors.
+Milvus is an open-source vector database that suits AI applications of every size from running a demo chatbot in Jupyter notebook to building web-scale search that serves billions of users. In this guide, we will walk you through how to set up Milvus locally within minutes and use the Python client library to generate, store and search vectors.
## Install Milvus
+
In this guide we use Milvus Lite, a python library included in `pymilvus` that can be embedded into the client application. Milvus also supports deployment on [Docker](https://milvus.io/docs/install_standalone-docker.md) and [Kubernetes](https://milvus.io/docs/install_cluster-milvusoperator.md) for production use cases.
Before starting, make sure you have Python 3.8+ available in the local environment. Install `pymilvus` which contains both the python client library and Milvus Lite:
-
```python
$ pip install -U pymilvus
```
+
> If you are using Google Colab, to enable dependencies just installed, you may need to **restart the runtime**. (Click on the "Runtime" menu at the top of the screen, and select "Restart session" from the dropdown menu).
@@ -28,8 +29,8 @@ $ pip install -U pymilvus
## Set Up Vector Database
-To create a local Milvus vector database, simply instantiate a `MilvusClient` by specifying a file name to store all data, such as "milvus_demo.db".
+To create a local Milvus vector database, simply instantiate a `MilvusClient` by specifying a file name to store all data, such as "milvus_demo.db".
```python
from pymilvus import MilvusClient
@@ -38,8 +39,8 @@ client = MilvusClient("milvus_demo.db")
```
## Create a Collection
-In Milvus, we need a collection to store vectors and their associated metadata. You can think of it as a table in traditional SQL databases. When creating a collection, you can define schema and index params to configure vector specs such as dimensionality, index types and distant metrics. There are also complex concepts to optimize the index for vector search performance. For now, let's just focus on the basics and use default for everything possible. At minimum, you only need to set the collection name and the dimension of the vector field of the collection.
+In Milvus, we need a collection to store vectors and their associated metadata. You can think of it as a table in traditional SQL databases. When creating a collection, you can define schema and index params to configure vector specs such as dimensionality, index types and distant metrics. There are also complex concepts to optimize the index for vector search performance. For now, let's just focus on the basics and use default for everything possible. At minimum, you only need to set the collection name and the dimension of the vector field of the collection.
```python
if client.has_collection(collection_name="demo_collection"):
@@ -50,25 +51,26 @@ client.create_collection(
)
```
-In the above setup,
+In the above setup,
+
- The primary key and vector fields use their default names ("id" and "vector").
- The metric type (vector distance definition) is set to its default value ([COSINE](https://milvus.io/docs/metric.md#Cosine-Similarity)).
- The primary key field accepts integers and does not automatically increments (namely not using [auto-id feature](https://milvus.io/docs/schema.md))
-Alternatively, you can formally define the schema of the collection by following this [instruction](https://milvus.io/api-reference/pymilvus/v2.4.x/MilvusClient/Collections/create_schema.md).
+ Alternatively, you can formally define the schema of the collection by following this [instruction](https://milvus.io/api-reference/pymilvus/v2.4.x/MilvusClient/Collections/create_schema.md).
## Prepare Data
+
In this guide, we use vectors to perform semantic search on text. We need to generate vectors for text by downloading embedding models. This can be easily done by using the utility functions from `pymilvus[model]` library.
## Represent text with vectors
-First, install the model library. This package includes essential ML tools such as PyTorch. The package download may take some time if your local environment has never installed PyTorch.
+First, install the model library. This package includes essential ML tools such as PyTorch. The package download may take some time if your local environment has never installed PyTorch.
```python
$ pip install "pymilvus[model]"
```
-Generate vector embeddings with default model. Milvus expects data to be inserted organized as a list of dictionaries, where each dictionary represents a data record, termed as an entity.
-
+Generate vector embeddings with default model. Milvus expects data to be inserted organized as a list of dictionaries, where each dictionary represents a data record, termed as an entity.
```python
from pymilvus import model
@@ -95,21 +97,21 @@ print("Dim:", embedding_fn.dim, vectors[0].shape) # Dim: 768 (768,)
# to demo metadata filtering later.
data = [
{"id": i, "vector": vectors[i], "text": docs[i], "subject": "history"}
- for i in range(len(vectors))
]
print("Data has", len(data), "entities, each with fields: ", data[0].keys())
print("Vector dim:", len(data[0]["vector"]))
```
- Dim: 768 (768,)
- Data has 3 entities, each with fields: dict_keys(['id', 'vector', 'text', 'subject'])
- Vector dim: 768
-
+```
+Dim: 768 (768,)
+Data has 3 entities, each with fields: dict_keys(['id', 'vector', 'text', 'subject'])
+Vector dim: 768
+```
## [Alternatively] Use fake representation with random vectors
-If you couldn't download the model due to network issues, as a walkaround, you can use random vectors to represent the text and still finish the example. Just note that the search result won't reflect semantic similarity as the vectors are fake ones.
+If you couldn't download the model due to network issues, as a walkaround, you can use random vectors to represent the text and still finish the example. Just note that the search result won't reflect semantic similarity as the vectors are fake ones.
```python
import random
@@ -131,13 +133,14 @@ print("Data has", len(data), "entities, each with fields: ", data[0].keys())
print("Vector dim:", len(data[0]["vector"]))
```
- Data has 3 entities, each with fields: dict_keys(['id', 'vector', 'text', 'subject'])
- Vector dim: 768
-
+```
+Data has 3 entities, each with fields: dict_keys(['id', 'vector', 'text', 'subject'])
+Vector dim: 768
+```
## Insert Data
-Let's insert the data into the collection:
+Let's insert the data into the collection:
```python
res = client.insert(collection_name="demo_collection", data=data)
@@ -145,15 +148,17 @@ res = client.insert(collection_name="demo_collection", data=data)
print(res)
```
- {'insert_count': 3, 'ids': [0, 1, 2], 'cost': 0}
-
+```
+{'insert_count': 3, 'ids': [0, 1, 2], 'cost': 0}
+```
## Semantic Search
+
Now we can do semantic searches by representing the search query text as vector, and conduct vector similarity search on Milvus.
### Vector search
-Milvus accepts one or multiple vector search requests at the same time. The value of the query_vectors variable is a list of vectors, where each vector is an array of float numbers.
+Milvus accepts one or multiple vector search requests at the same time. The value of the query_vectors variable is a list of vectors, where each vector is an array of float numbers.
```python
query_vectors = embedding_fn.encode_queries(["Who is Alan Turing?"])
@@ -170,14 +175,15 @@ res = client.search(
print(res)
```
- data: ["[{'id': 2, 'distance': 0.5859944820404053, 'entity': {'text': 'Born in Maida Vale, London, Turing was raised in southern England.', 'subject': 'history'}}, {'id': 1, 'distance': 0.5118255615234375, 'entity': {'text': 'Alan Turing was the first person to conduct substantial research in AI.', 'subject': 'history'}}]"] , extra_info: {'cost': 0}
-
+```
+data: ["[{'id': 2, 'distance': 0.5859944820404053, 'entity': {'text': 'Born in Maida Vale, London, Turing was raised in southern England.', 'subject': 'history'}}, {'id': 1, 'distance': 0.5118255615234375, 'entity': {'text': 'Alan Turing was the first person to conduct substantial research in AI.', 'subject': 'history'}}]"] , extra_info: {'cost': 0}
+```
The output is a list of results, each mapping to a vector search query. Each query contains a list of results, where each result contains the entity primary key, the distance to the query vector, and the entity details with specified `output_fields`.
## Vector Search with Metadata Filtering
-You can also conduct vector search while considering the values of the metadata (called "scalar" fields in Milvus, as scalar refers to non-vector data). This is done with a filter expression specifying certain criteria. Let's see how to search and filter with the `subject` field in the following example.
+You can also conduct vector search while considering the values of the metadata (called "scalar" fields in Milvus, as scalar refers to non-vector data). This is done with a filter expression specifying certain criteria. Let's see how to search and filter with the `subject` field in the following example.
```python
# Insert more docs in another subject.
@@ -206,19 +212,20 @@ res = client.search(
print(res)
```
- data: ["[{'id': 4, 'distance': 0.27030569314956665, 'entity': {'text': 'Computational synthesis with AI algorithms predicts molecular properties.', 'subject': 'biology'}}, {'id': 3, 'distance': 0.16425910592079163, 'entity': {'text': 'Machine learning has been used for drug design.', 'subject': 'biology'}}]"] , extra_info: {'cost': 0}
-
+```
+data: ["[{'id': 4, 'distance': 0.27030569314956665, 'entity': {'text': 'Computational synthesis with AI algorithms predicts molecular properties.', 'subject': 'biology'}}, {'id': 3, 'distance': 0.16425910592079163, 'entity': {'text': 'Machine learning has been used for drug design.', 'subject': 'biology'}}]"] , extra_info: {'cost': 0}
+```
-By default, the scalar fields are not indexed. If you need to perform metadata filtered search in large dataset, you can consider using fixed schema and also turn on the [index](https://milvus.io/docs/scalar_index.md) to improve the search performance.
+By default, the scalar fields are not indexed. If you need to perform metadata filtered search in large dataset, you can consider using fixed schema and also turn on the [index](https://milvus.io/docs/scalar_index.md) to improve the search performance.
In addition to vector search, you can also perform other types of searches:
### Query
+
A query() is an operation that retrieves all entities matching a cretria, such as a [filter expression](https://milvus.io/docs/boolean.md) or matching some ids.
For example, retrieving all entities whose scalar field has a particular value:
-
```python
res = client.query(
collection_name="demo_collection",
@@ -229,7 +236,6 @@ res = client.query(
Directly retrieve entities by primary key:
-
```python
res = client.query(
collection_name="demo_collection",
@@ -239,8 +245,8 @@ res = client.query(
```
## Delete Entities
-If you'd like to purge data, you can delete entities specifying the primary key or delete all entities matching a particular filter expression.
+If you'd like to purge data, you can delete entities specifying the primary key or delete all entities matching a particular filter expression.
```python
# Delete entities by primary key
@@ -257,13 +263,14 @@ res = client.delete(
print(res)
```
- [0, 2]
- [3, 4, 5]
-
+```
+[0, 2]
+[3, 4, 5]
+```
## Load Existing Data
-Since all data of Milvus Lite is stored in a local file, you can load all data into memory even after the program terminates, by creating a `MilvusClient` with the existing file. For example, this will recover the collections from "milvus_demo.db" file and continue to write data into it.
+Since all data of Milvus Lite is stored in a local file, you can load all data into memory even after the program terminates, by creating a `MilvusClient` with the existing file. For example, this will recover the collections from "milvus_demo.db" file and continue to write data into it.
```python
from pymilvus import MilvusClient
@@ -272,8 +279,8 @@ client = MilvusClient("milvus_demo.db")
```
## Drop the collection
-If you would like to delete all the data in a collection, you can drop the collection with
+If you would like to delete all the data in a collection, you can drop the collection with
```python
# Drop collection
@@ -281,8 +288,8 @@ client.drop_collection(collection_name="demo_collection")
```
## Learn More
-Milvus Lite is great for getting started with a local python program. If you have large scale data or would like to use Milvus in production, you can learn about deploying Milvus on [Docker](https://milvus.io/docs/install_standalone-docker.md) and [Kubernetes](https://milvus.io/docs/install_cluster-milvusoperator.md). All deployment modes of Milvus share the same API, so your client side code doesn't need to change much if moving to another deployment mode. Simply specify the [URI and Token](https://milvus.io/api-reference/pymilvus/v2.4.x/MilvusClient/Client/MilvusClient.md) of a Milvus server deployed anywhere:
+Milvus Lite is great for getting started with a local python program. If you have large scale data or would like to use Milvus in production, you can learn about deploying Milvus on [Docker](https://milvus.io/docs/install_standalone-docker.md) and [Kubernetes](https://milvus.io/docs/install_cluster-milvusoperator.md). All deployment modes of Milvus share the same API, so your client side code doesn't need to change much if moving to another deployment mode. Simply specify the [URI and Token](https://milvus.io/api-reference/pymilvus/v2.4.x/MilvusClient/Client/MilvusClient.md) of a Milvus server deployed anywhere:
```python
client = MilvusClient(uri="http://localhost:19530", token="root:Milvus")
diff --git a/site/en/getstarted/run-milvus-gpu/install_cluster-helm-gpu.md b/site/en/getstarted/run-milvus-gpu/install_cluster-helm-gpu.md
index cb3b1f28b..0024338d8 100644
--- a/site/en/getstarted/run-milvus-gpu/install_cluster-helm-gpu.md
+++ b/site/en/getstarted/run-milvus-gpu/install_cluster-helm-gpu.md
@@ -174,7 +174,7 @@ In addition to a single GPU device, you can also assign multiple GPU devices to
- The release name should only contain letters, numbers and dashes. Dots are not allowed in the release name.
- The default command line installs cluster version of Milvus while installing Milvus with Helm. Further setting is needed while installing Milvus standalone. -
- According to the deprecated API migration guide of Kuberenetes, the policy/v1beta1 API version of PodDisruptionBudget is not longer served as of v1.25. You are suggested to migrate manifests and API clients to use the policy/v1 API version instead.
As a workaround for users who still use the policy/v1beta1 API version of PodDisruptionBudget on Kuberenetes v1.25 and later, you can instead run the following command to install Milvus:
+ - According to the deprecated API migration guide of Kuberenetes, the policy/v1beta1 API version of PodDisruptionBudget is not longer served as of v1.25. You are suggested to migrate manifests and API clients to use the policy/v1 API version instead.
As a workaround for users who still use the policy/v1beta1 API version of PodDisruptionBudget on Kuberenetes v1.25 and later, you can instead run the following command to install Milvus:
helm install my-release milvus/milvus --set pulsar.bookkeeper.pdb.usePolicy=false,pulsar.broker.pdb.usePolicy=false,pulsar.proxy.pdb.usePolicy=false,pulsar.zookeeper.pdb.usePolicy=false - See Milvus Helm Chart and Helm for more information.
- The release name should only contain letters, numbers and dashes. Dots are not allowed in the release name.
- The default command line installs cluster version of Milvus while installing Milvus with Helm. Further setting is needed while installing Milvus standalone. -
- According to the deprecated API migration guide of Kubernetes, the policy/v1beta1 API version of PodDisruptionBudget is no longer served as of v1.25. You are suggested to migrate manifests and API clients to use the policy/v1 API version instead.
As a workaround for users who still use the policy/v1beta1 API version of PodDisruptionBudget on Kubernetes v1.25 and later, you can instead run the following command to install Milvus:
+ - According to the deprecated API migration guide of Kubernetes, the policy/v1beta1 API version of PodDisruptionBudget is no longer served as of v1.25. You are suggested to migrate manifests and API clients to use the policy/v1 API version instead.
As a workaround for users who still use the policy/v1beta1 API version of PodDisruptionBudget on Kubernetes v1.25 and later, you can instead run the following command to install Milvus:
helm install my-release milvus/milvus --set pulsar.bookkeeper.pdb.usePolicy=false,pulsar.broker.pdb.usePolicy=false,pulsar.proxy.pdb.usePolicy=false,pulsar.zookeeper.pdb.usePolicy=false - See Milvus Helm Chart and Helm for more information.
- 
-
+
- Install Milvus
+
+ Install Milvus
Learn how to install Milvus using either Docker Compose or on Kubernetes.
- 
-
+
- Quick Start
+
+ Quick Start
Learn how to quickly run Milvus with sample code.
- 
-
+
-
- Bootcamp
+
+ Bootcamp
Learn how to build vector similarity search applications with Milvus.
diff --git a/site/en/migrate/es2m.md b/site/en/migrate/es2m.md
index 88317c25e..e521541a7 100644
--- a/site/en/migrate/es2m.md
+++ b/site/en/migrate/es2m.md
@@ -118,7 +118,7 @@ The following table describes the parameters in the example config file. For a f
| Parameter | Description |
| --- | --- |
- | `target.mode` | Storage location for dumped files. Valid values:
- `local`: Store dumped files on local disks.
- `remote`: Store dumped files on object storage. |
+ | `target.mode` | Storage location for dumped files. Valid values:
- `local`: Store dumped files on local disks.
- `remote`: Store dumped files on object storage. |
| `target.remote.outputDir` | Output directory path in the cloud storage bucket. |
| `target.remote.cloud` | Cloud storage service provider. Example values: `aws`, `gcp`, `azure`. |
| `target.remote.region` | Cloud storage region. It can be any value if you use local MinIO. |
diff --git a/site/en/migrate/f2m.md b/site/en/migrate/f2m.md
index 637d7123c..77534510d 100644
--- a/site/en/migrate/f2m.md
+++ b/site/en/migrate/f2m.md
@@ -87,7 +87,7 @@ The following table describes the parameters in the example config file. For a f
| Parameter | Description |
| --- | --- |
- | `source.mode` | Specifies where the source files are read from. Valid values:
- `local`: reads files from a local disk.
- `remote`: reads files from remote storage. |
+ | `source.mode` | Specifies where the source files are read from. Valid values:
- `local`: reads files from a local disk.
- `remote`: reads files from remote storage. |
| `source.local.faissFile` | The directory path where the source files are located. For example, `/db/faiss.index`. |
- `target`
@@ -98,7 +98,7 @@ The following table describes the parameters in the example config file. For a f
| `target.create.collection.shardsNums` | Number of shards to be created in the collection. For more information on shards, refer to [Terminology](https://milvus.io/docs/glossary.md#Shard). |
| `target.create.collection.dim` | Dimension of the vector field. |
| `target.create.collection.metricType` | Metric type used to measure similarities between vectors. For more information, refer to [Terminology](https://milvus.io/docs/glossary.md#Metric-type). |
- | `target.mode` | Storage location for dumped files. Valid values:
- `local`: Store dumped files on local disks.
- `remote`: Store dumped files on object storage. |
+ | `target.mode` | Storage location for dumped files. Valid values:
- `local`: Store dumped files on local disks.
- `remote`: Store dumped files on object storage. |
| `target.remote.outputDir` | Output directory path in the cloud storage bucket. |
| `target.remote.cloud` | Cloud storage service provider. Example values: `aws`, `gcp`, `azure`. |
| `target.remote.endpoint` | Endpoint of Milvus 2.x storage. |
diff --git a/site/en/migrate/m2m.md b/site/en/migrate/m2m.md
index c01a32535..caddd36d1 100644
--- a/site/en/migrate/m2m.md
+++ b/site/en/migrate/m2m.md
@@ -114,14 +114,14 @@ The following table describes the parameters in the example config file. For a f
| Parameter | Description |
| --- | --- |
- | `source.mode` | Specifies where the source files are read from. Valid values:
- `local`: reads files from a local disk.
- `remote`: reads files from remote storage. |
+ | `source.mode` | Specifies where the source files are read from. Valid values:
- `local`: reads files from a local disk.
- `remote`: reads files from remote storage. |
| `source.local.tablesDir` | The directory path where the source files are located. For example, `/db/tables/`. |
- `target`
| Parameter | Description |
| --- | --- |
- | `target.mode` | Storage location for dumped files. Valid values:
- `local`: Store dumped files on local disks.
- `remote`: Store dumped files on object storage. |
+ | `target.mode` | Storage location for dumped files. Valid values:
- `local`: Store dumped files on local disks.
- `remote`: Store dumped files on object storage. |
| `target.remote.outputDir` | Output directory path in the cloud storage bucket. |
| `target.remote.ak` | Access key for Milvus 2.x storage. |
| `target.remote.sk` | Secret key for Milvus 2.x storage. |
diff --git a/site/en/reference/benchmark.md b/site/en/reference/benchmark.md
index e381ae55c..821113554 100644
--- a/site/en/reference/benchmark.md
+++ b/site/en/reference/benchmark.md
@@ -11,10 +11,10 @@ This report shows the major test results of Milvus 2.2.0. It aims to provide a p
- 
+
+
+
We have recently run a benchmark against Milvus 2.2.3 and have the following key findings:
- A 2.5x reduction in search latency @@ -33,7 +33,6 @@ This report shows the major test results of Milvus 2.2.0. It aims to provide a p - QPS increases linearly when expanding CPU cores from 8 to 32. - QPS increases linearly when expanding Querynode replicas from 1 to 8. - ## Terminology
metric_type: The metric type applied to this operation. This should be the same as the one used when you index the vector field specified above. Possible values are L2, IP, COSINE, JACCARD, HAMMING.params: Additional parameters. For details, refer to search().metric_type: The metric type applied to this operation. This should be the same as the one used when you index the vector field specified above. Possible values are L2, IP, COSINE, JACCARD, HAMMING.params: Additional parameters. For details, refer to search().metric_type: The metric type applied to this operation. This should be the same as the one used when you index the vector field specified above. Possible values are L2, IP, COSINE, JACCARD, HAMMING.params: Additional parameters. For details, refer to search_iterator().metric_type: The metric type applied to this operation. This should be the same as the one used when you index the vector field specified above. Possible values are L2, IP, COSINE, JACCARD, HAMMING.params: Additional parameters. For details, refer to search_iterator().
@@ -84,15 +83,15 @@ All tests are performed under the following environments.
### Software environment
-| Software | Version |
-| ------------- | --------------------------------------------------------------------- |
-| Milvus | v2.2.0 |
-| Milvus GO SDK | v2.2.0 |
+| Software | Version |
+| ------------- | ------- |
+| Milvus | v2.2.0 |
+| Milvus GO SDK | v2.2.0 |
### Deployment scheme
- Milvus instances (standalone or cluster) are deployed via [Helm](https://milvus.io/docs/install_standalone-helm.md) on a Kubernetes cluster based on physical or virtual machines.
-- Different tests merely vary in the number of CPU cores, the size of memory, and the number of replicas (worker nodes), which only applies to Milvus clusters.
+- Different tests merely vary in the number of CPU cores, the size of memory, and the number of replicas (worker nodes), which only applies to Milvus clusters.
- Unspecified configurations are identical to [default configurations](https://github.com/milvus-io/milvus-helm/blob/master/charts/milvus/values.yaml).
- Milvus dependencies (MinIO, Pulsar and Etcd) store data on the local SSD in each node.
- Search requests are sent to the Milvus instances via [Milvus GO SDK](https://github.com/milvus-io/milvus-sdk-go/tree/master/tests).
@@ -112,31 +111,29 @@ The test uses the open-source dataset SIFT (128 dimensions) from [ANN-Benchmarks
## Test results
-### Milvus 2.2.0 v.s. Milvus 2.1.0
+### Milvus 2.2.0 v.s. Milvus 2.1.0
#### Cluster
**Search performance**
| Milvus | QPS | RT(TP99) / ms | RT(TP50) / ms | fail/s |
-| ------ |------ |---------------|---------------|--------|
+| ------ | ----- | ------------- | ------------- | ------ |
| 2.1.0 | 6904 | 59 | 28 | 0 |
| 2.2.0 | 10248 | 63 | 24 | 0 |
@@ -146,27 +143,25 @@ queryNode:
**Search performance**
-| Milvus | QPS | RT(TP99) / ms | RT(TP50) / ms | fail/s |
-|------ |------|--------------- |---------------|--------|
-| 2.1.0 | 4287 | 104 | 76 | 0 |
-| 2.2.0 | 7522 | 127 | 79 | 0 |
+| Milvus | QPS | RT(TP99) / ms | RT(TP50) / ms | fail/s |
+| ------ | ---- | ------------- | ------------- | ------ |
+| 2.1.0 | 4287 | 104 | 76 | 0 |
+| 2.2.0 | 7522 | 127 | 79 | 0 |
@@ -176,29 +171,27 @@ Expand the CPU cores in one Querynode to check the capability to scale up.
**Search Performance**
-| CPU cores | Concurrent Number | QPS | RT(TP99) / ms | RT(TP50) / ms | fail/s |
-| ------|------|------|---------------|---------------|--------|
-| 8 | 500 | 7153 | 127 | 83 | 0 |
-| 12 | 300 | 10248 | 63 | 24 | 0 |
-| 16 | 600 | 14135 | 85 | 42 | 0 |
-| 32 | 600 | 20281 | 63 | 28 | 0 |
+| CPU cores | Concurrent Number | QPS | RT(TP99) / ms | RT(TP50) / ms | fail/s |
+| --------- | ----------------- | ----- | ------------- | ------------- | ------ |
+| 8 | 500 | 7153 | 127 | 83 | 0 |
+| 12 | 300 | 10248 | 63 | 24 | 0 |
+| 16 | 600 | 14135 | 85 | 42 | 0 |
+| 32 | 600 | 20281 | 63 | 28 | 0 |
@@ -214,28 +207,25 @@ Note: the number of Querynodes equals the `replica_number` when loading the coll
-
-| Replicas | Concurrent Number | QPS | RT(TP99) / ms | RT(TP50) / ms | fail/s |
-|------|------|------|---------------|---------------|--------|
-| 1 | 500 | 7153 | 127 | 83 | 0 |
-| 2 | 500 | 15903 | 105 | 27 | 0 |
-| 4 | 800 | 19281 | 109 | 40 | 0 |
-| 8 | 1200 | 30655 | 93 | 38 | 0 |
+| Replicas | Concurrent Number | QPS | RT(TP99) / ms | RT(TP50) / ms | fail/s |
+| -------- | ----------------- | ----- | ------------- | ------------- | ------ |
+| 1 | 500 | 7153 | 127 | 83 | 0 |
+| 2 | 500 | 15903 | 105 | 27 | 0 |
+| 4 | 800 | 19281 | 109 | 40 | 0 |
+| 8 | 1200 | 30655 | 93 | 38 | 0 |
diff --git a/site/en/reference/disk_index.md b/site/en/reference/disk_index.md
index 6e1aa1a36..746c10762 100644
--- a/site/en/reference/disk_index.md
+++ b/site/en/reference/disk_index.md
@@ -61,10 +61,10 @@ DiskIndex:
| Parameter | Description | Value Range | Default Value |
| --- | --- | --- | --- |
-| `MaxDegree` | Maximum degree of the Vamana graph.
A larger value offers a higher recall rate but increases the size of and time to build the index. | [1, 512] | 56 | -| `SearchListSize` | Size of the candidate list.
A larger value increases the time spent on building the index but offers a higher recall rate.
Set it to a value smaller than `MaxDegree` unless you need to reduce the index-building time. | [1, int32_max] | 100 | -| `PQCodeBugetGBRatio` | Size limit on the PQ code.
A larger value offers a higher recall rate but increases memory usage. | (0.0, 0.25] | 0.125 | -| `SearchCacheBudgetGBRatio` | Ratio of cached node numbers to raw data.
A larger value improves index-building performance with increased memory usage. | [0.0, 0.3) | 0.10 | +| `MaxDegree` | Maximum degree of the Vamana graph.
A larger value offers a higher recall rate but increases the size of and time to build the index. | [1, 512] | 56 | +| `SearchListSize` | Size of the candidate list.
A larger value increases the time spent on building the index but offers a higher recall rate.
Set it to a value smaller than `MaxDegree` unless you need to reduce the index-building time. | [1, int32_max] | 100 | +| `PQCodeBugetGBRatio` | Size limit on the PQ code.
A larger value offers a higher recall rate but increases memory usage. | (0.0, 0.25] | 0.125 | +| `SearchCacheBudgetGBRatio` | Ratio of cached node numbers to raw data.
A larger value improves index-building performance with increased memory usage. | [0.0, 0.3) | 0.10 | | `BeamWidthRatio` | Ratio between the maximum number of IO requests per search iteration and CPU number. | [1, max(128 / CPU number, 16)] | 4.0 | ## Troubleshooting diff --git a/site/en/userGuide/manage-collections.md b/site/en/userGuide/manage-collections.md index 6af8c27c2..54414853a 100644 --- a/site/en/userGuide/manage-collections.md +++ b/site/en/userGuide/manage-collections.md @@ -335,11 +335,11 @@ export fields='[{ \
Determines if the primary field automatically increments.
Setting this to True makes the primary field automatically increment. In this case, the primary field should not be included in the data to insert to avoid errors. The auto-generated IDs have a fixed length and cannot be altered.
+ Determines if the primary field automatically increments.
Setting this to True makes the primary field automatically increment. In this case, the primary field should not be included in the data to insert to avoid errors. The auto-generated IDs have a fixed length and cannot be altered.
Determines if Milvus saves the values of undefined fields in a dynamic field if the data being inserted into the target collection includes fields that are not defined in the collection's schema.
When you set this to True, Milvus will create a field called $meta to store any undefined fields and their values from the data that is inserted.
+ Determines if Milvus saves the values of undefined fields in a dynamic field if the data being inserted into the target collection includes fields that are not defined in the collection's schema.
When you set this to True, Milvus will create a field called $meta to store any undefined fields and their values from the data that is inserted.
Whether the current field is the primary field in a collection.
Each collection has only one primary field. A primary field should be of either the DataType.INT64 type or the DataType.VARCHAR type.
+ Whether the current field is the primary field in a collection.
Each collection has only one primary field. A primary field should be of either the DataType.INT64 type or the DataType.VARCHAR type.
The dimension of the vector embeddings.
This is mandatory for a field of the DataType.FLOAT_VECTOR, DataType.BINARY_VECTOR, DataType.FLOAT16_VECTOR, or DataType.BFLOAT16_VECTOR type. If you use DataType.SPARSE_FLOAT_VECTOR, omit this parameter.
+ The dimension of the vector embeddings.
This is mandatory for a field of the DataType.FLOAT_VECTOR, DataType.BINARY_VECTOR, DataType.FLOAT16_VECTOR, or DataType.BFLOAT16_VECTOR type. If you use DataType.SPARSE_FLOAT_VECTOR, omit this parameter.
@@ -378,15 +378,15 @@ export fields='[{ \
Whether the current field is the primary field in a collection.
Each collection has only one primary field. A primary field should be of either the DataType.Int64 type or the DataType.VarChar type.
+ Whether the current field is the primary field in a collection.
Each collection has only one primary field. A primary field should be of either the DataType.Int64 type or the DataType.VarChar type.
Whether allows the primary field to automatically increment.
Setting this to true makes the primary field automatically increment. In this case, the primary field should not be included in the data to insert to avoid errors.
+ Whether allows the primary field to automatically increment.
Setting this to true makes the primary field automatically increment. In this case, the primary field should not be included in the data to insert to avoid errors.
The dimension of the vector embeddings.
This is mandatory for a field of the DataType.FloatVector, DataType.BinaryVector, DataType.Float16Vector, or DataType.BFloat16Vector type.
+ The dimension of the vector embeddings.
This is mandatory for a field of the DataType.FloatVector, DataType.BinaryVector, DataType.Float16Vector, or DataType.BFloat16Vector type.
@@ -409,15 +409,15 @@ export fields='[{ \
Whether the current field is the primary field in a collection.
Each collection has only one primary field. A primary field should be of either the DataType.INT64 type or the DataType.VARCHAR type.
+ Whether the current field is the primary field in a collection.
Each collection has only one primary field. A primary field should be of either the DataType.INT64 type or the DataType.VARCHAR type.
Whether the primary field automatically increments upon data insertions into this collection.
The value defaults to False. Setting this to True makes the primary field automatically increment. Skip this parameter if you need to set up a collection with a customized schema.
+ Whether the primary field automatically increments upon data insertions into this collection.
The value defaults to False. Setting this to True makes the primary field automatically increment. Skip this parameter if you need to set up a collection with a customized schema.
The dimensionality of the collection field that holds vector embeddings.
The value should be an integer greater than 1 and is usually determined by the model you use to generate vector embeddings.
+ The dimensionality of the collection field that holds vector embeddings.
The value should be an integer greater than 1 and is usually determined by the model you use to generate vector embeddings.
@@ -980,11 +980,11 @@ Use [createCollection()](https://milvus.io/api-reference/node/v2.4.x/Collections
The schema of this collection.
Setting this to None indicates this collection will be created with default settings.
To set up a collection with a customized schema, you need to create a CollectionSchema object and reference it here. In this case, Milvus ignores all other schema-related settings carried in the request.
+ The schema of this collection.
Setting this to None indicates this collection will be created with default settings.
To set up a collection with a customized schema, you need to create a CollectionSchema object and reference it here. In this case, Milvus ignores all other schema-related settings carried in the request.
The parameters for building the index on the vector field in this collection. To set up a collection with a customized schema and automatically load the collection to memory, you need to create an IndexParams object and reference it here.
You should at least add an index for the vector field in this collection. You can also skip this parameter if you prefer to set up the index parameters later on.
+ The parameters for building the index on the vector field in this collection. To set up a collection with a customized schema and automatically load the collection to memory, you need to create an IndexParams object and reference it here.
You should at least add an index for the vector field in this collection. You can also skip this parameter if you prefer to set up the index parameters later on.
@@ -1003,7 +1003,7 @@ Use [createCollection()](https://milvus.io/api-reference/node/v2.4.x/Collections
The schema of this collection.
Leaving it empty indicates this collection will be created with default settings. To set up a collection with a customized schema, you need to create a CollectionSchema object and reference it here.
+ The schema of this collection.
Leaving it empty indicates this collection will be created with default settings. To set up a collection with a customized schema, you need to create a CollectionSchema object and reference it here.
response = client.createDatabase(createDatabaseParam);
```
```javascript
-const address = "http://localhost:19530"
+const address = "http://localhost:19530";
// 1. Set up a Milvus Client
-client = new MilvusClient({address});
+client = new MilvusClient({ address });
// 3. Create a database
res = await client.createDatabase({
- db_name: "my_database"
-})
+ db_name: "my_database",
+});
-console.log(res)
+console.log(res);
// {
// error_code: 'Success',
@@ -115,10 +115,10 @@ db.using_database("my_database")
```javascript
// 4. Activate another database
res = await client.useDatabase({
- db_name: "my_database"
-})
+ db_name: "my_database",
+});
-console.log(res)
+console.log(res);
```
You can also set a database to use upon connecting to your Milvus cluster as follows:
@@ -152,7 +152,7 @@ const address = "http://localhost:19530";
const db_name = "my_database";
// 1. Set up a Milvus Client
-client = new MilvusClient({address, db_name});
+client = new MilvusClient({ address, db_name });
```
## List databases
@@ -205,9 +205,9 @@ System.out.println(listDatabasesResponse.getData());
```
```javascript
-res = await client.listDatabases()
+res = await client.listDatabases();
-console.log(res.db_names)
+console.log(res.db_names);
// [ 'default', 'my_database' ]
```
@@ -261,8 +261,8 @@ response = client.dropDatabase(dropDatabaseParam);
```javascript
res = await client.dropDatabase({
- db_name: "my_database"
-})
+ db_name: "my_database",
+});
```
## Use RBAC with database
@@ -328,14 +328,14 @@ public class ConnectToMilvus {
const address = "http://localhost:19530";
const token = "root:Milvus";
-function connectToMilvus(dbName="default") {
- const client = new MilvusClient({
- address,
- token,
- dbName
- });
+function connectToMilvus(dbName = "default") {
+ const client = new MilvusClient({
+ address,
+ token,
+ dbName,
+ });
- return client;
+ return client;
}
```
@@ -347,116 +347,114 @@ function connectToMilvus(dbName="default") {
Node.js
- ```python
- _ROLE_NAME = "test_role"
- _PRIVILEGE_INSERT = "Insert"
-
- connect_to_milvus()
- role = Role(_ROLE_NAME)
- role.create()
-
- connect_to_milvus()
- role.grant("Collection", "*", _PRIVILEGE_INSERT)
- print(role.list_grants())
- print(role.list_grant("Collection", "*"))
- role.revoke("Global", "*", _PRIVILEGE_INSERT)
- ```
-
- ```java
- String ROLE_NAME = "test_role";
- String PRIVILEGE_INSERT = "Insert";
-
- MilvusServiceClient client = new ConnectToMilvus().build();
- R response = client.createRole(CreateRoleParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .build());
-
- if (response.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(response.getMessage());
- }
-
- response = client.grantRolePrivilege(GrantRolePriviledgeParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .withObject("Collection")
- .withObjectName("*")
- .withPrivilege(PRIVILEGE_INSERT)
- .build());
-
- if (response.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(response.getMessage());
- }
-
- R grants = client.selectGrantForRole(SelectGrantForRoleParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .build());
-
- if (grants.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(grants.getMessage());
- }
-
- System.out.println(grants.getData());
-
- grants = client.selectGrantForRoleAndObject(SelectGrantForRoleAndObjectParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .withObject("Collection")
- .withObjectName("*")
- .build());
-
- if (grants.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(grants.getMessage());
- }
-
- System.out.println(grants.getData());
-
- response = client.revokeRolePrivilege(RevokeRolePrivilegeParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .withObject("Global")
- .withObjectName("*")
- .withPrivilege(PRIVILEGE_INSERT)
- .build());
-
- if (response.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(response.getMessage());
- }
-
- response = client.revokeRolePrivilege(RevokeRolePrivilegeParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .withObject("Global")
- .withObjectName("*")
- .withPrivilege(PRIVILEGE_INSERT)
- .build());
-
- if (response.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(response.getMessage());
- }
- ```
-
- ```javascript
- const ROLE_NAME = "test_role";
- const PRIVILEGE_INSERT = "Insert";
-
- const client = connectToMilvus();
-
- async function demo() {
-
- }
- await client.createRole({
- roleName: ROLE_NAME
- })
-
- const grants = await client.listGrants({
- roleName: ROLE_NAME
- })
-
- console.log(grants.grants);
-
- await client.revokePrivilege({
- roleName: ROLE_NAME,
- object: "Global",
- objectName: "*",
- privilege: PRIVILEGE_INSERT
- })
- ```
+ ```python
+ _ROLE_NAME = "test_role"
+ _PRIVILEGE_INSERT = "Insert"
+
+ connect_to_milvus()
+ role = Role(_ROLE_NAME)
+ role.create()
+
+ connect_to_milvus()
+ role.grant("Collection", "*", _PRIVILEGE_INSERT)
+ print(role.list_grants())
+ print(role.list_grant("Collection", "*"))
+ role.revoke("Global", "*", _PRIVILEGE_INSERT)
+ ```
+
+ ```java
+ String ROLE_NAME = "test_role";
+ String PRIVILEGE_INSERT = "Insert";
+
+ MilvusServiceClient client = new ConnectToMilvus().build();
+ R response = client.createRole(CreateRoleParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .build());
+
+ if (response.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(response.getMessage());
+ }
+
+ response = client.grantRolePrivilege(GrantRolePriviledgeParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .withObject("Collection")
+ .withObjectName("*")
+ .withPrivilege(PRIVILEGE_INSERT)
+ .build());
+
+ if (response.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(response.getMessage());
+ }
+
+ R grants = client.selectGrantForRole(SelectGrantForRoleParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .build());
+
+ if (grants.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(grants.getMessage());
+ }
+
+ System.out.println(grants.getData());
+
+ grants = client.selectGrantForRoleAndObject(SelectGrantForRoleAndObjectParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .withObject("Collection")
+ .withObjectName("*")
+ .build());
+
+ if (grants.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(grants.getMessage());
+ }
+
+ System.out.println(grants.getData());
+
+ response = client.revokeRolePrivilege(RevokeRolePrivilegeParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .withObject("Global")
+ .withObjectName("*")
+ .withPrivilege(PRIVILEGE_INSERT)
+ .build());
+
+ if (response.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(response.getMessage());
+ }
+
+ response = client.revokeRolePrivilege(RevokeRolePrivilegeParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .withObject("Global")
+ .withObjectName("*")
+ .withPrivilege(PRIVILEGE_INSERT)
+ .build());
+
+ if (response.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(response.getMessage());
+ }
+ ```
+
+ ```javascript
+ const ROLE_NAME = "test_role";
+ const PRIVILEGE_INSERT = "Insert";
+
+ const client = connectToMilvus();
+
+ async function demo() {}
+ await client.createRole({
+ roleName: ROLE_NAME,
+ });
+
+ const grants = await client.listGrants({
+ roleName: ROLE_NAME,
+ });
+
+ console.log(grants.grants);
+
+ await client.revokePrivilege({
+ roleName: ROLE_NAME,
+ object: "Global",
+ objectName: "*",
+ privilege: PRIVILEGE_INSERT,
+ });
+ ```
- If a Milvus connection specifies a `db_name`, but a Permission API call afterward does not, **database** refers to the database whose name was specified in the Milvus connection.
@@ -466,92 +464,90 @@ function connectToMilvus(dbName="default") {
Node.js
- ```python
- # NOTE: please make sure the 'foo' db has been created
- connect_to_milvus(db_name="foo")
-
- # This role will have the insert permission of all collections under foo db,
- # excluding the insert permissions of collections under other dbs
- role.grant("Collection", "*", _PRIVILEGE_INSERT)
- print(role.list_grants())
- print(role.list_grant("Collection", "*"))
- role.revoke("Global", "*", _PRIVILEGE_INSERT)
- ```
-
- ```java
- // NOTE: please make sure the 'foo' db has been created
- MilvusServiceClient client = new ConnectToMilvus().withDbName("foo").build();
-
- // This role will have the insert permission of all collections under foo db,
- // excluding the insert permissions of collections under other dbs
- R response = client.grantRolePrivilege(GrantRolePriviledgeParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .withObject("Collection")
- .withObjectName("*")
- .withPrivilege(PRIVILEGE_INSERT)
- .build());
-
- if (response.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(response.getMessage());
- }
-
- R grants = client.selectGrantForRole(SelectGrantForRoleParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .build());
-
- if (grants.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(grants.getMessage());
- }
-
- System.out.println(grants.getData());
-
- grants = client.selectGrantForRoleAndObject(SelectGrantForRoleAndObjectParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .withObject("Collection")
- .withObjectName("*")
- .build());
-
- if (grants.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(grants.getMessage());
- }
-
- System.out.println(grants.getData());
-
- response = client.revokeRolePrivilege(RevokeRolePrivilegeParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .withObject("Global")
- .withObjectName("*")
- .withPrivilege(PRIVILEGE_INSERT)
- .build());
-
- if (response.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(response.getMessage());
- }
- ```
-
- ```javascript
- const client = connectToMilvus("foo");
-
- async function demo() {
-
- }
- await client.createRole({
- roleName: ROLE_NAME
- })
-
- const grants = await client.listGrants({
- roleName: ROLE_NAME
- })
-
- console.log(grants.grants);
-
- await client.revokePrivilege({
- roleName: ROLE_NAME,
- object: "Global",
- objectName: "*",
- privilege: PRIVILEGE_INSERT
- })
- ```
+ ```python
+ # NOTE: please make sure the 'foo' db has been created
+ connect_to_milvus(db_name="foo")
+
+ # This role will have the insert permission of all collections under foo db,
+ # excluding the insert permissions of collections under other dbs
+ role.grant("Collection", "*", _PRIVILEGE_INSERT)
+ print(role.list_grants())
+ print(role.list_grant("Collection", "*"))
+ role.revoke("Global", "*", _PRIVILEGE_INSERT)
+ ```
+
+ ```java
+ // NOTE: please make sure the 'foo' db has been created
+ MilvusServiceClient client = new ConnectToMilvus().withDbName("foo").build();
+
+ // This role will have the insert permission of all collections under foo db,
+ // excluding the insert permissions of collections under other dbs
+ R response = client.grantRolePrivilege(GrantRolePriviledgeParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .withObject("Collection")
+ .withObjectName("*")
+ .withPrivilege(PRIVILEGE_INSERT)
+ .build());
+
+ if (response.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(response.getMessage());
+ }
+
+ R grants = client.selectGrantForRole(SelectGrantForRoleParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .build());
+
+ if (grants.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(grants.getMessage());
+ }
+
+ System.out.println(grants.getData());
+
+ grants = client.selectGrantForRoleAndObject(SelectGrantForRoleAndObjectParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .withObject("Collection")
+ .withObjectName("*")
+ .build());
+
+ if (grants.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(grants.getMessage());
+ }
+
+ System.out.println(grants.getData());
+
+ response = client.revokeRolePrivilege(RevokeRolePrivilegeParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .withObject("Global")
+ .withObjectName("*")
+ .withPrivilege(PRIVILEGE_INSERT)
+ .build());
+
+ if (response.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(response.getMessage());
+ }
+ ```
+
+ ```javascript
+ const client = connectToMilvus("foo");
+
+ async function demo() {}
+ await client.createRole({
+ roleName: ROLE_NAME,
+ });
+
+ const grants = await client.listGrants({
+ roleName: ROLE_NAME,
+ });
+
+ console.log(grants.grants);
+
+ await client.revokePrivilege({
+ roleName: ROLE_NAME,
+ object: "Global",
+ objectName: "*",
+ privilege: PRIVILEGE_INSERT,
+ });
+ ```
- If a Permission API call is made upon a Milvus connection, with or without `db_name` specified, **database** refers to the database whose name was specified in the Permission API call.
@@ -561,75 +557,75 @@ function connectToMilvus(dbName="default") {
Node.js
- ```python
- # NOTE: please make sure the 'foo' db has been created
-
- db_name = "foo"
- connect_to_milvus()
- role.grant("Collection", "*", _PRIVILEGE_INSERT, db_name=db_name)
- print(role.list_grants(db_name=db_name))
- print(role.list_grant("Collection", "*", db_name=db_name))
- role.revoke("Global", "*", _PRIVILEGE_INSERT, db_name=db_name)
- ```
-
- ```java
- // NOTE: please make sure the 'foo' db has been created
-
- String dbName = "foo";
- MilvusServiceClient client = new ConnectToMilvus().build();
-
- R response = client.grantRolePrivilege(GrantRolePriviledgeParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .withObject("Collection")
- .withObjectName("*")
- .withPrivilege(PRIVILEGE_INSERT)
- .withDatabaseName(dbName)
- .build());
-
- if (response.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(response.getMessage());
- }
-
- R grants = client.selectGrantForRole(SelectGrantForRoleParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .withDatabaseName(dbName)
- .build());
-
- if (grants.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(grants.getMessage());
- }
-
- System.out.println(grants.getData());
-
- grants = client.selectGrantForRoleAndObject(SelectGrantForRoleAndObjectParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .withObject("Collection")
- .withObjectName("*")
- .withDatabaseName(dbName)
- .build());
-
- if (grants.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(grants.getMessage());
- }
-
- System.out.println(grants.getData());
-
- response = client.revokeRolePrivilege(RevokeRolePrivilegeParam.newBuilder()
- .withRoleName(ROLE_NAME)
- .withObject("Global")
- .withObjectName("*")
- .withPrivilege(PRIVILEGE_INSERT)
- .withDatabaseName(dbName)
- .build());
-
- if (response.getStatus() != R.Status.Success.getCode()) {
- throw new RuntimeException(response.getMessage());
- }
- ```
-
- ```javascript
- // The Node.js SDK currently cannot support this case.
- ```
+ ```python
+ # NOTE: please make sure the 'foo' db has been created
+
+ db_name = "foo"
+ connect_to_milvus()
+ role.grant("Collection", "*", _PRIVILEGE_INSERT, db_name=db_name)
+ print(role.list_grants(db_name=db_name))
+ print(role.list_grant("Collection", "*", db_name=db_name))
+ role.revoke("Global", "*", _PRIVILEGE_INSERT, db_name=db_name)
+ ```
+
+ ```java
+ // NOTE: please make sure the 'foo' db has been created
+
+ String dbName = "foo";
+ MilvusServiceClient client = new ConnectToMilvus().build();
+
+ R response = client.grantRolePrivilege(GrantRolePriviledgeParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .withObject("Collection")
+ .withObjectName("*")
+ .withPrivilege(PRIVILEGE_INSERT)
+ .withDatabaseName(dbName)
+ .build());
+
+ if (response.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(response.getMessage());
+ }
+
+ R grants = client.selectGrantForRole(SelectGrantForRoleParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .withDatabaseName(dbName)
+ .build());
+
+ if (grants.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(grants.getMessage());
+ }
+
+ System.out.println(grants.getData());
+
+ grants = client.selectGrantForRoleAndObject(SelectGrantForRoleAndObjectParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .withObject("Collection")
+ .withObjectName("*")
+ .withDatabaseName(dbName)
+ .build());
+
+ if (grants.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(grants.getMessage());
+ }
+
+ System.out.println(grants.getData());
+
+ response = client.revokeRolePrivilege(RevokeRolePrivilegeParam.newBuilder()
+ .withRoleName(ROLE_NAME)
+ .withObject("Global")
+ .withObjectName("*")
+ .withPrivilege(PRIVILEGE_INSERT)
+ .withDatabaseName(dbName)
+ .build());
+
+ if (response.getStatus() != R.Status.Success.getCode()) {
+ throw new RuntimeException(response.getMessage());
+ }
+ ```
+
+ ```javascript
+ // The Node.js SDK currently cannot support this case.
+ ```
## What's next
diff --git a/site/en/userGuide/search-query-get/single-vector-search.md b/site/en/userGuide/search-query-get/single-vector-search.md
index fccbd81d1..96e1ecd2a 100644
--- a/site/en/userGuide/search-query-get/single-vector-search.md
+++ b/site/en/userGuide/search-query-get/single-vector-search.md
@@ -478,15 +478,15 @@ console.log(res.results)
A list of vector embeddings.
Milvus searches for the most similar vector embeddings to the specified ones.
+ A list of vector embeddings.
Milvus searches for the most similar vector embeddings to the specified ones.
The total number of entities to return.
You can use this parameter in combination with offset in param to enable pagination.
The sum of this value and offset in param should be less than 16,384.
+ The total number of entities to return.
You can use this parameter in combination with offset in param to enable pagination.
The sum of this value and offset in param should be less than 16,384.
The parameter settings specific to this operation.
+ The parameter settings specific to this operation.
@@ -505,11 +505,11 @@ console.log(res.results)
A list of vector embeddings.
Milvus searches for the most similar vector embeddings to the specified ones.
+ A list of vector embeddings.
Milvus searches for the most similar vector embeddings to the specified ones.
The number of records to return in the search result. This parameter uses the same syntax as the limit parameter, so you should only set one of them.
You can use this parameter in combination with offset in param to enable pagination.
The sum of this value and offset in param should be less than 16,384.
+ The number of records to return in the search result. This parameter uses the same syntax as the limit parameter, so you should only set one of them.
You can use this parameter in combination with offset in param to enable pagination.
The sum of this value and offset in param should be less than 16,384.
@@ -528,11 +528,11 @@ console.log(res.results)
A list of vector embeddings.
Milvus searches for the most similar vector embeddings to the specified ones.
+ A list of vector embeddings.
Milvus searches for the most similar vector embeddings to the specified ones.
The total number of entities to return.
You can use this parameter in combination with offset in param to enable pagination.
The sum of this value and offset in param should be less than 16,384.
+ The total number of entities to return.
You can use this parameter in combination with offset in param to enable pagination.
The sum of this value and offset in param should be less than 16,384.
diff --git a/site/en/userGuide/search-query-get/with-iterators.md b/site/en/userGuide/search-query-get/with-iterators.md
index e32d3eba3..cdcb1d8b6 100644
--- a/site/en/userGuide/search-query-get/with-iterators.md
+++ b/site/en/userGuide/search-query-get/with-iterators.md
@@ -362,7 +362,7 @@ System.out.println(results.size());
A list of vector embeddings.
Milvus searches for the most similar vector embeddings to the specified ones.
+ A list of vector embeddings.
Milvus searches for the most similar vector embeddings to the specified ones.
The number of entities to return each time you call
The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
+ The number of entities to return each time you call
The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
The parameter settings specific to this operation.
+ The parameter settings specific to this operation.
A list of field names to include in each entity in return.
The value defaults to None. If left unspecified, only the primary field is included.
+ A list of field names to include in each entity in return.
The value defaults to None. If left unspecified, only the primary field is included.
The total number of entities to return.
The value defaults to -1, indicating all matching entities will be in return.
+ The total number of entities to return.
The value defaults to -1, indicating all matching entities will be in return.
@@ -409,7 +409,7 @@ System.out.println(results.size());
The number of entities to return each time you call
The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
+ The number of entities to return each time you call
The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
The number of entities to return each time you call
The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
+ The number of entities to return each time you call
The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
A scalar filtering condition to filter matching entities.
The value defaults to None, indicating that scalar filtering is ignored. To build a scalar filtering condition, refer to Boolean Expression Rules.
+ A scalar filtering condition to filter matching entities.
The value defaults to None, indicating that scalar filtering is ignored. To build a scalar filtering condition, refer to Boolean Expression Rules.
A list of field names to include in each entity in return.
The value defaults to None. If left unspecified, only the primary field is included.
+ A list of field names to include in each entity in return.
The value defaults to None. If left unspecified, only the primary field is included.
The total number of entities to return.
The value defaults to -1, indicating all matching entities will be in return.
+ The total number of entities to return.
The value defaults to -1, indicating all matching entities will be in return.
@@ -586,7 +586,7 @@ while (true) {
The number of entities to return each time you call
The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
+ The number of entities to return each time you call
The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
Server configurations (cluster)
- -```yaml -queryNode: - replicas: 1 - resources: - limits: - cpu: "12.0" - memory: 8Gi - requests: - cpu: "12.0" - memory: 8Gi -``` - + ```yaml + queryNode: + replicas: 1 + resources: + limits: + cpu: "12.0" + memory: 8Gi + requests: + cpu: "12.0" + memory: 8Gi + ```Server configurations (standalone)
- -```yaml -standalone: - replicas: 1 - resources: - limits: - cpu: "12.0" - memory: 16Gi - requests: - cpu: "12.0" - memory: 16Gi -``` - + ```yaml + standalone: + replicas: 1 + resources: + limits: + cpu: "12.0" + memory: 16Gi + requests: + cpu: "12.0" + memory: 16Gi + ```Server configurations (cluster)
- - ```yaml -queryNode: - replicas: 1 - resources: - limits: - cpu: "8.0" /"12.0" /"16.0" /"32.0" - memory: 8Gi - requests: - cpu: "8.0" /"12.0" /"16.0" /"32.0" - memory: 8Gi -``` - + ```yaml + queryNode: + replicas: 1 + resources: + limits: + cpu: "8.0" /"12.0" /"16.0" /"32.0" + memory: 8Gi + requests: + cpu: "8.0" /"12.0" /"16.0" /"32.0" + memory: 8Gi + ```Server configurations (cluster)
- -```yaml -queryNode: - replicas: 1 / 2 / 4 / 8 - resources: - limits: - cpu: "8.0" - memory: 8Gi - requests: - cpu: "8.0" - memory: 8Gi -``` - + ```yaml + queryNode: + replicas: 1 / 2 / 4 / 8 + resources: + limits: + cpu: "8.0" + memory: 8Gi + requests: + cpu: "8.0" + memory: 8Gi + ```A larger value offers a higher recall rate but increases the size of and time to build the index. | [1, 512] | 56 | -| `SearchListSize` | Size of the candidate list.
A larger value increases the time spent on building the index but offers a higher recall rate.
Set it to a value smaller than `MaxDegree` unless you need to reduce the index-building time. | [1, int32_max] | 100 | -| `PQCodeBugetGBRatio` | Size limit on the PQ code.
A larger value offers a higher recall rate but increases memory usage. | (0.0, 0.25] | 0.125 | -| `SearchCacheBudgetGBRatio` | Ratio of cached node numbers to raw data.
A larger value improves index-building performance with increased memory usage. | [0.0, 0.3) | 0.10 | +| `MaxDegree` | Maximum degree of the Vamana graph.
A larger value offers a higher recall rate but increases the size of and time to build the index. | [1, 512] | 56 | +| `SearchListSize` | Size of the candidate list.
A larger value increases the time spent on building the index but offers a higher recall rate.
Set it to a value smaller than `MaxDegree` unless you need to reduce the index-building time. | [1, int32_max] | 100 | +| `PQCodeBugetGBRatio` | Size limit on the PQ code.
A larger value offers a higher recall rate but increases memory usage. | (0.0, 0.25] | 0.125 | +| `SearchCacheBudgetGBRatio` | Ratio of cached node numbers to raw data.
A larger value improves index-building performance with increased memory usage. | [0.0, 0.3) | 0.10 | | `BeamWidthRatio` | Ratio between the maximum number of IO requests per search iteration and CPU number. | [1, max(128 / CPU number, 16)] | 4.0 | ## Troubleshooting diff --git a/site/en/userGuide/manage-collections.md b/site/en/userGuide/manage-collections.md index 6af8c27c2..54414853a 100644 --- a/site/en/userGuide/manage-collections.md +++ b/site/en/userGuide/manage-collections.md @@ -335,11 +335,11 @@ export fields='[{ \
auto_idSetting this to True makes the primary field automatically increment. In this case, the primary field should not be included in the data to insert to avoid errors. The auto-generated IDs have a fixed length and cannot be altered.
Setting this to True makes the primary field automatically increment. In this case, the primary field should not be included in the data to insert to avoid errors. The auto-generated IDs have a fixed length and cannot be altered.
enable_dynamic_fieldWhen you set this to True, Milvus will create a field called $meta to store any undefined fields and their values from the data that is inserted.
When you set this to True, Milvus will create a field called $meta to store any undefined fields and their values from the data that is inserted.
field_nameis_primaryEach collection has only one primary field. A primary field should be of either the DataType.INT64 type or the DataType.VARCHAR type.
Each collection has only one primary field. A primary field should be of either the DataType.INT64 type or the DataType.VARCHAR type.
dimThis is mandatory for a field of the DataType.FLOAT_VECTOR, DataType.BINARY_VECTOR, DataType.FLOAT16_VECTOR, or DataType.BFLOAT16_VECTOR type. If you use DataType.SPARSE_FLOAT_VECTOR, omit this parameter.
This is mandatory for a field of the DataType.FLOAT_VECTOR, DataType.BINARY_VECTOR, DataType.FLOAT16_VECTOR, or DataType.BFLOAT16_VECTOR type. If you use DataType.SPARSE_FLOAT_VECTOR, omit this parameter.
isPrimaryKeyEach collection has only one primary field. A primary field should be of either the DataType.Int64 type or the DataType.VarChar type.
Each collection has only one primary field. A primary field should be of either the DataType.Int64 type or the DataType.VarChar type.
autoIDSetting this to true makes the primary field automatically increment. In this case, the primary field should not be included in the data to insert to avoid errors.
Setting this to true makes the primary field automatically increment. In this case, the primary field should not be included in the data to insert to avoid errors.
dimensionThis is mandatory for a field of the DataType.FloatVector, DataType.BinaryVector, DataType.Float16Vector, or DataType.BFloat16Vector type.
This is mandatory for a field of the DataType.FloatVector, DataType.BinaryVector, DataType.Float16Vector, or DataType.BFloat16Vector type.
is_primary_keyEach collection has only one primary field. A primary field should be of either the DataType.INT64 type or the DataType.VARCHAR type.
Each collection has only one primary field. A primary field should be of either the DataType.INT64 type or the DataType.VARCHAR type.
auto_idThe value defaults to False. Setting this to True makes the primary field automatically increment. Skip this parameter if you need to set up a collection with a customized schema.
The value defaults to False. Setting this to True makes the primary field automatically increment. Skip this parameter if you need to set up a collection with a customized schema.
dimThe value should be an integer greater than 1 and is usually determined by the model you use to generate vector embeddings.
The value should be an integer greater than 1 and is usually determined by the model you use to generate vector embeddings.
schemaSetting this to None indicates this collection will be created with default settings.
To set up a collection with a customized schema, you need to create a CollectionSchema object and reference it here. In this case, Milvus ignores all other schema-related settings carried in the request.
Setting this to None indicates this collection will be created with default settings.
To set up a collection with a customized schema, you need to create a CollectionSchema object and reference it here. In this case, Milvus ignores all other schema-related settings carried in the request.
index_paramsYou should at least add an index for the vector field in this collection. You can also skip this parameter if you prefer to set up the index parameters later on.
You should at least add an index for the vector field in this collection. You can also skip this parameter if you prefer to set up the index parameters later on.
collectionSchemaLeaving it empty indicates this collection will be created with default settings. To set up a collection with a customized schema, you need to create a CollectionSchema object and reference it here.
Leaving it empty indicates this collection will be created with default settings. To set up a collection with a customized schema, you need to create a CollectionSchema object and reference it here.
indexParamsdataMilvus searches for the most similar vector embeddings to the specified ones.
Milvus searches for the most similar vector embeddings to the specified ones.
limitYou can use this parameter in combination with offset in param to enable pagination.
The sum of this value and offset in param should be less than 16,384.
You can use this parameter in combination with offset in param to enable pagination.
The sum of this value and offset in param should be less than 16,384.
search_paramsdataMilvus searches for the most similar vector embeddings to the specified ones.
Milvus searches for the most similar vector embeddings to the specified ones.
topKYou can use this parameter in combination with offset in param to enable pagination.
The sum of this value and offset in param should be less than 16,384.
You can use this parameter in combination with offset in param to enable pagination.
The sum of this value and offset in param should be less than 16,384.
dataMilvus searches for the most similar vector embeddings to the specified ones.
Milvus searches for the most similar vector embeddings to the specified ones.
limitYou can use this parameter in combination with offset in param to enable pagination.
The sum of this value and offset in param should be less than 16,384.
You can use this parameter in combination with offset in param to enable pagination.
The sum of this value and offset in param should be less than 16,384.
dataMilvus searches for the most similar vector embeddings to the specified ones.
Milvus searches for the most similar vector embeddings to the specified ones.
anns_fieldbatch_sizenext() on the current iterator.The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
next() on the current iterator.The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
paramoutput_fieldsThe value defaults to None. If left unspecified, only the primary field is included.
The value defaults to None. If left unspecified, only the primary field is included.
limitThe value defaults to -1, indicating all matching entities will be in return.
The value defaults to -1, indicating all matching entities will be in return.
withBatchSizenext() on the current iterator.The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
next() on the current iterator.The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
withParamsbatch_sizenext() on the current iterator.The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
next() on the current iterator.The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
exprThe value defaults to None, indicating that scalar filtering is ignored. To build a scalar filtering condition, refer to Boolean Expression Rules.
The value defaults to None, indicating that scalar filtering is ignored. To build a scalar filtering condition, refer to Boolean Expression Rules.
output_fieldsThe value defaults to None. If left unspecified, only the primary field is included.
The value defaults to None. If left unspecified, only the primary field is included.
limitThe value defaults to -1, indicating all matching entities will be in return.
The value defaults to -1, indicating all matching entities will be in return.
withBatchSizenext() on the current iterator.The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
next() on the current iterator.The value defaults to 1000. Set it to a proper value to control the number of entities to return per iteration.
addOutField