Skip to content

Commit

Permalink
docs: readme
Browse files Browse the repository at this point in the history
  • Loading branch information
LongxingTan committed Jul 3, 2024
1 parent 6cf22aa commit 02ca40d
Showing 1 changed file with 21 additions and 1 deletion.
22 changes: 21 additions & 1 deletion examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
- [RAG with Langchain](./rag_langchain_demo.py)


## Retrieval
## Embedding

**Data Format**
```
Expand Down Expand Up @@ -88,6 +88,26 @@ torchrun --nproc_per_node 1 \
```


## Retrieval

```shell
QUERY_ENCODE_DIR=nq-queries
OUT_DIR=temp
MODEL_DIR="BAAI/bge-base-zh-v1.5"
QUERY=nq-test-queries.json
mkdir $QUERY_ENCODE_DIR

python -m retrievals.pipelines.embed \
--model_name_or_path $MODEL_DIR \
--output_dir $OUT_DIR \
--do_encode \
--fp16 \
--per_device_eval_batch_size 256 \
--train_data $QUERY \
--is_query true
```


## Rerank

**Cross encoder reranking**
Expand Down

0 comments on commit 02ca40d

Please sign in to comment.