added Sectional Description (lancedb#206)

* assests and app name * update README * demo gifs * talk with github codespaces * talk with github codespaces * gitignore * linted * added version * link fix * added local llm tag * crag * link fix * lint * llm tags * non-clickable badge * non-clickable badge * fix * tutorial llm tags * added instructions and fix * colab fix * fix * formatted * hybrid search and rag colab * colab format * python test * node test * python test * blog link update * rag mlx * myntra search engine app * link fix * CrewAI Example * lint * node test * node test * node test * added readme * support for Gemini Pro * fix * chunking techniques * lint * Locally RAG from Scratch * lint * llama3 added * link finx * sdk manual cli chatbot phidata * sdk manual cli chatbot phidata * link fix * tags * advanced * update readme * remove key * lint * formatting fixes * lint * updated image * added demo image * change autogen notebook * lint * lint * rag evaluation with ragas * README update * broken link fix * Restructured README * updated titles * restructed README * sectional description * lint
raghavdixit99 · Jun 9, 2024 · 799d3a6 · 799d3a6
1 parent 08734ea
commit 799d3a6
Show file tree

Hide file tree

Showing 2 changed files with 46 additions and 13 deletions.
diff --git a/README.md b/README.md
@@ -29,13 +29,20 @@ Examples are available as:
 
 The following examples are organized into different tables to make similar types of examples easily accessible.
 
+### Build from Scratch
+
+Build applications/examples using LanceDB for efficient vector-based document retrieval.
+
 | Build from Scratch &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; | 
 |-------- | -------------: |
 |||
 | [Build RAG from Scratch](./tutorials/RAG-from-Scratch) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/RAG-from-Scratch/RAG_from_Scratch.ipynb) [![LLM](https://img.shields.io/badge/openai-api-white)](#) [![beginner](https://img.shields.io/badge/beginner-B5FF33)](#)|  |
 | [Local RAG from Scratch with Llama3](./tutorials/Local-RAG-from-Scratch) | [![Python](https://img.shields.io/badge/python-3670A0?style=for-the-badge&logo=python&logoColor=ffdd54)](./tutorials/Local-RAG-from-Scratch/rag.py) [![local LLM](https://img.shields.io/badge/local-llm-green)](#) [![beginner](https://img.shields.io/badge/beginner-B5FF33)](#)|  |
 ||||
 
+### MultiModal
+
+Create a multimodal search application using LanceDB for efficient vector-based retrieval of text and image data. Input text or image queries to find the most relevant documents and images from your corpus.
 
 | Multimodal &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; | Blog |
 | --------- | -------------------------- | ----------- |
@@ -45,7 +52,11 @@ The following examples are organized into different tables to make similar types
 | [Multimodal Image + Text Search](/examples/multimodal_search/) | <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/multimodal_search/main.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> [![Python](https://img.shields.io/badge/python-3670A0?style=for-the-badge&logo=python&logoColor=ffdd54)](./examples/multimodal_search/main.py) [![LLM](https://img.shields.io/badge/local-llm-green)](#)   [![intermediate](https://img.shields.io/badge/intermediate-FFDA33)](#)| [![Ghost](https://img.shields.io/badge/ghost-000?style=for-the-badge&logo=ghost&logoColor=%23F7DF1E)](https://blog.lancedb.com/multi-modal-ai-made-easy-with-lancedb-clip-5aaf8801c939/)|
 ||||
 
-| RAG &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; | Blog |
+### RAG
+
+Develop a Retrieval-Augmented Generation (RAG) application using LanceDB for efficient vector-based information retrieval. Input text queries to retrieve relevant documents and generate comprehensive answers by combining retrieved information.
+
+| RAG &nbsp; &nbsp;| Interactive Notebook & Scripts | Blog |
 | --------- | -------------------------- | ----------- |
 ||||
 | [Improve RAG with Re-ranking](/examples/RAG_Reranking/) | <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/RAG_Reranking/main.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> [![LLM](https://img.shields.io/badge/local-llm-green)](#) [![beginner](https://img.shields.io/badge/beginner-B5FF33)](#)|[![Ghost](https://img.shields.io/badge/ghost-000?style=for-the-badge&logo=ghost&logoColor=%23F7DF1E)](https://blog.lancedb.com/simplest-method-to-improve-rag-pipeline-re-ranking-cf6eaec6d544)|
@@ -61,6 +72,9 @@ The following examples are organized into different tables to make similar types
 | [Agentic RAG ](/tutorials/Agentic_RAG/) | <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/Agentic_RAG/main.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a>  [![LLM](https://img.shields.io/badge/openai-api-white)](#) [![advanced](https://img.shields.io/badge/advanced-FF3333)](#)|
 ||||
 
+### Vector Search
+
+Build a vector search application using LanceDB for efficient vector-based document retrieval. Input text queries to find the most relevant documents from your corpus.
 
 | Vector Search &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; | Blog |
 | --------- | -------------------------- | ----------- |
@@ -78,6 +92,10 @@ The following examples are organized into different tables to make similar types
 | [Accelerate Vector Search Applications Using OpenVINO](/examples/Accelerate-Vector-Search-Applications-Using-OpenVINO/) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/Accelerate-Vector-Search-Applications-Using-OpenVINO/clip_text_image_search.ipynb) [![local LLM](https://img.shields.io/badge/local-llm-green)](#) [![advanced](https://img.shields.io/badge/advanced-FF3333)](#)| [![Ghost](https://img.shields.io/badge/ghost-000?style=for-the-badge&logo=ghost&logoColor=%23F7DF1E)](https://blog.lancedb.com/accelerate-vector-search-applications-using-openvino-lancedb/)|
 ||||
 
+### Chatbot
+
+Create a chatbot application using LanceDB for efficient vector-based response generation. Input user queries to retrieve relevant context and generate coherent, context-aware replies.
+
 | Chatbot &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; | Blog &nbsp;|
 | --------- | -------------------------- | ----------- |
 ||||
@@ -88,13 +106,22 @@ The following examples are organized into different tables to make similar types
 | [Context-Aware Chatbot using Llama 2 & LanceDB](./tutorials/chatbot_using_Llama2_&_lanceDB) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/chatbot_using_Llama2_&_lanceDB/main.ipynb) [![local LLM](https://img.shields.io/badge/local-llm-green)](#) [![advanced](https://img.shields.io/badge/advanced-FF3333)](#)| [![Ghost](https://img.shields.io/badge/ghost-000?style=for-the-badge&logo=ghost&logoColor=%23F7DF1E)](https://blog.lancedb.com/context-aware-chatbot-using-llama-2-lancedb-as-vector-database-4d771d95c755) |
 ||||
 
+
+### Evaluation
+
+Develop an evaluation application. Input reference and candidate texts to measure their performance on various metrics.
+
 | Evaluation &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; | Blog |
 | --------- | -------------------------- | ----------- |
 ||||
 | [Evaluating Prompts with Prompttools](/examples/prompttools-eval-prompts/) | <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/prompttools-eval-prompts/main.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a>   [![LLM](https://img.shields.io/badge/openai-api-white)](#) [![local LLM](https://img.shields.io/badge/local-llm-green)](#) [![advanced](https://img.shields.io/badge/advanced-FF3333)](#)|  |
 | [Evaluating RAG with RAGAs](./examples/Evaluating_RAG_with_RAGAs/) | <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/Evaluating_RAG_with_RAGAs/Evaluating_RAG_with_RAGAs.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a>   [![LLM](https://img.shields.io/badge/openai-api-white)](#) [![intermediate](https://img.shields.io/badge/intermediate-FFDA33)](#)|  |
 ||||
 
+### AI Agents
+
+Design an AI agents coordination application with LanceDB for efficient vector-based communication and collaboration. Input queries to enable AI agents to exchange information, coordinate tasks, and achieve shared goals effectively.
+
 | AI Agents &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; | Blog |
 | --------- | -------------------------- | ----------- |
 ||||
@@ -103,8 +130,11 @@ The following examples are organized into different tables to make similar types
 | [SuperAgent Autogen](/examples/SuperAgent_Autogen) |<a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/SuperAgent_Autogen/main.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> [![LLM](https://img.shields.io/badge/openai-api-white)](#) [![intermediate](https://img.shields.io/badge/intermediate-FFDA33)](#)||
 ||||
 
+### Recommender Systems
+
+Create a recommender system application with LanceDB for efficient vector-based item recommendation. Input user preferences or item features to generate personalized recommendations and enhance user experience.
 
-| Recommender Systems &nbsp; &nbsp;| Interactive Notebook & Scripts &nbsp; | Blog |
+| Recommender Systems | Interactive Notebook & Scripts &nbsp; | Blog |
 | --------- | -------------------------- | ----------- |
 ||||
 | [Movie Recommender](/examples/movie-recommender/) | <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/movie-recommender/main.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> [![Python](https://img.shields.io/badge/python-3670A0?style=for-the-badge&logo=python&logoColor=ffdd54)](./examples/movie-recommender/main.py) [![beginner](https://img.shields.io/badge/beginner-B5FF33)](#)|  |
@@ -113,7 +143,11 @@ The following examples are organized into different tables to make similar types
 | [Arxiv paper recommender](/examples/arxiv-recommender) | <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/arxiv-recommender/main.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> [![Python](https://img.shields.io/badge/python-3670A0?style=for-the-badge&logo=python&logoColor=ffdd54)](./examples/arxiv-recommender/main.py) [![LLM](https://img.shields.io/badge/local-llm-green)](#)  [![beginner](https://img.shields.io/badge/beginner-B5FF33)](#)|  |
 ||||
 
-| Concepts | Interactive Notebook | Blog Link |
+### Concepts
+
+Checkout concepts of LLM applications pipeline to ensures accurate information retrieval.
+
+| Concepts | Interactive Notebook | Blog |
 | --------- | -------------------------- | ----------- |
 |           |                            |             |
 | [A Primer on Text Chunking and its Types](./tutorials/different-types-text-chunking-in-RAG) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/different-types-text-chunking-in-RAG/Text_Chunking_on_RAG_application_with_LanceDB.ipynb) [![beginner](https://img.shields.io/badge/beginner-B5FF33)](#)| [![Ghost](https://img.shields.io/badge/ghost-000?style=for-the-badge&logo=ghost&logoColor=%23F7DF1E)](https://blog.lancedb.com/a-primer-on-text-chunking-and-its-types-a420efc96a13) |

diff --git a/examples/LlamaIndex-demo/lancedb_cloud/main.ipynb b/examples/LlamaIndex-demo/lancedb_cloud/main.ipynb
@@ -113,6 +113,7 @@
     "import openai\n",
     "import logging\n",
     "import sys\n",
+    "\n",
     "# Uncomment to see debug logs\n",
     "# logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)\n",
     "# logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))\n",
@@ -145,7 +146,7 @@
    "source": [
     "! mkdir -p 'data/paul_graham/'\n",
     "! wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'\n",
-    "! ls 'data/paul_graham/'\n"
+    "! ls 'data/paul_graham/'"
    ]
   },
   {
@@ -166,7 +167,7 @@
    "outputs": [],
    "source": [
     "documents = SimpleDirectoryReader(\"data/paul_graham/\").load_data()\n",
-    "print(\"Document ID:\", documents[0].doc_id, \"Document Hash:\", documents[0].hash)\n"
+    "print(\"Document ID:\", documents[0].doc_id, \"Document Hash:\", documents[0].hash)"
    ]
   },
   {
@@ -218,9 +219,7 @@
    "source": [
     "storage_context = StorageContext.from_defaults(vector_store=vector_store)\n",
     "\n",
-    "index = VectorStoreIndex.from_documents(\n",
-    "    documents, storage_context=storage_context\n",
-    ")"
+    "index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)"
    ]
   },
   {
@@ -257,15 +256,15 @@
     "    MetadataFilter,\n",
     ")\n",
     "\n",
-    "date = datetime.today().strftime('%Y-%m-%d')\n",
+    "date = datetime.today().strftime(\"%Y-%m-%d\")\n",
     "query_filters = MetadataFilters(\n",
     "    filters=[\n",
     "        MetadataFilter(\n",
-    "            key=\"creation_date\", operator=FilterOperator.EQ, value=date  #using current date as the latest data is scraped\n",
-    "        ),\n",
-    "        MetadataFilter(\n",
-    "            key=\"file_size\", value=75040, operator=FilterOperator.GT\n",
+    "            key=\"creation_date\",\n",
+    "            operator=FilterOperator.EQ,\n",
+    "            value=date,  # using current date as the latest data is scraped\n",
     "        ),\n",
+    "        MetadataFilter(key=\"file_size\", value=75040, operator=FilterOperator.GT),\n",
     "    ],\n",
     "    condition=FilterCondition.AND,\n",
     ")"